Google Update Promises 50% Fresher Results

Dubbed Caffeine, Google's Carrie Grimes unveiled the improved search index yesterday evening. "Caffeine provides 50 percent fresher results for web searches than our last index, and it's the largest collection of web content we've offered," she said.

Grimes explained that with the old index, some layers were refreshed faster than others. This meant that there was quite a substantial delay from when Google found to when we could see it in search results. With Caffeine, Google is constantly examining and updating small sections of the search index. As the company finds new pages or new information on old pages, it adds them straight to the index, meaning fresher search results.

"Caffeine lets us index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second," Carrie writes.

"Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day. You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles."

Grimes failed to mention whether the transition to Caffeine will be a slow roll-out, similar to other products launched by Google, or if we'll all start seeing fresher search results from today onwards.

  • awood28211
    "100 million gigabytes"... There's my next hard drive... :D I wish!
    Reply
  • N.Broekhuijsen
    oooh shiny!!! but where is the money for all the storage going to come from??

    that's a lot of data I wonder how they do the searches so fast if there is 100 million GB of unique data, and it has to find your exact query.

    it cannot possibly all be on a RAM like solution in order to make it fast enough....

    this, (addressing steve jobs) is magical!
    Reply
  • awood28211
    Being simple, the money it's $10,000,000 US if they paid $100 per drive @ 1TB each is some super huge array (hoping I did my math right). Of course that's just drives and $100 is probably conservative but not too far off if they buy it all in bulk. That doesn't include ANYTHING else... Servers, racks, drive-chassis, software, development, physical buildings... even if the rest tripled the initial cost, $30 mil is probably not that hard to swallow for them. How I wish I could say that for my wallet. Of course I could have blown the math (as I'm being lazy right now) and I could be totally wrong.
    Reply
  • el_bastardo74
    very nice...even as sophisticated as the search engines are now, compared to 1998(my first pc), i still get frustrated doing more nuanced, detailed searches
    Reply
  • grieve
    awood28211Being simple, the money it's $10,000,000 US if they paid $100 per drive @ 1TB each is some super huge array (hoping I did my math right). Of course that's just drives and $100 is probably conservative but not too far off if they buy it all in bulk. That doesn't include ANYTHING else... Servers, racks, drive-chassis, software, development, physical buildings... even if the rest tripled the initial cost, $30 mil is probably not that hard to swallow for them. How I wish I could say that for my wallet. Of course I could have blown the math (as I'm being lazy right now) and I could be totally wrong.we just replaced our SAN which is 96 tarabytes = just under $100,000

    I would think Googles storage is around $700 a tarabyte (discounted)
    SOOO 100,000,000 gigs = 97656.25 tarabytes
    =$68,359,375


    if 1 gig = $700, Keep in mind this is not hardware you use @ home... there not hooking up 100,000 caviar black drives.
    Reply
  • killerclick
    We'll maybe read this news in some archive 15 years from now and laugh about the pitiful amount of data Google was working with.
    Reply
  • rizky_pratama
    Just wondering 100 million of data processed or stored that is a different thing.
    Reply
  • killerclick
    grievewe just replaced our SAN which is 96 tarabytes = just under $100,000I would think Googles storage is around $700 a tarabyte (discounted)SOOO 100,000,000 gigs = 97656.25 tarabytes=$68,359,375 if 1 gig = $700, Keep in mind this is not hardware you use @ home... there not hooking up 100,000 caviar black drives.
    What's a tarabyte?
    Reply
  • awood28211
    grievewe just replaced our SAN which is 96 tarabytes = just under $100,000I would think Googles storage is around $700 a tarabyte (discounted)SOOO 100,000,000 gigs = 97656.25 tarabytes=$68,359,375 if 1 gig = $700, Keep in mind this is not hardware you use @ home... there not hooking up 100,000 caviar black drives.
    Wow, what were you thinking? 700 per Terabyte? I am in tech and realize they don't buy SATA drives off the walmart shelf, but seriously the drive alone even at 100% duty cycle quality even if they add fibre channel drives should never cost 700/tb. At my last job, we had a 20 TB total EMC SAN with 15K RPM Fibre channel drives and the drives themselves only cost around 350 a TB and that was 3 years ago pricing. Not to argue, but I think you could have done better on a per TB price. Now, if your whole system came out at 700/TB that makes more sense. Our 20 TB was far more expensive than 7000 dollars in raw drive cost. As stated, there is a lot of infrastructure needed to support those drives. They just don't set around and soak up data...as you know.
    Reply
  • darksun20
    Google makes their own hard drives, that's why the storage space ticker on the g-mail page keeps going up.
    Reply