Google Gets Faster

Recent talks by Jeff Dean of google at Web Search Data Mining '09 (video) and an earlier talk at Univ of Washington present some interesting history on the evolution of the Google search technology.

The most tantalizing aspect of this is the notion of a "fast index", perhaps in-memory on many servers, dedicated to indexing (and computing authority or PageRank) for rapidly moving content like Digg and YouTube video honors. In general, with twitter bubbling, the notion of real time search is focusing the industry on one of Google's key relevance metrics, freshness.

I presented an overview of Off-Page SEO factors at the Atlanta Web Entrepeneurs SEO group this week and Sam Beckett was kind enough to YouTube it:

As I've been compiling my thoughts on this, I created a nifty Prezi with some observations on Jeff Dean's content.

Some 10 years ago, Google had to flip indexes to accomplish updates. This is described as happening on a per machine basis. We've seen increases in the speed of updates but recently the degree to which Google is paying attention to fast moving social media suggests an revolutionary speedup.

Jeff's talks hint at some of the mechanisms.

"Sub second latencies"... new pages are added to the index very rapidly and our experience, and Jeff's dialogue, hints at PageRank calculations at high speed.

The use of in-memory data structures offers some hints at how super rapid rank updates might happen.

Take Away

I'm still pretty early in assessing the impact of a new understanding of the underlying mechanisms, but I'll offer one hypothesis: Google is now capable of detecting the duration a link lives on a "hot list" like Digg's upcoming page. This means a successful social media promotion can have a much greater effect than simple social media participation.

We see the amount of diggs affect how long it takes for a Digg permalink page fall out of the top rankings. It's likely that the anchor text, or title of the Digg, is added to the index record for the page -- So pick your social media link text very carefully. It's also likely the thing that has the biggest impact on the long term effect of social media promotion.

A word from our Sponsor

Need more search engine success? We give you the basics and the hard-hitting science in the Stomping the Search Engines 2 DVD course. I teach the understanding search engines segment and try and walk a fine line between the basics and deep, long term insights.

Get it for just a $1 when you try the Net Effect magazine from StomperNet. It covers traffic, conversion, social media, business building and operations, and more.