3 ms·

There was an article a while back about how Googlebot is likely a version of Chrome that fully renders a page to process it. [1] You're right, without a full br

There was an article a while back about how Googlebot is likely a version of Chrome that fully renders a page to process it. [1] You're right, without a full browser engine, you could pull a whole host of tricks to make the page appear clean but then put in ads later. Since Googlebot is probably a full featured browser, it becomes a lot harder to trick. [1] http://ipullrank.com/googlebot-is-chrome/ http://ipullrank.com/googlebot-is-chrome/

PaulHoule 15y ago
I've seen a lot of evidence that Googlebot evaluates Javascript in a page. For one thing, I've got a site that has Facebook Connect on it, and I see Facebook's bot following on the heels of Googlebot. They work together just like those crabs that work in teams to cut up starfish...
HNatWORK 15y ago
There are definitely at least two different, shall we say classes of crawlers: The Googlebot and the Google Web Preview crawler. I don't know the extent of Googlebot's javascript parsing but the web preview crawler appears to parse javascript like plain webkit. Here's a link where they show the UA for the Google Web Preview crawler: http://www.webmasterworld.com/search_engine_spiders/4353651.htm http://www.webmasterworld.com/search_engine_spiders/4353651.... I believe this crawler renders your pages for the preview snippets you get in search results when you hover over the arrow that appears on the right side of a result. The previews I've seen would only look that way if javascript was being rendered and allowed to run for ~10-20s by my estimation--based on the progress of an animation that was previewed.