4 ms·

My understanding is that the state before was: Content augmented with JS -- OK to index. Blank page with all content rendered by JS -- not indexed And what th

My understanding is that the state before was: Content augmented with JS -- OK to index. Blank page with all content rendered by JS -- not indexed And what they're saying today is: "we are now going to be able to index single page apps that don't have any server rendered content" Perhaps I've missed something, but this is my interpretation: SPAs are now full citizens in the SEO world.

KMag 12y ago
Google has understood pages that have had only client-side rendered content for years. I worked mostly on JavaScript execution in the indexing system from 2006 until 2010, as well as other "rich content" indexing. Certainly somewhere in the 2008 to 2009 timeframe we saw that the Chinese language version of the Wall Street Journal had a lot of pages in their archive where all of the non-boilerplate content was rendered via JavaScript. Since they didn't do this with their English content, it didn't seem to be an attempt to hide content from search engines, but much more likely a workaround for an older browser that wouldn't properly render Unicode, but with a JavaScript engine that would properly render Unicode. Sometime in the 2008 to 2009 timeframe, Google's indexing system started understanding text that was written into documents from JavaScript body onload handlers, and the Chinese Wall Street Journal archive content was exhibit A in my argument that my changes should be turned on in production. I'm sure they've increased the accuracy of the analysis since, but they've certainly been able to index content written by JavaScript for something like 5 years now. Edit: without giving away any Google secrets, here's a pretty good analysis of my work from 2008: http://moz.com/ugc/new-reality-google-follows-links-in-javascript-4930 http://moz.com/ugc/new-reality-google-follows-links-in-javas... Edit 2: Since the 2008-2009 timeframe, Google also notices when you use JavaScript to change a page's title. I caused a crash in Google's indexing system when I made a bad assumption about Google's HTML parser's handling of XHTML-style empty title tags <title/> and tried to construct negative-length std::strings from them. When your code runs on every single webpage that Google can find, you're certain to hit corner cases you didn't anticipate. I did test for empty <title></title>, but not <title/>, and made incorrect assumptions about the two pointers I'd get to the beginning and end of the title.
- thatthatis 12y ago
  So would the more accurate interpretation be: google is starting to promote that it can and will index javascript single page apps? Thanks for sharing, and nice work btw.
  - KMag 12y ago
    There were a lot of caveats that both reduced fidelity and would have made announcements confusing back when I was at Google. Also, if one mentions a limitation in an announcement, a lot of the Search Engine Optimization community would be citing the announcement for more than 6 months. So, it's difficult and potentially counter-productive to have an announcement with lots of caveats. V8 and Chrome weren't even a glimmer in Google's eye back in 2006, so I hope they've largely replaced the code I was working on with something based on Chrome. As late as 2010, the DOM was a completely custom implementation that looked somewhat like Firefox, but with enough IE features to fool lots of other pages that would otherwise change their content to "You must run IE to view this page". (On a side note, as much as many people would like to see such pages heavily penalized and indexed as if the IE-only message were their only information, some of those pages are unique sources of invaluable information and users wouldn't be well served by such harsh treatment.)
- juretriglav 12y ago
  Thanks for the insight! I know it's very off-topic, but seeing your comment about indexing title tags I have to ask if you have any idea what could be wrong here: http://stackoverflow.com/questions/23732242/how-to-get-google-to-index-a-dynamic-title-in-an-angular-js-app http://stackoverflow.com/questions/23732242/how-to-get-googl... Google seems to be ignoring the title changes our JS makes. Should we not have the title tag there in the first place and then add it in when the title is known?
davemel37 12y ago
Everyone seems to be missing the point of this announcement. They are just saying that they are now adding a feature in Webmaster Tools to show you they see your site so you can diagnose problems.
- thatthatis 12y ago
  Perhaps you haven't been privy to any of the discussions about: "what is the seo implication of SPAs?" Up until now people were pursuing isomorphic JavaScript so they could render server side and client side so the search engines would see their content. This "just a tool" lets webmasters see how google sees their pages, which means SPAs are becoming safe for SEO. That tool plus the song and dance about understanding the modern web is a pretty strong signal in an industry that operates heavily on rumors.

KMag 12y ago

Google has understood pages that have had only client-side rendered content for years. I worked mostly on JavaScript execution in the indexing system from 2006 until 2010, as well as other "rich content" indexing. Certainly somewhere in the 2008 to 2009 timeframe we saw that the Chinese language version of the Wall Street Journal had a lot of pages in their archive where all of the non-boilerplate content was rendered via JavaScript. Since they didn't do this with their English content, it didn't seem to be an attempt to hide content from search engines, but much more likely a workaround for an older browser that wouldn't properly render Unicode, but with a JavaScript engine that would properly render Unicode. Sometime in the 2008 to 2009 timeframe, Google's indexing system started understanding text that was written into documents from JavaScript body onload handlers, and the Chinese Wall Street Journal archive content was exhibit A in my argument that my changes should be turned on in production. I'm sure they've increased the accuracy of the analysis since, but they've certainly been able to index content written by JavaScript for something like 5 years now. Edit: without giving away any Google secrets, here's a pretty good analysis of my work from 2008: http://moz.com/ugc/new-reality-google-follows-links-in-javascript-4930 http://moz.com/ugc/new-reality-google-follows-links-in-javas... Edit 2: Since the 2008-2009 timeframe, Google also notices when you use JavaScript to change a page's title. I caused a crash in Google's indexing system when I made a bad assumption about Google's HTML parser's handling of XHTML-style empty title tags <title/> and tried to construct negative-length std::strings from them. When your code runs on every single webpage that Google can find, you're certain to hit corner cases you didn't anticipate. I did test for empty <title></title>, but not <title/>, and made incorrect assumptions about the two pointers I'd get to the beginning and end of the title.

thatthatis 12y ago
So would the more accurate interpretation be: google is starting to promote that it can and will index javascript single page apps? Thanks for sharing, and nice work btw.
- KMag 12y ago
  There were a lot of caveats that both reduced fidelity and would have made announcements confusing back when I was at Google. Also, if one mentions a limitation in an announcement, a lot of the Search Engine Optimization community would be citing the announcement for more than 6 months. So, it's difficult and potentially counter-productive to have an announcement with lots of caveats. V8 and Chrome weren't even a glimmer in Google's eye back in 2006, so I hope they've largely replaced the code I was working on with something based on Chrome. As late as 2010, the DOM was a completely custom implementation that looked somewhat like Firefox, but with enough IE features to fool lots of other pages that would otherwise change their content to "You must run IE to view this page". (On a side note, as much as many people would like to see such pages heavily penalized and indexed as if the IE-only message were their only information, some of those pages are unique sources of invaluable information and users wouldn't be well served by such harsh treatment.)
juretriglav 12y ago
Thanks for the insight! I know it's very off-topic, but seeing your comment about indexing title tags I have to ask if you have any idea what could be wrong here: http://stackoverflow.com/questions/23732242/how-to-get-google-to-index-a-dynamic-title-in-an-angular-js-app http://stackoverflow.com/questions/23732242/how-to-get-googl... Google seems to be ignoring the title changes our JS makes. Should we not have the title tag there in the first place and then add it in when the title is known?

davemel37 12y ago

Everyone seems to be missing the point of this announcement. They are just saying that they are now adding a feature in Webmaster Tools to show you they see your site so you can diagnose problems.

thatthatis 12y ago
Perhaps you haven't been privy to any of the discussions about: "what is the seo implication of SPAs?" Up until now people were pursuing isomorphic JavaScript so they could render server side and client side so the search engines would see their content. This "just a tool" lets webmasters see how google sees their pages, which means SPAs are becoming safe for SEO. That tool plus the song and dance about understanding the modern web is a pretty strong signal in an industry that operates heavily on rumors.