4 ms·

For JavaScript-heavy sites, I still recommend following Google's AJAX-crawling guidelines, and supporting the pre-rendered view request variable _escaped_fragme

For JavaScript-heavy sites, I still recommend following Google's AJAX-crawling guidelines, and supporting the pre-rendered view request variable _escaped_fragment_ https://developers.google.com/webmasters/ajax-crawling/docs/specification https://developers.google.com/webmasters/ajax-crawling/docs/... You can do a good enough job by sending such requests to a PhantomJS instance, waiting for the page to load, outputting the PhantomJS-rendered HTML to the browser, and saving the HTML to a cache for faster access next time.[1] There are also plenty of SaaS apps that will handle the pre-rendering for you. An additional bonus of doing this is that you can intercept requests from engines and services that don't support _escaped_fragment (e.g. Facebook external hit), and always ensure that you serve pre-rendered HTML to them. (e.g. by matching on the user-agent string). [1a] One potential hitch is knowing when a page has finished rendering. You could potentially set a variable in your code and have PhantomJS wait for that. [1b] One other hitch is that it's possible for PhantomJS to time out e.g. waiting for an external JS library to load. It's sensible to check that your HTML output looks vaguely sane before sending to the browser (e.g. for Angular, make sure there are no {{ }} blocks in the HTML), and sending a temporary error code if something looks odd.

gildas 12y ago
Another additional bonus of doing that is that you can serve "true" 404 pages (i.e. not soft 404) for invalid URLs when the page is requested with the _escaped_fragment_= query parameter.
- mootothemax 12y ago
  Another additional bonus of doing is that you can serve "true" 404 pages (i.e. not soft 404) for invalid URLs when the page is requested with the _escaped_fragment_= query parameter. For one of our sites, the Angular router redirects the user to a "true" 404 page if the in-app page cannot be found, along with the troublesome URL. For example: https://example.com/products/name-does-not-exist/ https://example.com/products/name-does-not-exist/ Will do a JS redirect to: https://example.com/not-found/?url=/products/name-does-not-exist/ https://example.com/not-found/?url=/products/name-does-not-e... The /not-found/ URL serves a "true" 404 response. We're still in early days, and have very little data to back up the pros or cons of doing it this way. All I can say with any certainty is that for the several months now we have zero "soft" 404s listed in Webmaster Tools.
  - gildas 12y ago
    Interesting and good to know! I was actually wondering if a JS redirect would suffice.
    - mootothemax 12y ago
      Interesting and good to know! I was actually wondering if a JS redirect would suffice. Hehe, as I say, take it with a grain of salt, as I have very little real data to back it up with :)
toothbrush 12y ago
This sounds like something really cool to use along with NoScript! I often find myself allowing piles of scripts to run, simply because the page's content (often images, but sometimes even text) is loaded via JS instead of The Right Way.

gildas 12y ago

Another additional bonus of doing that is that you can serve "true" 404 pages (i.e. not soft 404) for invalid URLs when the page is requested with the _escaped_fragment_= query parameter.

mootothemax 12y ago
Another additional bonus of doing is that you can serve "true" 404 pages (i.e. not soft 404) for invalid URLs when the page is requested with the _escaped_fragment_= query parameter. For one of our sites, the Angular router redirects the user to a "true" 404 page if the in-app page cannot be found, along with the troublesome URL. For example: https://example.com/products/name-does-not-exist/ https://example.com/products/name-does-not-exist/ Will do a JS redirect to: https://example.com/not-found/?url=/products/name-does-not-exist/ https://example.com/not-found/?url=/products/name-does-not-e... The /not-found/ URL serves a "true" 404 response. We're still in early days, and have very little data to back up the pros or cons of doing it this way. All I can say with any certainty is that for the several months now we have zero "soft" 404s listed in Webmaster Tools.
- gildas 12y ago
  Interesting and good to know! I was actually wondering if a JS redirect would suffice.
  - mootothemax 12y ago
    Interesting and good to know! I was actually wondering if a JS redirect would suffice. Hehe, as I say, take it with a grain of salt, as I have very little real data to back it up with :)

toothbrush 12y ago

This sounds like something really cool to use along with NoScript! I often find myself allowing piles of scripts to run, simply because the page's content (often images, but sometimes even text) is loaded via JS instead of The Right Way.