3 ms·
Google can and does crawl Javascript driven web apps. Last year they even deprecated the AJAX crawling scheme that they used to recommend on the basis that they
by onion2k 11y ago
Google can and does crawl Javascript driven web apps. Last year they even deprecated the AJAX crawling scheme that they used to recommend on the basis that they can now crawl web apps: https://googlewebmastercentral.blogspot.co.uk/2015/10/deprecating-our-ajax-crawling-scheme.html https://googlewebmastercentral.blogspot.co.uk/2015/10/deprec...
- TimJYoung 11y agoYes, but in the case of SAAS apps, what exactly are they crawling (if allowed) ? In our case, it would be a bunch of edit controls, grids, buttons, containers, etc. that don't really make a whole lot of sense in the context of a search engine. I think it's important to start recognizing that there's two distinct types of web "applications" starting to emerge: the more traditional kind of "application" that is actually more like a web site, and the newer kind of application that is more like an actual application (desktop/mobile). Edit: Just read the new guidelines, and they are now considering serving up different content for the crawler as being "against the rules". So that probably cements the idea that you shouldn't allow Google to crawl anything that's mostly content-free and just an "app".
- onion2k 11y ago"Don't let Google crawl your site if it's not appropriate" is great advice, that's true, but it has nothing to do with whether you're serving up JS or HTML.
- TimJYoung 11y agoYeah, I'm conflating the two, based upon my personal experiences. But, are people really writing applications that generate all content completely on the client side via JS, but want the end result to be more like a traditional web site ? That seems weird, even to me. :-)
- bphogan 11y agoTons of news sites are doing exactly this. I can't tell you how many times I follow a link from Facebook or other social media, only to be brought to a page where a spinning gif spins for several seconds while JS loads the article.
- M2Ys4U 11y ago> Just read the new guidelines, and they are now considering serving up different content for the crawler as being "against the rules". That is called "cloaking", and Google has said this for a very, very long time. At least 10 years.
- madeofpalk 11y agoThey had an exception for it for sending Googlebot to a 'static' copy of your site for rendering SPA.
- madeofpalk 11y agoGoogle crawls Javascript sites, but it doesn't execute AJAX calls so most SPA's still won't be crawlable by Google.
- aws_ls 11y agoIn theory it works, but in practice it awfully degrades the page load time. The way it works is that it will call the init() method of JS app (ready() in jquery, main() in Dart etc). And it is very sub-optimal compared to server side generation. This is because it has to execute the ready()/start() method, which may spawn of some async requests to get some JSON, on getting which successfully you recreate your DOM. So too many possible failure points, and for these reasons it will timeout many times, resulting in increased crawl errors, apart from page load time degradation. I learned this the hard way, and had to revert to server side generation of crawlable pages. And now looking at something like react/redux to achieve what I originally wanted (which is basically again server side generation, but while maintaining the quality of the served pages).