4 ms·
A good search engine shouldn't process JS at all, and index only the content that is reachable via links pulled from HTML (which may, of course, be dynamic). T
by kazinator 7y ago
A good search engine shouldn't process JS at all, and index only the content that is reachable via links pulled from HTML (which may, of course, be dynamic).
The processing of JS explains a lot of the sheer garbage that is in the Google results.
I think they do it because it ties into their business model. JS is needed to unearth the kind of crap that a certain segment of the user base is looking for. That certain segment is worth catering to because it consists of users who are likely to click on ads.
Otherwise, nobody in their right mind would integrate JS into a crawler. "I'm going to write a program that automatically finds random source code, much of it malicious, and runs it!" Come on; without a business agenda justifying it, it's a complete non-starter.
- jdormit 7y agoThis is so out-of-touch with the modern web. Lots of totally legitimate websites, including major news outlets and Wikipedia, render their web pages on the client using JavaScript. Whether or not this is a good thing is a separate issue (personally I see nothing wrong with it), but it should be obvious that a useful search engine in 2019 needs to be able to index JavaScript-rendered content.
- irq11 7y agoI, for one, would welcome a force with the size and influence of Google telling devs to cut the Javascript crap. To me, it’s like violating the conditional rendering rule: there are plausible reasons to want to do it, but Google is right to push back on it. It’s not as if totally legitimate websites don’t have reasonable technical alternatives here.
- cameronbrown 7y ago> I, for one, would welcome a force with the size and influence of Google telling devs to cut the JavaScript crap. Wasn't this basically AMP though? The solution to the monster web developers were creating?
- kazinator 7y agoI just noticed I haven't been allowing any JS whatsoever from Wikipedia. Everything looks fine. I'm logged in and can edit, etc. (I fixed that now; all allowed). But that's not my point; how did we go from "search engines shouldn't execute JS" to "you're out of touch if you think you can use the web without JS".
- paulddraper 7y agoI guess the assumption was that search engines should be able to access the web, like humans do.
- rat9988 7y ago>A good search engine shouldn't process JS at all, and index only the content that is reachable via links pulled from HTML (which may, of course, be dynamic). A good search engine index based on what humans would consume.
- kazinator 7y agoJS doesn't ensure that; it is easily gamed to serve SEO rubbish to the spider, which is different from what it renders to a real user.
- hombre_fatal 7y agoYou’re forgetting that people use search to find things, not just to get their technical-snobbery nut off. If the thing I’m precisely looking for is a SPA, how could I or any average user care? Why would a search engine hide that from me?
- grandphuba 7y agoI'm not a fan of modern js-ridden site implementations but a search engine that displays a resource different from what will actually be consumed by the user is a bad search engine. Perhaps you can put the blame on the creator of the site, but that won't bring us anywhere if we just allow people to game the system.
- kazinator 7y agoThat's exactly what happens, though, and JS takes some of the blame. The spider is indexing stuff that is the result of code execution that won't reproduce!
- fooey 7y agoI quit catering to users who don't run JS a decade ago "nobody in their right mind" would expect to use the web in the last 15 years or so with JS disabled and have it actually work
- no_wizard 7y agoTo add to this: when a client comes to my employer, if it is required to work without Javascript it is the norm for that to be explicitly specified. Just to add to this point. It’s otherwise assumed not to be an issue
- Amezarak 7y agoPretty much the entire web works just fine without JavaScript. Indeed, most of it works better without it - everything loads faster and there’s fewer annoyances like interstitial ads, auto playing audio, etc. Unless you’re serving an actual app, like webmail or online office, a webpage that doesn’t even render without js is an immediate bounce. I can get your content elsewhere. I mean, almost the entire public web consists of blogs, forums, and news articles/essays. Almost all of these, with some modern exceptions, work perfectly fine. Of course, you may need JS to take full advantage of e.g., forum software features, but I can browse just fine. Sure, if you have a legitimate SPA like a mapping application or a game JS is a very reasonable expectation. But it’s hard for me to see what users gain otherwise. Actually, I don’t even see what developers gain, other than an improved resume.