2 ms·
Weird how the conclusion is that the sites are wrong, not that curl is an insufficient tool for indexing the modern web. Google search hasn't died because site
by shadowgovt 6y ago
Weird how the conclusion is that the sites are wrong, not that curl is an insufficient tool for indexing the modern web.
Google search hasn't died because sites use JS.
- kortilla 6y agoBut it shouldn’t require such a complex js execution engine when the end result is something that is equivalent to what used to be retrievable with a simple curl request.
- progval 6y ago> Google search hasn't died because sites use JS. I believe it's the other way around. Websites use just as much JS as they can get away with, without sinking in Google results. If Google never ran JS in the first place, most websites wouldn't depend on it. And now every crawler has to keep up with Googlebot's computing power if they want to archive the web.