3 ms·
Is that really a thing? That must be such a hazard for their developers. I usually have a test for sites that I work on, that scrapes a few URLs as Googlebot, t
by devrandomguy 9y ago
Is that really a thing? That must be such a hazard for their developers. I usually have a test for sites that I work on, that scrapes a few URLs as Googlebot, to verify that they are getting an optimized view (no JS, structural-only css).
- Normal_gaussian 9y agoGod. That's the reason so many sites look great in the results and are confusing interactive messes when I get into them.
- devrandomguy 9y agoIf it's any consolation, the site is well tested in Noscript mode.
- SomeCallMeTim 9y agoYou sir are a god among Web developers. If only they all did this. So many sites I get to and they're a blank page or an absolute disaster....
- SomeCallMeTim 9y agoYou sir are a god among Web developers. If only they all did this. So many sites I get to and they're a blank page or an absolute disaster....
- ChuckMcM 9y agoYes. Googlebot only crawls from legit addresses (even when their developers are trying new things) so it's an easy scraper/scammer signal to key off of.
- dmn001 9y agoNo. Most websites don't do this.