4 ms·
> And if you 429 Google’s bots they will reduce your pagerank. That’s straight up extortion from a company that also sells cloud services. Googlebot uses diffe
by dilyevsky 1y ago
> And if you 429 Google’s bots they will reduce your pagerank. That’s straight up extortion from a company that also sells cloud services.
Googlebot uses different IP space from gcp
- hinkley 1y agoThey use the same bank accounts and stock ticker. This is basically a non sequitur. The point is they’re getting paid to run cloud servers to keep their bots happy and not dropping your website to page six.
- dilyevsky 1y agoI thought the argument was that if you run on gcp you can masquerade as googlebot and not get a 429 which is obviously false. Instead it looks like the argument is more of a tinfoil hat variety. btw you don't get dropped if you issue temporary 429s only when it's consistent and/or the site is broken. that is well documented. and wtf else are they supposed to do if you don't allow to crawl it and it goes stale?