4 ms·
> It's a standard response code, and with a bit of luck google will scale the requests accordingly. Google already says in the comments [1] that it would be "a
by quadrifoliate 4y ago
> It's a standard response code, and with a bit of luck google will scale the requests accordingly.
Google already says in the comments [1] that it would be "a fair bit of work" for them to read the standard robots.txt for "boring technical reasons". I would not necessarily rely on them to respect HTTP 429 either.
In fact, given the facts at hand with the current situation, I would guess that they probably wouldn't handle it correctly, and would cite more "boring technical reasons" when asked.
----------------------------------------
[1] https://github.com/golang/go/issues/44577#issuecomment-1137883692 https://github.com/golang/go/issues/44577#issuecomment-11378...
- mro_name 4y ago> "a fair bit of work" for them to read the standard robots.txt for "boring technical reasons". that's embarrassing. What other reasons should there ever be? Stop taking their money, stop using their shit. Turn your attention to where it's beneficial.
- opmac 4y agoReturning a 429 would not just to hope that the Go proxy would obey the response, but rather it would significantly reduce the workload the server has to do when it gets a request it doesn't want to serve.