3 ms·
Why not just return a 429 (Too Many Requests) if the specific repo has been requested by google not too long ago ? (e.g. 1 hour earlier or). It's a standard re
by mrighele 4y ago
Why not just return a 429 (Too Many Requests) if the specific repo has been requested by google not too long ago ? (e.g. 1 hour earlier or).
It's a standard response code, and with a bit of luck google will scale the requests accordingly. If not, this will still allow the proxy to operate properly without burning too much server resources.
(I understand that this may leave some customers unhappy since the proxy may be up to 1 hour stale. If that's the case, the server could also add a check to see if there have been pushes since last request, though this is quite more complex)
- quadrifoliate 4y ago> It's a standard response code, and with a bit of luck google will scale the requests accordingly. Google already says in the comments [1] that it would be "a fair bit of work" for them to read the standard robots.txt for "boring technical reasons". I would not necessarily rely on them to respect HTTP 429 either. In fact, given the facts at hand with the current situation, I would guess that they probably wouldn't handle it correctly, and would cite more "boring technical reasons" when asked. ---------------------------------------- [1] https://github.com/golang/go/issues/44577#issuecomment-1137883692 https://github.com/golang/go/issues/44577#issuecomment-11378...
- mro_name 4y ago> "a fair bit of work" for them to read the standard robots.txt for "boring technical reasons". that's embarrassing. What other reasons should there ever be? Stop taking their money, stop using their shit. Turn your attention to where it's beneficial.
- opmac 4y agoReturning a 429 would not just to hope that the Go proxy would obey the response, but rather it would significantly reduce the workload the server has to do when it gets a request it doesn't want to serve.