3 ms·
Why the burden of making this google-specific rate-limiter should fall on Sourcehut ? They are a small team and a user-agent-per-repo specific limiter have rare
by wut42 4y ago
Why the burden of making this google-specific rate-limiter should fall on Sourcehut ? They are a small team and a user-agent-per-repo specific limiter have rare uses.
Google should definitely fix their shit, but when we see that a googler said "it would be a fair bit of extra work for us to read robots.txt", I understand the frustration on Sourcehut side, and banning an user agent is usually just a single line of configuration to add.