4 ms·
The bots can still do a git clone and index everything, this just inconveniences normal users working on "some other" PC (or browser or incognito tab), where th
by ajsnigrutin 3y ago
The bots can still do a git clone and index everything, this just inconveniences normal users working on "some other" PC (or browser or incognito tab), where they are not logged in, and/or don't want to log in (coworkers PC, 2fa, whatever).
- Turing_Machine 3y ago> The bots can still do a git clone and index everything Of course they can, but then they're gonna be chewing up their own disk space and bandwidth for anything after the initial hit. I think the real problem is that the bots hit the GitHub servers over, and over, and over again.
- mmahemoff 3y agoMaybe if the bot operators have the resources, but it's far from trivial to keep an up-to-date mirror of every project on Github, especially if Github is actively putting up barriers to prevent it. Once login is required, it becomes much harder to bypass rate limits because the company can rate-limit signups from unknown domains, enforce 2FA, etc.
- ajsnigrutin 3y agoThey can do all this (rate limits, etc.) for unregister accounts already, and most users would never notice that (since a human only does a few searches per time unit), but they decided to require a login anyway.
- arp242 3y agoA lot of bot traffic is just mindless "follow any link" traffic, not specialized bots to do X. It really is hugely pointless and wasteful to have tons of these bots request tons of comparatively expensive search links.