4 ms·
> The OP notes at some point that the scrapers were generating lots of source tarballs individual commits. I refuse to believe OAI has a scraper that is design
by bob1029 1y ago
> The OP notes at some point that the scrapers were generating lots of source tarballs individual commits.
I refuse to believe OAI has a scraper that is designed to push commits. It is obvious to me that we are conflating other forms of malicious use with the boogeyman.
- kassner 1y agoThey are talking about the standard “Download zip”, which can also be done for commits individually. They aren’t precomputed, which causes the load.
- bob1029 1y agoThey don't need to be pre-computed or stored. You can directly stream stdout from git archive to the HTTP response. This is the default behavior of git archive.
- loloquwowndueo 1y agoAnd git archive uses zero cpu therefore it can handle thousands of archive requests per second. Right? Right??
- bob1029 1y agoYes, your CPU should be nearly idle. Git archive is I/O bound.