3 ms·

The OP notes at some point that the scrapers were generating lots of source tarballs individual commits. That takes server resources, and a small server can't s

The OP notes at some point that the scrapers were generating lots of source tarballs individual commits. That takes server resources, and a small server can't sustain many of those requests per second. I wonder why you (apparently) don't suffer from this — in fact one can even wonder why such tarballs are generated in the first place, it sounds very pointless.

bob1029 1y ago
> The OP notes at some point that the scrapers were generating lots of source tarballs individual commits. I refuse to believe OAI has a scraper that is designed to push commits. It is obvious to me that we are conflating other forms of malicious use with the boogeyman.
- kassner 1y ago
  They are talking about the standard “Download zip”, which can also be done for commits individually. They aren’t precomputed, which causes the load.
  - bob1029 1y ago
    They don't need to be pre-computed or stored. You can directly stream stdout from git archive to the HTTP response. This is the default behavior of git archive.
    - loloquwowndueo 1y ago
      And git archive uses zero cpu therefore it can handle thousands of archive requests per second. Right? Right??
      - bob1029 1y ago
        Yes, your CPU should be nearly idle. Git archive is I/O bound.