4 ms·

It appears to be the exact opposite to me, `git clone --depth 1 ...` will give you a code that you can know exactly how to parse, vs. webpages that have all sor

It appears to be the exact opposite to me, `git clone --depth 1 ...` will give you a code that you can know exactly how to parse, vs. webpages that have all sorts of semantical issues.

marginalia_nu 3y ago
Git clone is a very expensive operation. Git hosts generally will try to prohibit mass git clone:ing for this reason.
- blowski 3y ago
  What makes it so expensive? I’d always assumed it downloaded the .git directory statically, and the computational bits were down by the local client.
  - ablob 3y ago
    I'd assume this is in relation to how much other operations cost. With 'git clone' you at least download the whole repository. Compare that to 'git fetch', which is essentially a lookup at the last-modified timestamp.
    - marginalia_nu 3y ago
      Yeah. Git repositories can grow very large very quickly. A single clone here and there isn't too bad, but if you're scraping tens of thousands of projects, you can easily rack up terabytes in disk and network access.
    - deleted 3y ago
      [deleted]