3 ms·
I am not a git fan. After many years (following use of RCS, SCCS, CVS, SVN) I tried it and found that its whole mental model was weird and awkward. I can get
by dekhn 2y ago
I am not a git fan. After many years (following use of RCS, SCCS, CVS, SVN) I tried it and found that its whole mental model was weird and awkward. I can get around in it but any complicated merge is just painful.
Anyway, the comment I really wanted to make was that I tried git lfs for the first time. I downloaded 44TB (https://huggingface.co/datasets/HuggingFaceFW/fineweb/tree/main https://huggingface.co/datasets/HuggingFaceFW/fineweb/tree/m...) over 3-4 days which was pretty impressive until I noticed that it seems to double disk space (90TB total). I did a little reading just to confirm it, and even learned a new term "git smudge". double disk space isn't an issue, except when you're using git to download terabytes.
- jeroenhd 2y agoGit is absolutely terrible for large files, especially binary files. That's why git LFS rarely ever uses git as a storage mechanism. I know programmers like everything to be in version control, but AI models and git just aren't compatible.
- curiousinspo 2y ago[dead]