4 ms·
Curious, what challenges would git still have to deal with to work with big repos? 500 GB isn't all that much data in 2024, as long as you can resume a broken
by MenhirMike 3y ago
Curious, what challenges would git still have to deal with to work with big repos?
500 GB isn't all that much data in 2024, as long as you can resume a broken network connection on fetch (like using HTTP range requests if using http, and when using git LFS), and while 15 million files is a pain in most file systems because of all the overhead, a relatively modern NVMe SSD should be able to cope with it just fine, but I don't know enough about the innermost internals of git on how it manages files. Is there something that's inherently slow/expensive that doesn't scale well with the number of files?
(Basically, I'm trying to find a non-rhetorical answer to the question of "Why not?")
- m_niedoba 3y agoGit does pretty well on large repos, when you use sparse checkout and Git LFS. It makes Git behave like a centralized VC system. https://www.anchorpoint.app/blog/scaling-git-to-1tb-of-files-with-gitlab-and-anchorpoint-using-git-lfs https://www.anchorpoint.app/blog/scaling-git-to-1tb-of-files...