4 ms·

You say this, but Git has made great strides in scaling to huge repositories in recent years. You can currently do the "checkout just a subtree of the monorepo

You say this, but Git has made great strides in scaling to huge repositories in recent years. You can currently do the "checkout just a subtree of the monorepo" just fine, and you can use shallow clones to approximate a centralized system (and most importantly to use less local storage). > If you had such a system, getting people off git wouldn't be the issue - offer git compatibility and [...] Git is already doing exactly that.

vlovich123 2y ago
Git itself isn’t though, not in any real way that matters. Having to know all the sub trees to clone in a mono repo is a usability nonstarter. You need a pseudo filesystem that knows how to pull files on access. And one ideally integrated with the build system to offset the cost of doing remote operations on demand and improve parallelism. Facebook is open sourcing a lot of their work but it’s based on mercurial. Microsoft is bought into git but afaik hasn’t open sourced their supporting git tooling that makes this feasible. TLDR: the problem is more complex and pretending like “you can checkout a subtree” solves the problem is missing the proverbial forest for the (sub)tree
- neerajsi 2y ago
  Microsoft's vfs for git is open source. So is scalar. These are the two main approaches used at Microsoft for large repos. Unfortunately the technically superior vfs approach was a nonstarter on macOS.
fragmede 2y ago
> "checkout just a subtree of the monorepo" How do I check out, eg https://github.com/neovim/neovim/tree/master/scripts https://github.com/neovim/neovim/tree/master/scripts into a directory and work with it as if it was a repo unto itself?
- nolist_policy 2y ago
  You can't (since commits are snapshots of the repo root). You can have this approximation however: git clone --filter=blob:none --sparse https://github.com/neovim/neovim cd neovim git sparse-checkout add scripts Unfortunately, GitHub does not support --filter=sparse:oid=master:scripts, so blobs will be fetched on demand as you use the repo.

vlovich123 2y ago

Git itself isn’t though, not in any real way that matters. Having to know all the sub trees to clone in a mono repo is a usability nonstarter. You need a pseudo filesystem that knows how to pull files on access. And one ideally integrated with the build system to offset the cost of doing remote operations on demand and improve parallelism. Facebook is open sourcing a lot of their work but it’s based on mercurial. Microsoft is bought into git but afaik hasn’t open sourced their supporting git tooling that makes this feasible. TLDR: the problem is more complex and pretending like “you can checkout a subtree” solves the problem is missing the proverbial forest for the (sub)tree

neerajsi 2y ago
Microsoft's vfs for git is open source. So is scalar. These are the two main approaches used at Microsoft for large repos. Unfortunately the technically superior vfs approach was a nonstarter on macOS.

fragmede 2y ago

> "checkout just a subtree of the monorepo" How do I check out, eg https://github.com/neovim/neovim/tree/master/scripts https://github.com/neovim/neovim/tree/master/scripts into a directory and work with it as if it was a repo unto itself?

nolist_policy 2y ago
You can't (since commits are snapshots of the repo root). You can have this approximation however: git clone --filter=blob:none --sparse https://github.com/neovim/neovim cd neovim git sparse-checkout add scripts Unfortunately, GitHub does not support --filter=sparse:oid=master:scripts, so blobs will be fetched on demand as you use the repo.