3 ms·

I've found that git subtrees are a much better option. Good intro here:https://www.atlassian.com/git/tutorials/git-subtree https://www.atlassian.com/git/tutoria

I've found that git subtrees are a much better option. Good intro here:https://www.atlassian.com/git/tutorials/git-subtree https://www.atlassian.com/git/tutorials/git-subtree They're like submodules, but you can edit them and they're embedded in your history, not just a reference.

rkangel 6y ago
For those who aren't familiar with them, subtrees and submodules are two different approaches to the same problem - wanting the contents of another git repo available to you. To understand the differences, imagine two solutions you might hack together: Have your build script do a 'git clone' of a specific repo into your folder structure, so that your build can reference the files. You can imagine some scripts you might write to simplify it, and you can also imagine doing it on a git hook so it happen automatically on checkout. This is git submodules. Alternatively, you could just take that other repo, copy all the files into a subfolder of your repo and check them in. When you want to update it, you need to do a bit of manual fiddling and then check in again. This is git subtrees (with commands to help you more easily do the updates). I think people create repos when they shouldn't anyway - I subscribe to a more monorepo approach with lots of things in one repo, and git subtrees are an obvious match to that.
- IshKebab 6y ago
  So git subtree is just a monorepo basically?
  - Phlogistique 6y ago
    Yes - or rather, it's a tool that helps you build a monorepo from separate repos.
  - rkangel 6y ago
    It's a monorepo, where some of the contents are a copy of another existing repo. I'm sure we've all 'vendored in' a dependency before - just copying in the files and checking them in. The fiddly bit is when you then want to update that vendored dependency to the newer version (more manual copying of files and checking in), and it gets tedious and error prone when you have 10 of them. Subtrees do the same thing, but provide tooling to make it a lot easier.
asplake 6y ago
I use a subtree to maintain an open repo within a private one. It works, but why is it so slow? You change just one file and before attempting a push it seems to scan the whole thing. I tried this recently without a network connection, pretty clear that the slow bit is entirely local.
- vaughan 6y ago
  https://stackoverflow.com/questions/16134975/how-can-i-reduce-the-ever-increasing-time-to-push-a-subtree https://stackoverflow.com/questions/16134975/how-can-i-reduc... I think it's just inefficiently implemented. `git subtree` is just a shell script using other git commands. Take a look at the `--rejoin` flag.
  - asplake 6y ago
    Thanks > When you run git subtree push, it will recreate all commits for this subtree. It has to do that, as their SHA depends on the previous commit and needs those SHAs to be able to link the new commits to the old ones. It could cache that, but it doesn’t. > I guess this is the price you pay for using subtree vs. submodules. Subtree is very stateless in your repository, which is nice on the hand but causes this long computation on the other hand. Submodules store all their information, which requires you to manage it but also makes stuff like this a lot faster.
rrosen326 6y ago
My two cents- subtrees were SO much harder for my use case. Rename a directory? Ugh. Subtree split? Commit history? Just a mess. I switched back to sub modules and it’s such a relief.
- vaughan 6y ago
  Yeh, subtrees can get hairy. I think there is a need for a cli wrapper tool (with nice visualization) to hand-hold through the process. Too much can go wrong otherwise.
vaughan 6y ago
Yep, subtrees are the answer. I wrote [this piece](https://vjpr.medium.com/the-multi-monorepo-209041932fbf https://vjpr.medium.com/the-multi-monorepo-209041932fbf) yesterday about using a monorepo of monorepos using git subtree over git submodules, which should cover almost all use cases. Basically, you only need other to create other git repos depending on access rights. That is, proprietary code vs open-source. The biggest mistake is thinking: "I want to open-source these two packages, so I will create two repos and make them public and then add them as submodules in my private company repo". The problem though is that it's such a natural thought, and people don't realize the pain they are about to inflict.

rkangel 6y ago

For those who aren't familiar with them, subtrees and submodules are two different approaches to the same problem - wanting the contents of another git repo available to you. To understand the differences, imagine two solutions you might hack together: Have your build script do a 'git clone' of a specific repo into your folder structure, so that your build can reference the files. You can imagine some scripts you might write to simplify it, and you can also imagine doing it on a git hook so it happen automatically on checkout. This is git submodules. Alternatively, you could just take that other repo, copy all the files into a subfolder of your repo and check them in. When you want to update it, you need to do a bit of manual fiddling and then check in again. This is git subtrees (with commands to help you more easily do the updates). I think people create repos when they shouldn't anyway - I subscribe to a more monorepo approach with lots of things in one repo, and git subtrees are an obvious match to that.

IshKebab 6y ago
So git subtree is just a monorepo basically?
- Phlogistique 6y ago
  Yes - or rather, it's a tool that helps you build a monorepo from separate repos.
- rkangel 6y ago
  It's a monorepo, where some of the contents are a copy of another existing repo. I'm sure we've all 'vendored in' a dependency before - just copying in the files and checking them in. The fiddly bit is when you then want to update that vendored dependency to the newer version (more manual copying of files and checking in), and it gets tedious and error prone when you have 10 of them. Subtrees do the same thing, but provide tooling to make it a lot easier.

asplake 6y ago

I use a subtree to maintain an open repo within a private one. It works, but why is it so slow? You change just one file and before attempting a push it seems to scan the whole thing. I tried this recently without a network connection, pretty clear that the slow bit is entirely local.

vaughan 6y ago
https://stackoverflow.com/questions/16134975/how-can-i-reduce-the-ever-increasing-time-to-push-a-subtree https://stackoverflow.com/questions/16134975/how-can-i-reduc... I think it's just inefficiently implemented. `git subtree` is just a shell script using other git commands. Take a look at the `--rejoin` flag.
- asplake 6y ago
  Thanks > When you run git subtree push, it will recreate all commits for this subtree. It has to do that, as their SHA depends on the previous commit and needs those SHAs to be able to link the new commits to the old ones. It could cache that, but it doesn’t. > I guess this is the price you pay for using subtree vs. submodules. Subtree is very stateless in your repository, which is nice on the hand but causes this long computation on the other hand. Submodules store all their information, which requires you to manage it but also makes stuff like this a lot faster.

rrosen326 6y ago

My two cents- subtrees were SO much harder for my use case. Rename a directory? Ugh. Subtree split? Commit history? Just a mess. I switched back to sub modules and it’s such a relief.

vaughan 6y ago
Yeh, subtrees can get hairy. I think there is a need for a cli wrapper tool (with nice visualization) to hand-hold through the process. Too much can go wrong otherwise.

vaughan 6y ago

Yep, subtrees are the answer. I wrote [this piece](https://vjpr.medium.com/the-multi-monorepo-209041932fbf https://vjpr.medium.com/the-multi-monorepo-209041932fbf) yesterday about using a monorepo of monorepos using git subtree over git submodules, which should cover almost all use cases. Basically, you only need other to create other git repos depending on access rights. That is, proprietary code vs open-source. The biggest mistake is thinking: "I want to open-source these two packages, so I will create two repos and make them public and then add them as submodules in my private company repo". The problem though is that it's such a natural thought, and people don't realize the pain they are about to inflict.