11 ms·

Git Submodules: Adding, Using, Removing, Updating (2009)

110 points by tigerlily 6y ago

flohofwoe 6y ago
IME submodules are mostly fine for external dependencies that only very rarely change and remain pinned at a specific version over weeks or months. Giving submodules a special set of git commands, and providing the users so many easy opportunities to shoot themselves in the foot are baffling design decisions though. E.g. I'd really like to know why "git clone --recursive" isn't the default.
- maweki 6y ago
  And then you always forget what you need to do after you didn't do it for months. I have a blog project (written with pelican) where I submodule the whole pelican-plugins project which is, basically, again a compendium of submodules. Now when running Python creates pycache-folders in the submodules and other stuff in the directories sometimes changes a bit while running. When it comes to updating the plugins-submodule, I have to go through every sub-submodule and git clean it and reset it so that I can pin a new version for the submodule, and then update all sub-submodules. Mind you, I have not found it easier to delete the whole submodule-tree and re-initialise it.
  - angrais 6y ago
    Wouldn't this be resolved by a good README.md?
    - maweki 6y ago
      Of course, the correct way would be to write down all the steps, operationalise them, and then implements parts and then the whole as a shell script. The question I would ask is, whether this should be necessary to pin the version of some project dependency? That's basically all we're talking about here.
    - Pxtl 6y ago
      Code is the best documentation. update-submodules-latest.sh (well, PS1 for my team, we're a .net shop) to move to latest along their branch and stage the submodule change
- c03 6y ago
  Both --recursive not being default and then using --recurse-submodules when pulling are my top annoyances in git.
  - junon 6y ago
    Another tip: you can do a deep clone of the main repo, then do `git submodule update --init --recursive --depth=1` to just pull the heads of your dependencies to cut down on times. Especially useful if you don't work within the submodules at all.
    - nerdponx 6y ago
      IMO this should be the default!
      - junon 6y ago
        No. History is the whole point of git. It would be antithetical to make this the default. I'm more on board with the recursive options, though.
  - jpxw 6y ago
    At the very least, it could warn you when you pull a repo with submodules without —recurse-submodules. But it doesn’t, it just silently does nothing.
  - nonbirithm 6y ago
    I can't for the life of me understand why this isn't at least a config option. That and an option for setting --upstream in git push by default. Those could at least preserve backwards compatibility. Are the Git maintainers so stringent on maintaining backwards compatibility that the implementation of submodules has remained in such a dire state for so long? I find it hard to believe that a VCS this widely adopted would fall over so easily for this relatively popular use case.
    - jayd16 6y ago
- nemetroid 6y ago
  I’ve found that setting submodule.recurse to true makes submodules behave closer to how you’d expect them to.
- user-the-name 6y ago
  Mercurial's subrepos implement the exact same functionality as git submodules, but they do it without being as bafflingly broken as git's attempt.
wheybags 6y ago
I've found that git subtrees are a much better option. Good intro here:https://www.atlassian.com/git/tutorials/git-subtree https://www.atlassian.com/git/tutorials/git-subtree They're like submodules, but you can edit them and they're embedded in your history, not just a reference.
- rkangel 6y ago
  For those who aren't familiar with them, subtrees and submodules are two different approaches to the same problem - wanting the contents of another git repo available to you. To understand the differences, imagine two solutions you might hack together: Have your build script do a 'git clone' of a specific repo into your folder structure, so that your build can reference the files. You can imagine some scripts you might write to simplify it, and you can also imagine doing it on a git hook so it happen automatically on checkout. This is git submodules. Alternatively, you could just take that other repo, copy all the files into a subfolder of your repo and check them in. When you want to update it, you need to do a bit of manual fiddling and then check in again. This is git subtrees (with commands to help you more easily do the updates). I think people create repos when they shouldn't anyway - I subscribe to a more monorepo approach with lots of things in one repo, and git subtrees are an obvious match to that.
  - IshKebab 6y ago
    So git subtree is just a monorepo basically?
    - Phlogistique 6y ago
      Yes - or rather, it's a tool that helps you build a monorepo from separate repos.
    - rkangel 6y ago
      It's a monorepo, where some of the contents are a copy of another existing repo. I'm sure we've all 'vendored in' a dependency before - just copying in the files and checking them in. The fiddly bit is when you then want to update that vendored dependency to the newer version (more manual copying of files and checking in), and it gets tedious and error prone when you have 10 of them. Subtrees do the same thing, but provide tooling to make it a lot easier.
crowdhailer 6y ago
Submodules aren't perfect but we've made use of them in a few projects and one we had a process around them they were useful.
nathell 6y ago
Oh yeah, submodules can bite. At my previous job, we tried open-sourcing a part of our monorepo using them, which resulted in a clunky workflow. So we whipped up a syncing bot integrating with GitHub that would propagate changes both ways: https://github.com/WorksHub/flow-bot https://github.com/WorksHub/flow-bot
NalNezumi 6y ago
Oh I remember at previous work when the (now lead engineer) decided that the main repo that depended on actively changing 4 subrepo currently included as subtrees had to be changed to submodule. This in a rapidly developing repo where the main branch hadn't been update for 18 months while "develop" branch was switched to other "sub" branches on monthly basis. Needless to say it was a total shitshow. My first experience of how "wanting to use the cool stuff I just found out about" type of perso in position of decision making can be catastrophic. The dude wanted to switch all code base to C++20 in 2020, 3 days after the conference
sto_hristo 6y ago
My lord, so much hate for them submodules. I'm actively using them for my project, where i have common components that are shared between other components. Having them as dependencies managed by a build tool is cumbersome as sometimes, purely for the sake of convenience, i need to update some shared component in place and have the changes propagate to other dependents. Had a few mishaps, but those were the result of lack of experience. Overall, my experience with submodules is great. I can imagine what terror awaits those with bigger projects where everyone starts pushing random stuff from all directions.
- robin21 6y ago
  Yeh for teams they are terrible. For solo development using a single branch they can see okay.
junon 6y ago
Most of the "complaints" here are people misusing submodules or not taking the time to learn how Git itself works. I work with submodules almost daily for dependencies in game engines, OS projects, and even network services. I never have issues. However, that's not to say they have a strange set of commands (or lack thereof). I have to touch the .git folder way too often, and that shouldn't be the case. Maybe I'll write a small cookbook of how to deal with issues, including an "everything is fucked, how do I start over?" section.
BuckarooBanzay 6y ago
Despite all the comments saying to avoid submodules i can only recommend them. I'm using them in several of my game servers as a "meta-repo" that points to other git repositories (for example here: https://github.com/pandorabox-io/pandorabox-mods https://github.com/pandorabox-io/pandorabox-mods) It makes updating, finding/fixing bugs and testing much easier (we are using github's dependabot to update and kick off initial tests)
- pornel 6y ago
  What they achieve is very useful. How they do it is terrible. I use them too, but their "user interface" is half-assed and full of traps. I have to have a bunch of extra scripts to check and fix them, and a ton of warnings in the README for poor souls who haven't experienced the pain yet.
  - nerdponx 6y ago
    "user interface" is half-assed and full of traps Welcome to Git. And least it's a lot better than it used to be.
- JamesSwift 6y ago
  Yes, I think as a way of vendoring external dependencies they are ok. That's about the only use case that makes sense to me. The friction they cause isn't worth it otherwise.
OskarS 6y ago
Here's the lesson I've learned from my experience with submodules in git in several different companies: avoid them like the plague. NEVER USE THEM FOR ANYTHING. Use any other solution available to you: use package managers, use monorepos, use subtrees, just copy/paste the files in, avoid the dependency entirely, do ANYTHING it takes to avoid using submodules. They just become a constant source of friction. Basically every action you do in git, there's some tiny bit of annoyance caused by submodules, which adds up to tearing your hair out. Like, read this StackOverflow question and answer, and tell me this is something you want to be dealing with on daily basis (and you will be, you'll regularly be dealing with far worse): https://stackoverflow.com/questions/9314365/git-clean-is-not-removing-a-submodule-added-to-a-branch-when-switching-branches https://stackoverflow.com/questions/9314365/git-clean-is-not... The correct way to handle dependencies in general is package managers, but if for some reason that is not available to you, and you wish to avoid git subtree for some reason, copy/pasting the code from the other repository and making a note in the commit message which commit in the dependency repo you copied from is a far preferable solution. Yeah, you lose the history of the sub-repo, but it's well worth it to avoid the complications from using submodule (and you never examine the history of the submodule anyway, how often do you look at the history of your dependencies?). Submodules: not even once.
- m_eiman 6y ago
  Don't use subtrees, they're even worse. At least when we tried using them for a while before moving to submodules; perhaps they're less of a broken hack these days. For example, whenever we did something with the subtree it reprocessed all commits in the repo - obviously taking longer and longer as time went on.
- junon 6y ago
  Disagree. Package managers cause more headache than anything. I have used submodules for almost all of my projects (aside from e.g. Node.js projects). Not once have had issues with them. I don't know why people raise such a stink about them.
  - dataflow 6y ago
deleted 6y ago
[deleted]
Pxtl 6y ago
I'm using submodules in a project and I have to agree - they're awful. The fact that their ergonomics are so bad in as trivial cases as switching branches is flabbergasting. I mean, it was the Right Tool for the Job for my use-case but the experience was awful. I'm pretty well convinced that the emperor wears no clothes when it comes to Git in general.
- pydry 6y ago
  This has bugged me for a while. Culturally there are three types of programmer tool: * Those unknown or disliked enough that anything that goes wrong when using them is the tools fault even if it isn't. * Those well known and liked enough that they get blamed for their own issues. * Those that are put on a pedestal such that any problems they do have are the fault of the user. git falls squarely into the latter camp.
- mpawelski 6y ago
  Is it still that bad if you set submodule.recurse[0] config? [0] https://git-scm.com/docs/git-config#Documentation/git-config.txt-submodulerecurse https://git-scm.com/docs/git-config#Documentation/git-config...
krona 6y ago
Git submodule should be renamed 'git conway', named after Conway's Law: there is no good reason to use it except within those companies that have to compensate for dysfunctional technical leadership.
- junon 6y ago
  There are great reasons to use them. You've never been bitten by the alternative, it seems.
dboreham 6y ago
Yes submodules are to be avoided, the 99% of the problem is poor documentation that fails to explain submodule behavior adequately, leading the reader to assume the feature conforms to a mental model that it doesn't.
- ta8645 6y ago
  Do you have an example of this mismatch you could share?
  - pornel 6y ago
    Users assume submodules would behave sensibly, and they don't. State of submodules is part of a commit, so you'd expect that checking out a commit checks out the submodules. It doesn't. Or if you notice git doesn't care about the content of submodules in the working tree, you'll be surprised when rebasing (change of a dir to/from submodule breaks rebase). Merging has "ours"/"theirs" startegies, except it doesn't for submodules (there's suddenly local/remote distinction, and breaks -Xtheirs in rebase). From the diffs it looks like submodules are defined in .gitmodules and the tree, so removing them from these places would delete them. It doesn't (it fucks up internal state of .git dir). And you'd think .git dir is a dir, except when it isn't. It's like whoever designed submodules just loves saying "gotcha!" and telling frustrated users they suck at git. Everything submodules do is subtly different with sharp edges.
40four 6y ago
This post should have (2009) appended to it. It's not obvious because there is no date on the post, but if you scroll down to comments, they are from 2009. That means these code examples were written in Git v1.6. The current Git release is v 2.30. I haven't inspected the commands, but I'd be willing to bet the interface has changed since then.
zoobab 6y ago
Docker hub rails to build an image if you call hit submodules in there. I had the problem with DirtyJTAG Dockerfile, build fine on my laptop, does not build on Dockerhub. I mean submodules were a pain in 2010, they are still a pain in 2021. So much for 'open source'.
deleted 6y ago
[deleted]
1MachineElf 6y ago
QMK is only one repository I ever spend time working with, and it has some submodules. I find it annoying that they must be pulled each time I switch branches. Should I look into changing them so that they at least point to a repository on my local machine instead of having to re-download from GitHub each time I want to create a new branch? Is that possible, and does anyone else do it?
matheusmoreira 6y ago
I really hate how submodules are just pointers to specific commits. This forces an update to the superproject every time I update the submodule. Even if I add commits to the submodule, it won't check out the changes until the superproject's pointer is updated. I just like to develop libraries and the application that uses them at the same time. Whenever I change the library, I want to see the changes in the application immediately. The only tool I've used that got this right is Python's setuptools. It's got a development mode which essentially replaces the package with a symbolic link to a local repository. Why doesn't git support this?
juangacovas 6y ago
Anyone can compare the "externals" in SVN with "submodules" in git? I'm curious since I'm used to externals in SVN and they're ok but as always you have to keep some things in mind
- Izkata 6y ago
  A git submodule is almost the same thing as an svn external that's pinned to a specific revision. From that baseline, these are the differences I can think of: * The submodules aren't automatically updated like externals are with "svn up", you need a second command. * Adding a submodule is done with a command, not editing files/props directly. * Any changes to the pinned revision are done inside the submodule (git pull, new commits, etc), which automatically changes the reference in the parent repo and then is committed in the parent repo. * Removing a submodule is two commands now, not a bunch of manual work (article is out of date: https://news.ycombinator.com/item?id=26171333 https://news.ycombinator.com/item?id=26171333 ).
rubyist5eva 6y ago
Git is an insane tool. I've never seen a single command line tool blogged about so much. You don't see anything like this with regards to Mercurial or other distributed version control systems(and most of them are just as powerful as git, if not moreso), the documentation is usually enough for these tools. For some reason git is special - it has this ability to be completely opaque to so many people and yet still be deployed almost literally everywhere. I am completely baffled by its ubiquity. Thanks Github, I guess..?
Izkata 6y ago
Note that the "2009" is quite important, a few things here are solved (though with the usual odd interface if you're not used to it). > If that repository also has submodules, then your submodule’s submodules will have to be populated by following the steps below from within your project’s submodule directory (confusing yet?). Now: git clone repo target cd target git submodule update --init --recursive > Unfortunately, this is wrong. Git does not have a built in way to remove submodules. It's a two stage process (the first cleans up .git/config, the second actually removes the code and submodule), but does exist now: git deinit lib/billboard git rm lib/billboard
tigerlily 6y ago
I want to use Dear ImGui [1] in a C++ project. Sure I can copy the relevant files into my repo by hand, but my instinct says that is not the way. Git submodule seems like the right tool, but there's plenty of comments here advising not to use it. So if not git-submodule, then what is the procedure? I just want to have the external repo as a subfolder in my own repo, updating that subfolder as updates become available. [1] https://github.com/ocornut/imgui https://github.com/ocornut/imgui
- nerdponx 6y ago
  Git submodule is the right tool. Git submodule works well for externally-developed "read-only" parts of your project that you need to update infrequently. Git submodules a few years ago used to be a pain in the ass, partly because the CLI was missing a lot of important functionality. Pretty much all of those concerns are in the past, and now it's just people complaining about having to pass a couple extra command line flags (which you can/should just add to your README).
  - tigerlily 6y ago
    Thanks! A quick perusal of some reputable docs corroborates your recommendation [1]. [1] https://www.atlassian.com/git/tutorials/git-submodule https://www.atlassian.com/git/tutorials/git-submodule
newusertoday 6y ago
Android combines multiple git repositories with manifest(xml file) and has associated tooling for it. Not sure why that approach is not popular?
tpoacher 6y ago
Sometimes I almost feel like git is slowly trying to reinvent svn ...

110 points by tigerlily 6y ago

IME submodules are mostly fine for external dependencies that only very rarely change and remain pinned at a specific version over weeks or months. Giving submodules a special set of git commands, and providing the users so many easy opportunities to shoot themselves in the foot are baffling design decisions though. E.g. I'd really like to know why "git clone --recursive" isn't the default.

maweki 6y ago
And then you always forget what you need to do after you didn't do it for months. I have a blog project (written with pelican) where I submodule the whole pelican-plugins project which is, basically, again a compendium of submodules. Now when running Python creates pycache-folders in the submodules and other stuff in the directories sometimes changes a bit while running. When it comes to updating the plugins-submodule, I have to go through every sub-submodule and git clean it and reset it so that I can pin a new version for the submodule, and then update all sub-submodules. Mind you, I have not found it easier to delete the whole submodule-tree and re-initialise it.
- angrais 6y ago
  Wouldn't this be resolved by a good README.md?
  - maweki 6y ago
    Of course, the correct way would be to write down all the steps, operationalise them, and then implements parts and then the whole as a shell script. The question I would ask is, whether this should be necessary to pin the version of some project dependency? That's basically all we're talking about here.
  - Pxtl 6y ago
    Code is the best documentation. update-submodules-latest.sh (well, PS1 for my team, we're a .net shop) to move to latest along their branch and stage the submodule change
c03 6y ago
Both --recursive not being default and then using --recurse-submodules when pulling are my top annoyances in git.
- junon 6y ago
  Another tip: you can do a deep clone of the main repo, then do `git submodule update --init --recursive --depth=1` to just pull the heads of your dependencies to cut down on times. Especially useful if you don't work within the submodules at all.
  - nerdponx 6y ago
    IMO this should be the default!
    - junon 6y ago
      No. History is the whole point of git. It would be antithetical to make this the default. I'm more on board with the recursive options, though.
- jpxw 6y ago
  At the very least, it could warn you when you pull a repo with submodules without —recurse-submodules. But it doesn’t, it just silently does nothing.
- nonbirithm 6y ago
  I can't for the life of me understand why this isn't at least a config option. That and an option for setting --upstream in git push by default. Those could at least preserve backwards compatibility. Are the Git maintainers so stringent on maintaining backwards compatibility that the implementation of submodules has remained in such a dire state for so long? I find it hard to believe that a VCS this widely adopted would fall over so easily for this relatively popular use case.
  - jayd16 6y ago
nemetroid 6y ago
I’ve found that setting submodule.recurse to true makes submodules behave closer to how you’d expect them to.
user-the-name 6y ago
Mercurial's subrepos implement the exact same functionality as git submodules, but they do it without being as bafflingly broken as git's attempt.

wheybags 6y ago

I've found that git subtrees are a much better option. Good intro here:https://www.atlassian.com/git/tutorials/git-subtree https://www.atlassian.com/git/tutorials/git-subtree They're like submodules, but you can edit them and they're embedded in your history, not just a reference.

rkangel 6y ago
For those who aren't familiar with them, subtrees and submodules are two different approaches to the same problem - wanting the contents of another git repo available to you. To understand the differences, imagine two solutions you might hack together: Have your build script do a 'git clone' of a specific repo into your folder structure, so that your build can reference the files. You can imagine some scripts you might write to simplify it, and you can also imagine doing it on a git hook so it happen automatically on checkout. This is git submodules. Alternatively, you could just take that other repo, copy all the files into a subfolder of your repo and check them in. When you want to update it, you need to do a bit of manual fiddling and then check in again. This is git subtrees (with commands to help you more easily do the updates). I think people create repos when they shouldn't anyway - I subscribe to a more monorepo approach with lots of things in one repo, and git subtrees are an obvious match to that.
- IshKebab 6y ago
  So git subtree is just a monorepo basically?
  - Phlogistique 6y ago
    Yes - or rather, it's a tool that helps you build a monorepo from separate repos.
  - rkangel 6y ago
    It's a monorepo, where some of the contents are a copy of another existing repo. I'm sure we've all 'vendored in' a dependency before - just copying in the files and checking them in. The fiddly bit is when you then want to update that vendored dependency to the newer version (more manual copying of files and checking in), and it gets tedious and error prone when you have 10 of them. Subtrees do the same thing, but provide tooling to make it a lot easier.

crowdhailer 6y ago

Submodules aren't perfect but we've made use of them in a few projects and one we had a process around them they were useful.

nathell 6y ago

Oh yeah, submodules can bite. At my previous job, we tried open-sourcing a part of our monorepo using them, which resulted in a clunky workflow. So we whipped up a syncing bot integrating with GitHub that would propagate changes both ways: https://github.com/WorksHub/flow-bot https://github.com/WorksHub/flow-bot

NalNezumi 6y ago

Oh I remember at previous work when the (now lead engineer) decided that the main repo that depended on actively changing 4 subrepo currently included as subtrees had to be changed to submodule. This in a rapidly developing repo where the main branch hadn't been update for 18 months while "develop" branch was switched to other "sub" branches on monthly basis. Needless to say it was a total shitshow. My first experience of how "wanting to use the cool stuff I just found out about" type of perso in position of decision making can be catastrophic. The dude wanted to switch all code base to C++20 in 2020, 3 days after the conference

sto_hristo 6y ago

My lord, so much hate for them submodules. I'm actively using them for my project, where i have common components that are shared between other components. Having them as dependencies managed by a build tool is cumbersome as sometimes, purely for the sake of convenience, i need to update some shared component in place and have the changes propagate to other dependents. Had a few mishaps, but those were the result of lack of experience. Overall, my experience with submodules is great. I can imagine what terror awaits those with bigger projects where everyone starts pushing random stuff from all directions.

robin21 6y ago
Yeh for teams they are terrible. For solo development using a single branch they can see okay.

junon 6y ago

Most of the "complaints" here are people misusing submodules or not taking the time to learn how Git itself works. I work with submodules almost daily for dependencies in game engines, OS projects, and even network services. I never have issues. However, that's not to say they have a strange set of commands (or lack thereof). I have to touch the .git folder way too often, and that shouldn't be the case. Maybe I'll write a small cookbook of how to deal with issues, including an "everything is fucked, how do I start over?" section.

BuckarooBanzay 6y ago

Despite all the comments saying to avoid submodules i can only recommend them. I'm using them in several of my game servers as a "meta-repo" that points to other git repositories (for example here: https://github.com/pandorabox-io/pandorabox-mods https://github.com/pandorabox-io/pandorabox-mods) It makes updating, finding/fixing bugs and testing much easier (we are using github's dependabot to update and kick off initial tests)

pornel 6y ago
What they achieve is very useful. How they do it is terrible. I use them too, but their "user interface" is half-assed and full of traps. I have to have a bunch of extra scripts to check and fix them, and a ton of warnings in the README for poor souls who haven't experienced the pain yet.
- nerdponx 6y ago
  "user interface" is half-assed and full of traps Welcome to Git. And least it's a lot better than it used to be.
JamesSwift 6y ago
Yes, I think as a way of vendoring external dependencies they are ok. That's about the only use case that makes sense to me. The friction they cause isn't worth it otherwise.

OskarS 6y ago

Here's the lesson I've learned from my experience with submodules in git in several different companies: avoid them like the plague. NEVER USE THEM FOR ANYTHING. Use any other solution available to you: use package managers, use monorepos, use subtrees, just copy/paste the files in, avoid the dependency entirely, do ANYTHING it takes to avoid using submodules. They just become a constant source of friction. Basically every action you do in git, there's some tiny bit of annoyance caused by submodules, which adds up to tearing your hair out. Like, read this StackOverflow question and answer, and tell me this is something you want to be dealing with on daily basis (and you will be, you'll regularly be dealing with far worse): https://stackoverflow.com/questions/9314365/git-clean-is-not-removing-a-submodule-added-to-a-branch-when-switching-branches https://stackoverflow.com/questions/9314365/git-clean-is-not... The correct way to handle dependencies in general is package managers, but if for some reason that is not available to you, and you wish to avoid git subtree for some reason, copy/pasting the code from the other repository and making a note in the commit message which commit in the dependency repo you copied from is a far preferable solution. Yeah, you lose the history of the sub-repo, but it's well worth it to avoid the complications from using submodule (and you never examine the history of the submodule anyway, how often do you look at the history of your dependencies?). Submodules: not even once.

m_eiman 6y ago
Don't use subtrees, they're even worse. At least when we tried using them for a while before moving to submodules; perhaps they're less of a broken hack these days. For example, whenever we did something with the subtree it reprocessed all commits in the repo - obviously taking longer and longer as time went on.
junon 6y ago
Disagree. Package managers cause more headache than anything. I have used submodules for almost all of my projects (aside from e.g. Node.js projects). Not once have had issues with them. I don't know why people raise such a stink about them.
- dataflow 6y ago

deleted 6y ago

[deleted]

Pxtl 6y ago

I'm using submodules in a project and I have to agree - they're awful. The fact that their ergonomics are so bad in as trivial cases as switching branches is flabbergasting. I mean, it was the Right Tool for the Job for my use-case but the experience was awful. I'm pretty well convinced that the emperor wears no clothes when it comes to Git in general.

pydry 6y ago
This has bugged me for a while. Culturally there are three types of programmer tool: * Those unknown or disliked enough that anything that goes wrong when using them is the tools fault even if it isn't. * Those well known and liked enough that they get blamed for their own issues. * Those that are put on a pedestal such that any problems they do have are the fault of the user. git falls squarely into the latter camp.
mpawelski 6y ago
Is it still that bad if you set submodule.recurse[0] config? [0] https://git-scm.com/docs/git-config#Documentation/git-config.txt-submodulerecurse https://git-scm.com/docs/git-config#Documentation/git-config...

krona 6y ago

Git submodule should be renamed 'git conway', named after Conway's Law: there is no good reason to use it except within those companies that have to compensate for dysfunctional technical leadership.

junon 6y ago
There are great reasons to use them. You've never been bitten by the alternative, it seems.

dboreham 6y ago

Yes submodules are to be avoided, the 99% of the problem is poor documentation that fails to explain submodule behavior adequately, leading the reader to assume the feature conforms to a mental model that it doesn't.

ta8645 6y ago
Do you have an example of this mismatch you could share?
- pornel 6y ago
  Users assume submodules would behave sensibly, and they don't. State of submodules is part of a commit, so you'd expect that checking out a commit checks out the submodules. It doesn't. Or if you notice git doesn't care about the content of submodules in the working tree, you'll be surprised when rebasing (change of a dir to/from submodule breaks rebase). Merging has "ours"/"theirs" startegies, except it doesn't for submodules (there's suddenly local/remote distinction, and breaks -Xtheirs in rebase). From the diffs it looks like submodules are defined in .gitmodules and the tree, so removing them from these places would delete them. It doesn't (it fucks up internal state of .git dir). And you'd think .git dir is a dir, except when it isn't. It's like whoever designed submodules just loves saying "gotcha!" and telling frustrated users they suck at git. Everything submodules do is subtly different with sharp edges.

40four 6y ago

This post should have (2009) appended to it. It's not obvious because there is no date on the post, but if you scroll down to comments, they are from 2009. That means these code examples were written in Git v1.6. The current Git release is v 2.30. I haven't inspected the commands, but I'd be willing to bet the interface has changed since then.

zoobab 6y ago

Docker hub rails to build an image if you call hit submodules in there. I had the problem with DirtyJTAG Dockerfile, build fine on my laptop, does not build on Dockerhub. I mean submodules were a pain in 2010, they are still a pain in 2021. So much for 'open source'.

deleted 6y ago

[deleted]

1MachineElf 6y ago

QMK is only one repository I ever spend time working with, and it has some submodules. I find it annoying that they must be pulled each time I switch branches. Should I look into changing them so that they at least point to a repository on my local machine instead of having to re-download from GitHub each time I want to create a new branch? Is that possible, and does anyone else do it?

matheusmoreira 6y ago

I really hate how submodules are just pointers to specific commits. This forces an update to the superproject every time I update the submodule. Even if I add commits to the submodule, it won't check out the changes until the superproject's pointer is updated. I just like to develop libraries and the application that uses them at the same time. Whenever I change the library, I want to see the changes in the application immediately. The only tool I've used that got this right is Python's setuptools. It's got a development mode which essentially replaces the package with a symbolic link to a local repository. Why doesn't git support this?

juangacovas 6y ago

Anyone can compare the "externals" in SVN with "submodules" in git? I'm curious since I'm used to externals in SVN and they're ok but as always you have to keep some things in mind

Izkata 6y ago
A git submodule is almost the same thing as an svn external that's pinned to a specific revision. From that baseline, these are the differences I can think of: * The submodules aren't automatically updated like externals are with "svn up", you need a second command. * Adding a submodule is done with a command, not editing files/props directly. * Any changes to the pinned revision are done inside the submodule (git pull, new commits, etc), which automatically changes the reference in the parent repo and then is committed in the parent repo. * Removing a submodule is two commands now, not a bunch of manual work (article is out of date: https://news.ycombinator.com/item?id=26171333 https://news.ycombinator.com/item?id=26171333 ).

rubyist5eva 6y ago

Git is an insane tool. I've never seen a single command line tool blogged about so much. You don't see anything like this with regards to Mercurial or other distributed version control systems(and most of them are just as powerful as git, if not moreso), the documentation is usually enough for these tools. For some reason git is special - it has this ability to be completely opaque to so many people and yet still be deployed almost literally everywhere. I am completely baffled by its ubiquity. Thanks Github, I guess..?

Izkata 6y ago

Note that the "2009" is quite important, a few things here are solved (though with the usual odd interface if you're not used to it). > If that repository also has submodules, then your submodule’s submodules will have to be populated by following the steps below from within your project’s submodule directory (confusing yet?). Now: git clone repo target cd target git submodule update --init --recursive > Unfortunately, this is wrong. Git does not have a built in way to remove submodules. It's a two stage process (the first cleans up .git/config, the second actually removes the code and submodule), but does exist now: git deinit lib/billboard git rm lib/billboard

tigerlily 6y ago

I want to use Dear ImGui [1] in a C++ project. Sure I can copy the relevant files into my repo by hand, but my instinct says that is not the way. Git submodule seems like the right tool, but there's plenty of comments here advising not to use it. So if not git-submodule, then what is the procedure? I just want to have the external repo as a subfolder in my own repo, updating that subfolder as updates become available. [1] https://github.com/ocornut/imgui https://github.com/ocornut/imgui

nerdponx 6y ago
Git submodule is the right tool. Git submodule works well for externally-developed "read-only" parts of your project that you need to update infrequently. Git submodules a few years ago used to be a pain in the ass, partly because the CLI was missing a lot of important functionality. Pretty much all of those concerns are in the past, and now it's just people complaining about having to pass a couple extra command line flags (which you can/should just add to your README).
- tigerlily 6y ago
  Thanks! A quick perusal of some reputable docs corroborates your recommendation [1]. [1] https://www.atlassian.com/git/tutorials/git-submodule https://www.atlassian.com/git/tutorials/git-submodule

newusertoday 6y ago

Android combines multiple git repositories with manifest(xml file) and has associated tooling for it. Not sure why that approach is not popular?

tpoacher 6y ago

Sometimes I almost feel like git is slowly trying to reinvent svn ...