4 ms·

> I have used submodules for almost all of my projects (aside from e.g. Node.js projects). Not once have had issues with them. Do you do things with git beside

> I have used submodules for almost all of my projects (aside from e.g. Node.js projects). Not once have had issues with them. Do you do things with git beside the trivial operations (add/commit/etc.)? Like rebasing/merging and such across commits that introduce or remove submodules? Do you never encounter friction when e.g. removing submodules? Do you never run into the errors people complain about?

P_I_Staker 6y ago
I absolutely despise submodules, but under many use cases you don't add or remove that often. We definitely run into errors people complain about. In fact there are errors that the average user (that doesn't really care about SVC) pretty much can't fix... or with much difficulty. I don't know that I'd still say it's a dealbreaker, but that's definitely very bad. Very much does break some of the spirit of git and what makes it "nice". We've been using them, and our system is still very usable (obviously this is a ridiculous thing to brag about, but worth pointing out). I'm wondering if there's a better way to handle shared code between two repos, where the users of either don't have access to the other parent? You might be able to use subtrees? Email source code and just create n separate commits for n projects, with a standardized commit message? I'd be happy to do something different, but I never had some idea that I could sell in a meeting, and didn't really have the energy or clout to fight this.
- sibrahim 6y ago
  My preferred solution has been git-subrepo: https://github.com/ingydotnet/git-subrepo https://github.com/ingydotnet/git-subrepo Basically is what (I think) submodules should have been. Creates a vendored copy with metadata about what commit it came from. Normal operations like clone, commit (even touching multiple main/subrepo files which subtree struggles with) are unaffected (normal files from git's point of view). Pull, push, branching all work as expected on main project (most devs don't even need to know there's a subrepo). If you want to pull/push vendored changes from subrepo, there's a new command, but that's it.
  - junon 6y ago
    This just described every feature that submodules gives you out of the box. I'm curious what value this adds, if any...
    - Izkata 6y ago
      I can't answer in general, but I do see one possible misunderstanding: "vendored" means the code is copied into the parent repo. If a submodule's original source goes offline, a new clone of the parent repo won't be able to retrieve it, since it just stores a reference to where to clone from. If subrepo really vendors the code, that'll never be a problem since a full copy is committed directly to the parent repo.
      - junon 6y ago
        This is what mirroring or forking is for...
    - sibrahim 6y ago
      As Izkata mentions, the superproject has _all_ the files for a given commit without any need for users to have access other repos, additional submodule init commands, etc. Basically, after a subrepo clone, you've copied the file tree for the subrepo and can make commits on it in the superproject to your heart's content (branching, etc). This is basically a fork/mirror, but adds a single metadata file to track the last upstream commit you pushed/pulled from to allow reconciling later. So with git subrepo, you make commits in the superproject first and can choose (at some later point, if at all) to merge with the subrepo upstream. This is arguably consistent with the git model writ large (make local commits, later choose if/how to integrate those with upstream). Importantly, people that clone your superproject repo don't have to know anything about subrepo or special commands to send changes back to you. For submodules, changes flow in the other direction. If you want to make a change to the child repo, you must 1) commit in the submodule, 2) push the submodule commit to its upstream, 3) make a commit in the super project that changes the commit the submodule is pointing to. When someone pulls, switches branches, etc in the superproject, they need to do a submodule update with various failure modes or else they end up with empty/out-of-date content. Subtrees are a bit similar to subrepos, but in practice you still need to be aware of their boundaries since you can't mix subtree/superproject modifications in the same commit. Moreover, you need to use a special merge strategy rather than git default merge/rebase which subrepo uses.
- dataflow 6y ago
  > I'd be happy to do something different, but I never had some idea that I could sell in a meeting, and didn't really have the energy or clout to fight this. Would the alternative I suggested here work? I think you might even be able to do it in tandem with submodules so you try out both and see what works for you: https://news.ycombinator.com/item?id=26165644 https://news.ycombinator.com/item?id=26165644
- junon 6y ago
  > pretty much can't fix... or with much difficulty. I don't believe this for a second.
junon 6y ago
> Do you do things with git beside the trivial operations (add/commit/etc.)? Like rebasing/merging and such across commits that introduce or remove submodules? Yes, often (I'm a stickler for clean histories on my projects). The trick is to only work with submodule refs during any sort of complex series of commands, then update the submodules once all of the ref munging is done. If you have a ref conflict or something (rare IME, unless you're running git commands haphazardly, without understanding them), the error message tells you exactly which ref it couldn't check out and why - in which case, simply cd in and fix it manually. If you need to move submodules around, `git submodule --deinit` it first, then re-add it. If you need to remove one, deinit it then make sure it's purged from .gitmodules. > Do you never encounter friction when e.g. removing submodules? Nope, though admittedly the commands need some polish here. Worst case, remove the directory in the working tree, remove the entry from .gitmodules, delete .git/module/x/y/z/, then add -A the now-deleted worktree path and your .gitmodules. > Do you never run into the errors people complain about? Not since I sat down and read through the concepts section on the official Git docs, which outline what e.g. objects, trees, commits, tags and refs were, no. Git submodules are up there on the list of the most misunderstood (but entirely useful) features ever, in my opinion.
- dataflow 6y ago
  > The trick is to only work with submodule refs during any sort of complex series of commands, then update the submodules once all of the ref munging is done. So you don't actually check out the submodule, ok. But this means you can't actually use the correct submodule commit during the rebase? Like trying to build with it to check if it works before continuing? > Nope, though admittedly the commands need some polish here. Worst case, remove the directory in the working tree, remove the entry from .gitmodules, delete .git/module/x/y/z/, then add -A the now-deleted worktree path and your .gitmodules. How do you admit this and yet simultaneously say "I don't know why people raise such a stink about them" and "Not once have had issues with them"? You somehow know there are pain points, you know the various workarounds which can consist of N hoops to jump through, and yet you claim you haven't encountered issues even once and don't even understand what people dislike about submodules? Aren't these clearly contradictory? > Not since I sat down and read through the concepts section on the official Git docs, which outline what e.g. objects, trees, commits, tags and refs were, no. I'm pretty sure I know what all of those are and still find submodules painful.

P_I_Staker 6y ago

I absolutely despise submodules, but under many use cases you don't add or remove that often. We definitely run into errors people complain about. In fact there are errors that the average user (that doesn't really care about SVC) pretty much can't fix... or with much difficulty. I don't know that I'd still say it's a dealbreaker, but that's definitely very bad. Very much does break some of the spirit of git and what makes it "nice". We've been using them, and our system is still very usable (obviously this is a ridiculous thing to brag about, but worth pointing out). I'm wondering if there's a better way to handle shared code between two repos, where the users of either don't have access to the other parent? You might be able to use subtrees? Email source code and just create n separate commits for n projects, with a standardized commit message? I'd be happy to do something different, but I never had some idea that I could sell in a meeting, and didn't really have the energy or clout to fight this.

sibrahim 6y ago
My preferred solution has been git-subrepo: https://github.com/ingydotnet/git-subrepo https://github.com/ingydotnet/git-subrepo Basically is what (I think) submodules should have been. Creates a vendored copy with metadata about what commit it came from. Normal operations like clone, commit (even touching multiple main/subrepo files which subtree struggles with) are unaffected (normal files from git's point of view). Pull, push, branching all work as expected on main project (most devs don't even need to know there's a subrepo). If you want to pull/push vendored changes from subrepo, there's a new command, but that's it.
- junon 6y ago
  This just described every feature that submodules gives you out of the box. I'm curious what value this adds, if any...
  - Izkata 6y ago
    I can't answer in general, but I do see one possible misunderstanding: "vendored" means the code is copied into the parent repo. If a submodule's original source goes offline, a new clone of the parent repo won't be able to retrieve it, since it just stores a reference to where to clone from. If subrepo really vendors the code, that'll never be a problem since a full copy is committed directly to the parent repo.
    - junon 6y ago
      This is what mirroring or forking is for...
  - sibrahim 6y ago
    As Izkata mentions, the superproject has _all_ the files for a given commit without any need for users to have access other repos, additional submodule init commands, etc. Basically, after a subrepo clone, you've copied the file tree for the subrepo and can make commits on it in the superproject to your heart's content (branching, etc). This is basically a fork/mirror, but adds a single metadata file to track the last upstream commit you pushed/pulled from to allow reconciling later. So with git subrepo, you make commits in the superproject first and can choose (at some later point, if at all) to merge with the subrepo upstream. This is arguably consistent with the git model writ large (make local commits, later choose if/how to integrate those with upstream). Importantly, people that clone your superproject repo don't have to know anything about subrepo or special commands to send changes back to you. For submodules, changes flow in the other direction. If you want to make a change to the child repo, you must 1) commit in the submodule, 2) push the submodule commit to its upstream, 3) make a commit in the super project that changes the commit the submodule is pointing to. When someone pulls, switches branches, etc in the superproject, they need to do a submodule update with various failure modes or else they end up with empty/out-of-date content. Subtrees are a bit similar to subrepos, but in practice you still need to be aware of their boundaries since you can't mix subtree/superproject modifications in the same commit. Moreover, you need to use a special merge strategy rather than git default merge/rebase which subrepo uses.
dataflow 6y ago
> I'd be happy to do something different, but I never had some idea that I could sell in a meeting, and didn't really have the energy or clout to fight this. Would the alternative I suggested here work? I think you might even be able to do it in tandem with submodules so you try out both and see what works for you: https://news.ycombinator.com/item?id=26165644 https://news.ycombinator.com/item?id=26165644
junon 6y ago
> pretty much can't fix... or with much difficulty. I don't believe this for a second.

junon 6y ago

> Do you do things with git beside the trivial operations (add/commit/etc.)? Like rebasing/merging and such across commits that introduce or remove submodules? Yes, often (I'm a stickler for clean histories on my projects). The trick is to only work with submodule refs during any sort of complex series of commands, then update the submodules once all of the ref munging is done. If you have a ref conflict or something (rare IME, unless you're running git commands haphazardly, without understanding them), the error message tells you exactly which ref it couldn't check out and why - in which case, simply cd in and fix it manually. If you need to move submodules around, `git submodule --deinit` it first, then re-add it. If you need to remove one, deinit it then make sure it's purged from .gitmodules. > Do you never encounter friction when e.g. removing submodules? Nope, though admittedly the commands need some polish here. Worst case, remove the directory in the working tree, remove the entry from .gitmodules, delete .git/module/x/y/z/, then add -A the now-deleted worktree path and your .gitmodules. > Do you never run into the errors people complain about? Not since I sat down and read through the concepts section on the official Git docs, which outline what e.g. objects, trees, commits, tags and refs were, no. Git submodules are up there on the list of the most misunderstood (but entirely useful) features ever, in my opinion.

dataflow 6y ago
> The trick is to only work with submodule refs during any sort of complex series of commands, then update the submodules once all of the ref munging is done. So you don't actually check out the submodule, ok. But this means you can't actually use the correct submodule commit during the rebase? Like trying to build with it to check if it works before continuing? > Nope, though admittedly the commands need some polish here. Worst case, remove the directory in the working tree, remove the entry from .gitmodules, delete .git/module/x/y/z/, then add -A the now-deleted worktree path and your .gitmodules. How do you admit this and yet simultaneously say "I don't know why people raise such a stink about them" and "Not once have had issues with them"? You somehow know there are pain points, you know the various workarounds which can consist of N hoops to jump through, and yet you claim you haven't encountered issues even once and don't even understand what people dislike about submodules? Aren't these clearly contradictory? > Not since I sat down and read through the concepts section on the official Git docs, which outline what e.g. objects, trees, commits, tags and refs were, no. I'm pretty sure I know what all of those are and still find submodules painful.