3 ms·

Anyone know, what's the advantage of this over a big composite repo with several git submdolues? I think that submodules are better suited for separation of co

Anyone know, what's the advantage of this over a big composite repo with several git submdolues? I think that submodules are better suited for separation of concerns and performance, even while achieving the same composite structure as an equivalent monorepo?

tantalor 3y ago
Monorepo allows a single commit to update across components, eg an API change
- steffres 3y ago
  for each submodule affected by some change you would need an additional commits, yes. But those commits are bundled together in the commit of the parent repo where they act as one. So, atomicity of changes can be guaranteed, but you need to write a few more commits. However this effort of small increases of commits is far outweighed by the modularity imo.
  - marksomnian 3y ago
    Is it? I'm slightly struggling to understand what benefit you gain from having the "parent" repo but also having individual submodules. Sure, working in each individual project's module makes cloning faster, until you need to work on a module that references another module (at which point you need to check out the parent repo or risk using the wrong version), and now every change you make needs two commits (one to the sub-repo, and one to the base to bump the submodule reference),
    - steffres 3y ago
      In our case, we have a codebase that involves two submodules: one for persistence and one for python based management of internal git repos. Both of these are standalone applications and can run on their own. They are then used in a parent repo which represents the overarching architecture, which calls into the submodules. The advantage of this is, that work can be done by devs on the individual modules without much knowledge of the overarching architecture, nor strong code ties into it. Right now our persistence is done with SQL, but we could swap it with anything else, e.g. mongo, and the parent codebase wouldn't notice a thing since the submodule only returns well defined python objects. Of course, this comes at the cost of higher number of commits as you mentioned. But in my opinion these are still cheap because they only affect trivial quantity and not brain-demanding quality.
      - marksomnian 3y ago
        But what do you do as soon as one of the submodules has a dependency on another? I imagine you might not hit it in your simple case, but I feel like scenarios like that are where the advantages of monorepos lie. To take a concrete example, I'm working on a codebase that houses both a Node.js server-side application and an Electron app that communicates with it (using tRPC [0]). The Electron app can directly import the API router types from the Node app, thus gaining full type safety, and whenever the backend API is changed the Electron app can be updated at the same time (or type checks in CI will fail). If this weren't in a monorepo, you would need to first update the Node app, then pick up those changes in the Electron app. This becomes risky in the presence of automated deployment, because, if the Node app's changes accidentally introduced a breaking API change, the Electron app is now broken until the changes are picked up. In a monorepo you'd spot this scenario right away. (Mind you, there is still the issue of updating the built Electron app on the users' machines, but the point remains - you can easily imagine a JS SPA or some other downstream dependency in its place.) [0]: https://trpc.io/ https://trpc.io/
        steffres 3y ago
        yes, if one submodule would depend on another, this would cause problems indeed. So far, we could avoid it though, by strict encapsulation. But I definitely see the point in your example and wouldn't follow through with submodules there probably too. It's just that in OP's link, I'm quite sceptical as the monorepo approach requires quite some heavy tweaking.
  - tantalor 3y ago
    > this effort of small increases of commits is far outweighed by the modularity Not remotely, as the scale of the codebase increases, the benefit of modularity goes to zero and the benefit of atomic changes increases. Also: it's not always feasible to break up a change into smaller commits. Sometimes atomic change is the only way to do it.
    - crabbone 3y ago
      With --recurse-submodules the atomicity doesn't seem to suffer. It used to be the case that you couldn't ensure all changes in the source tree couldn't be pushed atomically, now you can, but I'm not sure it's the default behavior.
  - crabbone 3y ago
    I missed the git push --recurse-submodules flag, even though it seems like it's been there for a long time. Yeah, it seems like it would work, except you need to configure it to be always "check" and be always on when you push.
aseipp 3y ago
The advantage is simple: Git submodules suck and are a chore to manage for any dependency that sees remotely high traffic or requires frequent synchronization. As the number of developers, submodules, and synchronization requirements increase, this pain increases dramatically. Basic git features, like cherry picking and bisecting to find errors become dramatically worse. You cannot even run `git checkout` without potentially introducing an error, because you might need to update the submodule! All your most basic commands become worse. I have worked on and helped maintain projects with 10+ submodules, and they were one of the most annoying, constantly problematic pain points of the entire project, that every single developer screwed up repeatedly, whether they were established contributors or new ones. We had to finally give in and start using pre-push hooks to ban people from touching submodules without specific commit message patterns. And every single time we eliminated a submodule -- mostly by merging them and their history into the base project, where they belonged anyway -- people were happier, development speed increased, and people made less errors. The reasons for those things being separate projects had a history (dating to a time before Git was popular, even) and can be explained, but ultimately it doesn't matter; by the time I was around, all of those reasons ceased to exist or were simply not important. I will personally never, ever, ever, ever allow Git submodules in any project I manage unless they are both A) extremely low traffic, so updating them constantly doesn't suck, and B) a completely external dependency that is mostly outside of my control, that cannot be managed any other way. Save yourself hair and time and at least use worktrees instead.

tantalor 3y ago

Monorepo allows a single commit to update across components, eg an API change

steffres 3y ago
for each submodule affected by some change you would need an additional commits, yes. But those commits are bundled together in the commit of the parent repo where they act as one. So, atomicity of changes can be guaranteed, but you need to write a few more commits. However this effort of small increases of commits is far outweighed by the modularity imo.
- marksomnian 3y ago
  Is it? I'm slightly struggling to understand what benefit you gain from having the "parent" repo but also having individual submodules. Sure, working in each individual project's module makes cloning faster, until you need to work on a module that references another module (at which point you need to check out the parent repo or risk using the wrong version), and now every change you make needs two commits (one to the sub-repo, and one to the base to bump the submodule reference),
  - steffres 3y ago
    In our case, we have a codebase that involves two submodules: one for persistence and one for python based management of internal git repos. Both of these are standalone applications and can run on their own. They are then used in a parent repo which represents the overarching architecture, which calls into the submodules. The advantage of this is, that work can be done by devs on the individual modules without much knowledge of the overarching architecture, nor strong code ties into it. Right now our persistence is done with SQL, but we could swap it with anything else, e.g. mongo, and the parent codebase wouldn't notice a thing since the submodule only returns well defined python objects. Of course, this comes at the cost of higher number of commits as you mentioned. But in my opinion these are still cheap because they only affect trivial quantity and not brain-demanding quality.
    - marksomnian 3y ago
      But what do you do as soon as one of the submodules has a dependency on another? I imagine you might not hit it in your simple case, but I feel like scenarios like that are where the advantages of monorepos lie. To take a concrete example, I'm working on a codebase that houses both a Node.js server-side application and an Electron app that communicates with it (using tRPC [0]). The Electron app can directly import the API router types from the Node app, thus gaining full type safety, and whenever the backend API is changed the Electron app can be updated at the same time (or type checks in CI will fail). If this weren't in a monorepo, you would need to first update the Node app, then pick up those changes in the Electron app. This becomes risky in the presence of automated deployment, because, if the Node app's changes accidentally introduced a breaking API change, the Electron app is now broken until the changes are picked up. In a monorepo you'd spot this scenario right away. (Mind you, there is still the issue of updating the built Electron app on the users' machines, but the point remains - you can easily imagine a JS SPA or some other downstream dependency in its place.) [0]: https://trpc.io/ https://trpc.io/
      - steffres 3y ago
        yes, if one submodule would depend on another, this would cause problems indeed. So far, we could avoid it though, by strict encapsulation. But I definitely see the point in your example and wouldn't follow through with submodules there probably too. It's just that in OP's link, I'm quite sceptical as the monorepo approach requires quite some heavy tweaking.
- tantalor 3y ago
  > this effort of small increases of commits is far outweighed by the modularity Not remotely, as the scale of the codebase increases, the benefit of modularity goes to zero and the benefit of atomic changes increases. Also: it's not always feasible to break up a change into smaller commits. Sometimes atomic change is the only way to do it.
  - crabbone 3y ago
    With --recurse-submodules the atomicity doesn't seem to suffer. It used to be the case that you couldn't ensure all changes in the source tree couldn't be pushed atomically, now you can, but I'm not sure it's the default behavior.
- crabbone 3y ago
  I missed the git push --recurse-submodules flag, even though it seems like it's been there for a long time. Yeah, it seems like it would work, except you need to configure it to be always "check" and be always on when you push.

aseipp 3y ago

The advantage is simple: Git submodules suck and are a chore to manage for any dependency that sees remotely high traffic or requires frequent synchronization. As the number of developers, submodules, and synchronization requirements increase, this pain increases dramatically. Basic git features, like cherry picking and bisecting to find errors become dramatically worse. You cannot even run `git checkout` without potentially introducing an error, because you might need to update the submodule! All your most basic commands become worse. I have worked on and helped maintain projects with 10+ submodules, and they were one of the most annoying, constantly problematic pain points of the entire project, that every single developer screwed up repeatedly, whether they were established contributors or new ones. We had to finally give in and start using pre-push hooks to ban people from touching submodules without specific commit message patterns. And every single time we eliminated a submodule -- mostly by merging them and their history into the base project, where they belonged anyway -- people were happier, development speed increased, and people made less errors. The reasons for those things being separate projects had a history (dating to a time before Git was popular, even) and can be explained, but ultimately it doesn't matter; by the time I was around, all of those reasons ceased to exist or were simply not important. I will personally never, ever, ever, ever allow Git submodules in any project I manage unless they are both A) extremely low traffic, so updating them constantly doesn't suck, and B) a completely external dependency that is mostly outside of my control, that cannot be managed any other way. Save yourself hair and time and at least use worktrees instead.