5 ms·

What is the beneficial distinction between this and composing the monorepo with git submodules? I have been doing that in my codebase after suffering from all t

What is the beneficial distinction between this and composing the monorepo with git submodules? I have been doing that in my codebase after suffering from all the regrets of attempting to emulate npm package releases of my modules. I feel that utilizing submodules feels more pure and conveys isolation and separation. It feels like I am not grokking something vital about why Monorepos > Repo w/ git submodules.

n42 4y ago
if submodules work for you, there’s no point in changing how you’re working. that said, monorepos solve a certain set of real problems for organizations by allowing contributions across code bases from one VCS repository, and by reframing the release/deploy pipeline around singular atomic refs. the impact is multiplicative and happens in downstream tooling and processes. they squash some problems in one place in the org (product development) that bubble up elsewhere (devops/release engineering). depending on the org this is a sensible/cost effective approach. submodules don’t really accomplish either of those things, unless I’m missing something about your workflow.
- blorenz 4y ago
  Our org is a very small team. The biggest boon is the capability to create cohesive experiences and UI in a small handful of applications that are in isolated codebases by sharing Component libraries. I also use it for improved DX with up-to-date scrubbed and sanitized production data to ensure we aren't breaking things utilizing simplified test data.
TAKEMYMONEY 4y ago
Is it literally just your codebase or a shared one? Generally with submodules we run into issues like each dev having to maintain the setup on their own machine, and unique commands to work with module repositories. For WAY more writeup: https://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/ https://codingkilledthecat.wordpress.com/2012/04/28/why-your... Whole tools were created to get around these issues (git-subtree for ex).
- blorenz 4y ago
  My codebase. I haven't experienced any pains as in this blog post. The biggest issue is to know about `git submodule init` and `git submodule update` for the one-time commands that have to be run. After that, each submodules' branches can be isolated and your traditional `git pull` will incorporate branch changes. When I cut a release to go to prod, each submodule has its SHA reflected in that.
- paulmd 4y ago
  This isn't the only place this pops up - commit hooks are another example, since they live in .git they don't travel with the repo when it is cloned (or maybe when it is updated, don't remember the specifics). I really feel there needs to be like a git --pull-config option or something to pull all project configs (including submodules, commit hooks, etc). Or perhaps move those things into the top-level folder and allow them to be git-add'd.
  - toastal 4y ago
    You can easily make a `.githooks` directory and have your docs suggest running `git config --local core.hooksPath .githooks`. This could have security concerns in open source though as users should rightfully be skeptical of running arbitrary code if malicious code is in there (similar to the security issue of your shell hooking into git info in directories and an untarred archive containing `.git` with malicious code in it). However, at least if it's in the docs, it's consensual unlike tools like Husky that inject scripts at runtime as well as creating yet another NPM module to audit.
IndepAmoeba 4y ago
The biggest advantage I've found is being able to change library code AND all usages of the library in the same commit. Say you've got three projects: - library - app-a - app-b and you need to make a change to `library` to support some new thing in `app-a`. If you can publish a new version of `library` and update `app-a` to use it, it's really easy to make a change that's incompatible with `app-b`. Even with a comprehensive `library` test suite, it's easy for Hyrum's law to make an appearance and now you've got some unexpected corner case that's depended on. With a monorepo, you can immediately see `app-b`'s tests failing and either fix the usage, or re-think your `library` changes.
- blorenz 4y ago
  This is true. There is definitely a subsequent commit that is required to push those newly created SHAs.
skybrian 4y ago
When you fix a bug in a library, is it important for all your apps to get that bugfix right away? Do you want to run all tests and fix any apps you broke immediately? If so, you want a monorepo. If you're fine with apps using an older version of the library then you don't need this. However, you might want to think about what you'd do about a security bug when many apps are using old versions of a library and it would take a lot of work to bring them up to date. For the hobbyist development I do now, I'm fine with old apps using out of date libraries. I'm mostly not maintaining the apps and I'm not writing shared libraries.
tsimionescu 4y ago
The biggest gain I get from monorepos is being able to branch many inter-dependendent projects in one command. This is especially valuable when you have a chain of dependencies (A -> B -> C ->... Z). With package-based dependencies living in individual repos, this is tedious work - branch A, modify package.json, build, branch B, update A version to new build, give B new version number, build,..., branch Z. Submodules don't particularly help. In contrast, with a monorepo and dependencies based on your local copies, you just create a new branch of the repo, and now everything is automatically pointing to the branched code. The benefits can be major. If you also integrate something to generate nice branched version numbers, things living outside the monorepo won't even notice - you branch, then you start a new build for each library, and any external user can get the new version, while you still get the simple branching support.
- blorenz 4y ago
  Great points here. We probably have a non-conformist flow when, even though the package deps are defined in the projects that aren't stand-alone, we define them and utilize them in the root projects. This has potential to footgun us. I have seen several articles addressing the folder/file structure. The author has done an excellent job and taken it a bit further. Now I have questions to research about deployment. What are best practice for isolation of code for Docker images which are to become k8s Deployments and Pods? Does it matter to stuff it all into a mega image? Are Monoimages a thing and what susceptibilities to performance do these yield?
sevenf0ur 4y ago
I imagine it can get annoying to create another git repo every single time you want to share code between some projects. It becomes an even bigger pain when you have to update more than one repo at the same time. A monorepo makes it easier to keep versions in sync and encourages code reuse because it's all in one place.
- blorenz 4y ago
  That does sound annoying. Though, in our reality it is only a half dozen evergreen projects and applications which are incorporated this way. I do imagine it will grow over time.

n42 4y ago

if submodules work for you, there’s no point in changing how you’re working. that said, monorepos solve a certain set of real problems for organizations by allowing contributions across code bases from one VCS repository, and by reframing the release/deploy pipeline around singular atomic refs. the impact is multiplicative and happens in downstream tooling and processes. they squash some problems in one place in the org (product development) that bubble up elsewhere (devops/release engineering). depending on the org this is a sensible/cost effective approach. submodules don’t really accomplish either of those things, unless I’m missing something about your workflow.

blorenz 4y ago
Our org is a very small team. The biggest boon is the capability to create cohesive experiences and UI in a small handful of applications that are in isolated codebases by sharing Component libraries. I also use it for improved DX with up-to-date scrubbed and sanitized production data to ensure we aren't breaking things utilizing simplified test data.

TAKEMYMONEY 4y ago

Is it literally just your codebase or a shared one? Generally with submodules we run into issues like each dev having to maintain the setup on their own machine, and unique commands to work with module repositories. For WAY more writeup: https://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/ https://codingkilledthecat.wordpress.com/2012/04/28/why-your... Whole tools were created to get around these issues (git-subtree for ex).

blorenz 4y ago
My codebase. I haven't experienced any pains as in this blog post. The biggest issue is to know about `git submodule init` and `git submodule update` for the one-time commands that have to be run. After that, each submodules' branches can be isolated and your traditional `git pull` will incorporate branch changes. When I cut a release to go to prod, each submodule has its SHA reflected in that.
paulmd 4y ago
This isn't the only place this pops up - commit hooks are another example, since they live in .git they don't travel with the repo when it is cloned (or maybe when it is updated, don't remember the specifics). I really feel there needs to be like a git --pull-config option or something to pull all project configs (including submodules, commit hooks, etc). Or perhaps move those things into the top-level folder and allow them to be git-add'd.
- toastal 4y ago
  You can easily make a `.githooks` directory and have your docs suggest running `git config --local core.hooksPath .githooks`. This could have security concerns in open source though as users should rightfully be skeptical of running arbitrary code if malicious code is in there (similar to the security issue of your shell hooking into git info in directories and an untarred archive containing `.git` with malicious code in it). However, at least if it's in the docs, it's consensual unlike tools like Husky that inject scripts at runtime as well as creating yet another NPM module to audit.

IndepAmoeba 4y ago

The biggest advantage I've found is being able to change library code AND all usages of the library in the same commit. Say you've got three projects: - library - app-a - app-b and you need to make a change to `library` to support some new thing in `app-a`. If you can publish a new version of `library` and update `app-a` to use it, it's really easy to make a change that's incompatible with `app-b`. Even with a comprehensive `library` test suite, it's easy for Hyrum's law to make an appearance and now you've got some unexpected corner case that's depended on. With a monorepo, you can immediately see `app-b`'s tests failing and either fix the usage, or re-think your `library` changes.

blorenz 4y ago
This is true. There is definitely a subsequent commit that is required to push those newly created SHAs.

skybrian 4y ago

When you fix a bug in a library, is it important for all your apps to get that bugfix right away? Do you want to run all tests and fix any apps you broke immediately? If so, you want a monorepo. If you're fine with apps using an older version of the library then you don't need this. However, you might want to think about what you'd do about a security bug when many apps are using old versions of a library and it would take a lot of work to bring them up to date. For the hobbyist development I do now, I'm fine with old apps using out of date libraries. I'm mostly not maintaining the apps and I'm not writing shared libraries.

tsimionescu 4y ago

The biggest gain I get from monorepos is being able to branch many inter-dependendent projects in one command. This is especially valuable when you have a chain of dependencies (A -> B -> C ->... Z). With package-based dependencies living in individual repos, this is tedious work - branch A, modify package.json, build, branch B, update A version to new build, give B new version number, build,..., branch Z. Submodules don't particularly help. In contrast, with a monorepo and dependencies based on your local copies, you just create a new branch of the repo, and now everything is automatically pointing to the branched code. The benefits can be major. If you also integrate something to generate nice branched version numbers, things living outside the monorepo won't even notice - you branch, then you start a new build for each library, and any external user can get the new version, while you still get the simple branching support.

blorenz 4y ago
Great points here. We probably have a non-conformist flow when, even though the package deps are defined in the projects that aren't stand-alone, we define them and utilize them in the root projects. This has potential to footgun us. I have seen several articles addressing the folder/file structure. The author has done an excellent job and taken it a bit further. Now I have questions to research about deployment. What are best practice for isolation of code for Docker images which are to become k8s Deployments and Pods? Does it matter to stuff it all into a mega image? Are Monoimages a thing and what susceptibilities to performance do these yield?

sevenf0ur 4y ago

I imagine it can get annoying to create another git repo every single time you want to share code between some projects. It becomes an even bigger pain when you have to update more than one repo at the same time. A monorepo makes it easier to keep versions in sync and encourages code reuse because it's all in one place.

blorenz 4y ago
That does sound annoying. Though, in our reality it is only a half dozen evergreen projects and applications which are incorporated this way. I do imagine it will grow over time.