5 ms·

How far does this have to go before people realize how ridiculous this is. The Multi-mono-multi-monorepo stage? You don't need a monorepo. It's an anti-pattern

How far does this have to go before people realize how ridiculous this is. The Multi-mono-multi-monorepo stage? You don't need a monorepo. It's an anti-pattern that people resort to when their code is a tangled, tightly coupled mess. If the modules are loosely coupled and high cohesion, multiple repos is the ideal approach. The solution to the problem of having to 'constantly update dependencies' is not to bring them all into a single monorepo. The solution is to ensure that these dependencies handle separate concerns and have simple interfaces which allow them to be loosely coupled with the main project logic in such a way that they can be updated independently of each other. If different module dependencies often need to be updated together whenever you add a new feature or fix a bug, it almost certainly means that you have a problem with coupling and/or cohesion. You don't need tools that make it easier to work with a tangled mess. The correct solution is to untangle the mess. Otherwise the mess will keep getting worse.

TechBro8615 5y ago
What if you want to publish some of your modules and keep the rest private but depending on the published ones?
choeger 5y ago
Tightly coupled code is, and should be, the norm. Loose coupling is a pipe dream as soon as domain-specific data structures are involved. Loose coupling over domain-specific data structures (and their related procedures) just means that errors that should be caught by a run-of-the-mill type checker are now found in production. An example of this pattern can be seen pretty much whenever you open the developer console in a non-chrome browser on any site using some JavaScript framework. Downloading some code from the website and executing in the browser is an extreme form of loose coupling and by my experience it is pretty much never free of errors, except for the very specific combinations of browser/platform that the developer could test.
- ragebol 5y ago
  You can have the procedural/functional code only tightly coupled to the datastructures but loose coupling between the other bits.
- Zababa 5y ago
  You can also see the difficulty of loose coupling when people transition from a monolith to microservices. What used to be a function call is now a network request that can fail in all sorts of interesting ways.
- cryptica 5y ago
  This line of thinking goes against not only everything that the original inventors of programming languages ever taught, but it goes against almost 20 years of my own professional experience... And I've written and worked on many highly maintainable projects which had loose coupling (including in some very complex business domains). In fact, they're the only 2 rules that have consistently delivered value across all the languages that I've tried and for all different kinds of system that I've built. Closest thing possible to a silver bullet. I find this kind of mindset that x is not possible to be defeatist. Instead of acknowledging that programming is difficult and can take many decades to master, people prefer to pretend that programming is easy but unavoidably messy. People always try to come up with narratives which make it easier to accept mediocrity rather than work hard to keep improving themselves.
exitheone 5y ago
I think I could not disagree more. Coupling and cohesion are problems you will have, regardless of the repo approach you take. But a mono-repo gives you a host of incredibly beneficial advantages, especially for larger organisations. Any downstream library team can instantly tell if their change has broken any upstream users because a good CI system will just automatically run all tests of all relevant projects. That way a library developer can either iterate until all projects depending on it are green, or the library developer can even proactively change all dependant projects and roll all fixes out in a single atomic commit. This has proven incredibly beneficial to our development speed and the number of avoidable code conflicts that crop up.
- adev_ 5y ago
  > Any downstream library team can instantly tell if their change has broken any upstream users because a good CI system will just automatically run all tests of all relevant projects. That way a library developer can either iterate until all projects depending on it are green, or the library developer can even proactively change all dependant projects and roll all fixes out in a single atomic commit. Any proper (recent) package manager will do the same with a good CI without having any of the mono-repo disadvantages ( duplicated libraries, diamond dependency mess ). The only real advantage of mono repo is that editing simultaneously multiple software components with API breaking changes is made easier, much easier.
  - kronin 5y ago
    Please share the package managers that handle this, thanks!
    - adev_ 5y ago
      Nix, Guix or Spack do that correctly.
mtone 5y ago
In the JavaScript ecosystem, most the challenges with repos are not the code but the tooling. SOLID principles are not going to solve that. In our monorepo, few changes are cross module. But its a grand day to reduce total package count by 20 by finally getting rid of some old project dependencies and being able to use a single test runner config and version across packages. I keep my sanity that way. Meanwhile, our non-JS repos haven’t required as much attention in this area.
xorcist 5y ago
I see this thinking frequently, it is really is a fundamental issue how differently we design software. Everyone starts with an idea how how the software should turn out. A vision where there often is remarkable agreement among people, software should be modular, independent parts encapsulated in some way, and easy to change over time. Then some people starts from this vision by focusing on the ideals. Since stateless software is so much easier to reason about, let's build our architecture on that it should be. The same goes for other ideals such as side effect-free and idempotency. Any parts that deviate from this vision are dirty little special cases and can be treated as such. But software without state is useless. State, and side effects, are the whole reason for the software to exist. Some people take this second approach and starts with the state and side effects, how these are represented and stored and how to allow for change over time. Then the rest of the software, the easy parts if you will, is sketched out after that to accommodate to this design. These tends to be the same people who starts thinking in data structures and thinks the design of data is more important than the design of code. Just as an example, sometimes I see people with monstrous Kubernetes-style architecture for a web app and then all state shoved in a Postgres in the back without even a thought. Well, in reality that's your whole application right there, in the back. There are a million ways to start stateless web workers, all perfectly fine, that's not where your energy should be spent. Maybe the above is a simplification. I know for certain that I am in the latter camp. But over the years where I have found myself in disagreement over architecture, it is often with people I have later come to see as in the first camp. And this keeps coming back again and again, on all levels of software architecture.
rkangel 5y ago
The problem is not when you need to change the implementation of a module, it's when you need to move the boundaries between them. What you are describing is a reasonable approach if you can get the module boundaries right in advance. For more open ended software problems, that is impossible - you need the freedom to rearchitect and change responsibilities. That's the situation where a monorepo is much simpler than anything else.
lhnz 5y ago
> The solution is to ensure that these dependencies handle > separate concerns and have simple interfaces which allow > them to be loosely coupled with the main project logic > in such a way that they can be updated independently of > each other. If your requirements ever change, you will need to make changes to these simple interfaces. At this point you will have to make changes to multiple repositories. Some changes can be done without breaking changes, but sometimes a change must be a breaking change. A 'single' monorepo makes your problem easily solvable using automation and code. You can use a multirepo if you prefer to solve problems using human coordination.
- cryptica 5y ago
  I've worked on several projects/codebases which could handle requirement changes whilst rarely requiting changing the interfaces of dependencies. It's not easy to design a system this way, but it's possible and it's a worthy goal to aim for. Making it out as if it's simply not possible is not only mediocre, it's untrue.
  - lhnz 5y ago
    It would be mediocre if I suggested that all changes are breaking changes and didn't know about additive changes, deprecations, or (kelvin) versioning. If an API turns out to be insecure and has to be removed or a mitigation put in place that changes it's behaviour that can be an unavoidable breaking change. Either way, my point wasn't that "breaking changes always have to be made" but "avoiding making breaking changes because everything has to be deployed granularly slows you down and so does needing to make the same change in N places".
throw_m239339 5y ago
The issue to me is more like Git is terrible for some enterprise use cases yet people try to force a square object into a round shaped hole. Git isn't the best tool for THAT job. I've been saying the same thing for 10 years now, nobody wants to listen because a lot of developers only know Git.

TechBro8615 5y ago

What if you want to publish some of your modules and keep the rest private but depending on the published ones?

choeger 5y ago

Tightly coupled code is, and should be, the norm. Loose coupling is a pipe dream as soon as domain-specific data structures are involved. Loose coupling over domain-specific data structures (and their related procedures) just means that errors that should be caught by a run-of-the-mill type checker are now found in production. An example of this pattern can be seen pretty much whenever you open the developer console in a non-chrome browser on any site using some JavaScript framework. Downloading some code from the website and executing in the browser is an extreme form of loose coupling and by my experience it is pretty much never free of errors, except for the very specific combinations of browser/platform that the developer could test.

ragebol 5y ago
You can have the procedural/functional code only tightly coupled to the datastructures but loose coupling between the other bits.
Zababa 5y ago
You can also see the difficulty of loose coupling when people transition from a monolith to microservices. What used to be a function call is now a network request that can fail in all sorts of interesting ways.
cryptica 5y ago
This line of thinking goes against not only everything that the original inventors of programming languages ever taught, but it goes against almost 20 years of my own professional experience... And I've written and worked on many highly maintainable projects which had loose coupling (including in some very complex business domains). In fact, they're the only 2 rules that have consistently delivered value across all the languages that I've tried and for all different kinds of system that I've built. Closest thing possible to a silver bullet. I find this kind of mindset that x is not possible to be defeatist. Instead of acknowledging that programming is difficult and can take many decades to master, people prefer to pretend that programming is easy but unavoidably messy. People always try to come up with narratives which make it easier to accept mediocrity rather than work hard to keep improving themselves.

exitheone 5y ago

I think I could not disagree more. Coupling and cohesion are problems you will have, regardless of the repo approach you take. But a mono-repo gives you a host of incredibly beneficial advantages, especially for larger organisations. Any downstream library team can instantly tell if their change has broken any upstream users because a good CI system will just automatically run all tests of all relevant projects. That way a library developer can either iterate until all projects depending on it are green, or the library developer can even proactively change all dependant projects and roll all fixes out in a single atomic commit. This has proven incredibly beneficial to our development speed and the number of avoidable code conflicts that crop up.

adev_ 5y ago
> Any downstream library team can instantly tell if their change has broken any upstream users because a good CI system will just automatically run all tests of all relevant projects. That way a library developer can either iterate until all projects depending on it are green, or the library developer can even proactively change all dependant projects and roll all fixes out in a single atomic commit. Any proper (recent) package manager will do the same with a good CI without having any of the mono-repo disadvantages ( duplicated libraries, diamond dependency mess ). The only real advantage of mono repo is that editing simultaneously multiple software components with API breaking changes is made easier, much easier.
- kronin 5y ago
  Please share the package managers that handle this, thanks!
  - adev_ 5y ago
    Nix, Guix or Spack do that correctly.

mtone 5y ago

In the JavaScript ecosystem, most the challenges with repos are not the code but the tooling. SOLID principles are not going to solve that. In our monorepo, few changes are cross module. But its a grand day to reduce total package count by 20 by finally getting rid of some old project dependencies and being able to use a single test runner config and version across packages. I keep my sanity that way. Meanwhile, our non-JS repos haven’t required as much attention in this area.

xorcist 5y ago

I see this thinking frequently, it is really is a fundamental issue how differently we design software. Everyone starts with an idea how how the software should turn out. A vision where there often is remarkable agreement among people, software should be modular, independent parts encapsulated in some way, and easy to change over time. Then some people starts from this vision by focusing on the ideals. Since stateless software is so much easier to reason about, let's build our architecture on that it should be. The same goes for other ideals such as side effect-free and idempotency. Any parts that deviate from this vision are dirty little special cases and can be treated as such. But software without state is useless. State, and side effects, are the whole reason for the software to exist. Some people take this second approach and starts with the state and side effects, how these are represented and stored and how to allow for change over time. Then the rest of the software, the easy parts if you will, is sketched out after that to accommodate to this design. These tends to be the same people who starts thinking in data structures and thinks the design of data is more important than the design of code. Just as an example, sometimes I see people with monstrous Kubernetes-style architecture for a web app and then all state shoved in a Postgres in the back without even a thought. Well, in reality that's your whole application right there, in the back. There are a million ways to start stateless web workers, all perfectly fine, that's not where your energy should be spent. Maybe the above is a simplification. I know for certain that I am in the latter camp. But over the years where I have found myself in disagreement over architecture, it is often with people I have later come to see as in the first camp. And this keeps coming back again and again, on all levels of software architecture.

rkangel 5y ago

The problem is not when you need to change the implementation of a module, it's when you need to move the boundaries between them. What you are describing is a reasonable approach if you can get the module boundaries right in advance. For more open ended software problems, that is impossible - you need the freedom to rearchitect and change responsibilities. That's the situation where a monorepo is much simpler than anything else.

lhnz 5y ago

> The solution is to ensure that these dependencies handle > separate concerns and have simple interfaces which allow > them to be loosely coupled with the main project logic > in such a way that they can be updated independently of > each other. If your requirements ever change, you will need to make changes to these simple interfaces. At this point you will have to make changes to multiple repositories. Some changes can be done without breaking changes, but sometimes a change must be a breaking change. A 'single' monorepo makes your problem easily solvable using automation and code. You can use a multirepo if you prefer to solve problems using human coordination.

cryptica 5y ago
I've worked on several projects/codebases which could handle requirement changes whilst rarely requiting changing the interfaces of dependencies. It's not easy to design a system this way, but it's possible and it's a worthy goal to aim for. Making it out as if it's simply not possible is not only mediocre, it's untrue.
- lhnz 5y ago
  It would be mediocre if I suggested that all changes are breaking changes and didn't know about additive changes, deprecations, or (kelvin) versioning. If an API turns out to be insecure and has to be removed or a mitigation put in place that changes it's behaviour that can be an unavoidable breaking change. Either way, my point wasn't that "breaking changes always have to be made" but "avoiding making breaking changes because everything has to be deployed granularly slows you down and so does needing to make the same change in N places".

throw_m239339 5y ago

The issue to me is more like Git is terrible for some enterprise use cases yet people try to force a square object into a round shaped hole. Git isn't the best tool for THAT job. I've been saying the same thing for 10 years now, nobody wants to listen because a lot of developers only know Git.