4 ms·

To those who are swinging towards monorepos, I don't think that is a good solution. The reason being is that developers simply cannot be trusted to "do the righ

To those who are swinging towards monorepos, I don't think that is a good solution. The reason being is that developers simply cannot be trusted to "do the right thing" on data and module boundaries. Someone comes in new to the project and does something they don't know they shouldn't. It is the honor system backed by weak linters and tooling. In our monorepo, everyone passes around django orm objects and boundaries are practically non-existent. N+1 queries abound. Tests are full of patching and mocking and are _slow_. Our build takes over an hour to run tests. Someone on team A can and absolutely will mess up what someone on team B is doing. We are now having to spend quarter upon quarter as we define and enforce domain boundaries within the python code base. It is all bolted on checks. Tests are getting worse and people are actively trying to figure out ways around the testing system because it sucks. Compare to my last gig. We had several hundred production repos. Each repo starts from a template with its own build pipeline. All production repos are gated so that any PR must pass tests before it can merge. Any merge has to pass tests before it could be deployed. As the base build processes matured, teams could, at their leisure, pull their services up to the latest and greatest. We even migrated from Jenkins to Buildkite; yeah, it took N pulls into N repos. Not a big deal. Most projects' tests and builds could get code out to production in under 10 minutes, including all those checks. Due to the network boundary, you couldn't accidentally get around someone's abstraction. And if one team blew up their build doing something dumb? No problem, it only affects that one team. The argument is "gah, managing all those services!" Keep data behind APIs. Keep APIs backwards compatible. Keep dependencies acyclic. This is _possible_ with monorepos, but you have to do extra work compared to networked services -- yes, when any particular team/service can deploy in minutes due to low build system complexity you are winning. Can you get that wrong and make strange cyclic dependencies and introduce performance issues due to network hops? Yeah, of course. However, we were processing, literally, 10s of billions of api requests on this system and teams could work untethered from one another. The new gig does eerily similar software, but is several orders of magnitude slower in their ability to process data and their ability to move new features. yes, yes, you could have networked services and a monorepo and you can leverage tooling like Pants to minimize the testing to only account for changed files. It is just fighting what I have found to be a better model. Keep things separate. Keep things fast to change.

alumic 3y ago
While I have yet to work with a large Django monorepo, it's the architecture I'm leaning towards for a product I'm building. Are you guys using packages like nplusone [0] to help with some of this ORM complexity? As I mentioned, I am headed in this direction so I'm curious to hear your insight. I'd also love to pick your brain on any other pitfalls of this approach. [0] https://github.com/jmcarp/nplusone https://github.com/jmcarp/nplusone
lijok 3y ago
You're describing a failure in leadership, not a failure in the monorepo model. If your dev leads are not bringing these issues up to management, or management are not backing them on solving these issues, you're doomed whether you're doing monorepos or not. Monorepos have their drawbacks (and I personally don't like them either), but being unable to trust devs is not one of them. If you can't make monorepos work, what makes you think you can get "microrepos" to work?
aylmao 3y ago
There's three systems an org can (and should) have in place to prevent this: - Dev-time analysis. This is, types, linters and tests that will throw errors and block merges before they happen. - A good code-review environment. Not one that's so tedious that everyone just seeks the stamp to move on, but also no one that's so lax there's no code review at all. One in which code-review is really an opportunity to improve the code before it's merged, people are committing to unblocking others, and everyone assumes their responsibility in having the right people look at changes. - A better team structure. One where ownership of the code is shared, and code quality goals are present. Usually if computer systems, peers and the org at large all agree in creating a good dev environment, dev environments tend to not be bad. In my experience not having a monorepo doesn't fix the issue, it just hides it away. Repos don't have the same code-quality, and overall everyone tries to stay in their little island and not deal with problems and disagreements outside of it. You lose the opportunity to talk about these issues. There isn't that learning and teaching dynamic in the wider org, and the whole company is worse off as a consequence. Plus, of course, there's a lot of extra-overhead in developing with APIs between each component.

alumic 3y ago

While I have yet to work with a large Django monorepo, it's the architecture I'm leaning towards for a product I'm building. Are you guys using packages like nplusone [0] to help with some of this ORM complexity? As I mentioned, I am headed in this direction so I'm curious to hear your insight. I'd also love to pick your brain on any other pitfalls of this approach. [0] https://github.com/jmcarp/nplusone https://github.com/jmcarp/nplusone

lijok 3y ago

You're describing a failure in leadership, not a failure in the monorepo model. If your dev leads are not bringing these issues up to management, or management are not backing them on solving these issues, you're doomed whether you're doing monorepos or not. Monorepos have their drawbacks (and I personally don't like them either), but being unable to trust devs is not one of them. If you can't make monorepos work, what makes you think you can get "microrepos" to work?

aylmao 3y ago

There's three systems an org can (and should) have in place to prevent this: - Dev-time analysis. This is, types, linters and tests that will throw errors and block merges before they happen. - A good code-review environment. Not one that's so tedious that everyone just seeks the stamp to move on, but also no one that's so lax there's no code review at all. One in which code-review is really an opportunity to improve the code before it's merged, people are committing to unblocking others, and everyone assumes their responsibility in having the right people look at changes. - A better team structure. One where ownership of the code is shared, and code quality goals are present. Usually if computer systems, peers and the org at large all agree in creating a good dev environment, dev environments tend to not be bad. In my experience not having a monorepo doesn't fix the issue, it just hides it away. Repos don't have the same code-quality, and overall everyone tries to stay in their little island and not deal with problems and disagreements outside of it. You lose the opportunity to talk about these issues. There isn't that learning and teaching dynamic in the wider org, and the whole company is worse off as a consequence. Plus, of course, there's a lot of extra-overhead in developing with APIs between each component.