5 ms·

> Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team? > More directly, what'

by hmeh 3y ago

> Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team? > More directly, what's wrong with a monolith at the dozen person team size? Great question. It's honestly a hard question to answer because the real answer requires tacit knowledge, but I'll try anyway because you asked. I'll link a couple articles of longer form writing that may help to answer as well. The benefit ultimately is productivity. That productivity comes from autonomy, which comes from partitioning. Each component can be worked, tested, and deployed on its own. We never need to spin up more than one at a time, which means any time we are working on our code, we are typically working on 1/400th of it, and we know we are because we have 400 separate GitHub repositories. We can't accidentally get peanut butter in our mayonaise. Each part is independently built and tested. When it's time to upgrade Rails, we can upgrade one application. This is 1/20th of our Rails code. That's a smaller batch. We know from basic flow principles that (see: Principles of Product Development Flow, Lean, etc.) that small batches are better for productivity. We rarely have merge conflicts. We don't do pull requests. We branch when we need to. We are often working on a brand new, never been deployed project. We don't have our tests running in CI. Our tests run in less than a second in most projects on our laptops. With those that have UI, our tests typically run in less than 30 seconds which include thorough Capybara (UI Interaction) testing. When we are working in any application, we only need to consider its direct efferents when making a change to any interfaces it exposes. We can trace those changes through easily when we need to make them. Because we practice software design, we don't often need to make large sweeping changes, but when we do, we can do them methodically without disruption. The "worst" part is that some things are tedious. But we can automate tedious things. We can invent around tedious things. We used to have to deploy 15 web applications any time we had a style change. Now we deploy one application that supplies the CSS for all our applications via SSI (server-side-include). If we need to deploy 70 back end components, we can script that (with manual verification at each step). We can even spread that out across a couple team members and do it as an ensemble. It takes about an hour to deploy everything. We maybe do this once a month or so (when something highly afferent changes -- and guess what, we structure our design so that afferent things change as infrequently as possible). Most things we work on we work until they are "complete". That is, they rarely change after that. We have components that have been in production for 3 years and haven't been substantially touched aside from Ruby upgrades and the like. They are still perfectly in control. The list goes on and on. Here are a few articles, feel free to poke around the others and ask any follow up questions. Partitions and Compositions: https://github.com/aaronjensen/software-development/blob/master/partitions-and-compositions.md https://github.com/aaronjensen/software-development/blob/mas... The Mythical Monolith: https://github.com/aaronjensen/software-development/blob/master/monoliths.md https://github.com/aaronjensen/software-development/blob/mas...

the_gipsy 3y ago
> we have 400 separate GitHub repositories Do you think this sounds good? > We can't accidentally get peanut butter in our mayonaise Hint: you have 400 different jars of peanut butter with mayonnaise.
- hmeh 3y ago
  How do you know that? Hint: you don’t. You’re not the first to tell us what we do is impossible, and you won’t be the last. By the way, I know how it sounds. And I know why a person would fear it. And, I know why they’d be incorrect in their assessment, because, unlike them, I have significant material experience in both camps.
  - outsomnia 3y ago
    This whole business has had few conceptual ideas that stand apart from fashion and stood the tests of time. A lot of people look down on C now for various reasons, but it's still extremely useful and going strong 50+ years later. Even though I'm older than C itself in the last year or two using it intensively, I found new insights in how to arrange things and what to avoid to make things more robust and maintainable. It's not likely that hundreds of "microrepos" on github and even rails are The Last Word that people will eventually converge on and be using into the 2100s. It sounds like each repo is its own standalone thing with its own deps. There are advantages and disadvantages to maintaining this kind of thing, some of the disadvantages will only become clear as the deps evolve and mutate. Although you may feel you are in a local optimum with this architecture in 2023, ahead of the game, energized by your belief in it, you probably are not finished evolving your understanding even on the scale of one or five years. One way or another almost everything in this business is ephemeral, disposable and will be disposed of in favour of something else.
    - hmeh 3y ago
      Do you know what does stand the test of time? Design principles that are based on the immutable laws of physics. There's a reason that I can quote Plato to support the notion of partitioning. There's a reason that the basic principles are used in the design of every single significant thing in the world. There's a reason Linux has as many modules as it has (for them, a single monorepo works, but multiple works too -- see any library or application that isn't in the primary monorepo but is distributed via a package manager). Also, no one said "microrepos". Don't put the current fad's fixation on what I am describing. I'm simply describing structural design. Does thing X and thing Y need common behavior? Is it actually generalized behavior? That's what we have called a "library" since we've been building non-trivial applications out of software. Are there two disparate business processes that don't need to share states, and you know they don't need to share states because you've done the design work? Those are two separate components. Pub/Sub exists for collaboration between those components as necessary. I'm not trying to say that our current technique for all things is the end-all-be-all. That would deny the continuous improvement mindset (i.e., the relentless pursuit of perfection) that got us where we are. But, there are things that won't change. Smaller batches will always be (generally) easier to manage than larger. Taking something that has high afferent coupling and adding specializations to it will always be a mistake (See Fat Model, Skinny Controller). The reality is that there are software design mistakes that are knowably mistakes. They've been known for some time as well and can be measured. Those are the things that we avoid and those are the principles that have guided where we are. I know this sounds highfalutin, but there is someone on our team that has been doing this for over 30 years (by "this", I mean continuous improvement/seeking perfection) and has studied under many of the greats in our industry. We aren't just making this stuff up.

by hmeh 3y ago

the_gipsy 3y ago

> we have 400 separate GitHub repositories Do you think this sounds good? > We can't accidentally get peanut butter in our mayonaise Hint: you have 400 different jars of peanut butter with mayonnaise.

hmeh 3y ago
How do you know that? Hint: you don’t. You’re not the first to tell us what we do is impossible, and you won’t be the last. By the way, I know how it sounds. And I know why a person would fear it. And, I know why they’d be incorrect in their assessment, because, unlike them, I have significant material experience in both camps.
- outsomnia 3y ago
  This whole business has had few conceptual ideas that stand apart from fashion and stood the tests of time. A lot of people look down on C now for various reasons, but it's still extremely useful and going strong 50+ years later. Even though I'm older than C itself in the last year or two using it intensively, I found new insights in how to arrange things and what to avoid to make things more robust and maintainable. It's not likely that hundreds of "microrepos" on github and even rails are The Last Word that people will eventually converge on and be using into the 2100s. It sounds like each repo is its own standalone thing with its own deps. There are advantages and disadvantages to maintaining this kind of thing, some of the disadvantages will only become clear as the deps evolve and mutate. Although you may feel you are in a local optimum with this architecture in 2023, ahead of the game, energized by your belief in it, you probably are not finished evolving your understanding even on the scale of one or five years. One way or another almost everything in this business is ephemeral, disposable and will be disposed of in favour of something else.
  - hmeh 3y ago
    Do you know what does stand the test of time? Design principles that are based on the immutable laws of physics. There's a reason that I can quote Plato to support the notion of partitioning. There's a reason that the basic principles are used in the design of every single significant thing in the world. There's a reason Linux has as many modules as it has (for them, a single monorepo works, but multiple works too -- see any library or application that isn't in the primary monorepo but is distributed via a package manager). Also, no one said "microrepos". Don't put the current fad's fixation on what I am describing. I'm simply describing structural design. Does thing X and thing Y need common behavior? Is it actually generalized behavior? That's what we have called a "library" since we've been building non-trivial applications out of software. Are there two disparate business processes that don't need to share states, and you know they don't need to share states because you've done the design work? Those are two separate components. Pub/Sub exists for collaboration between those components as necessary. I'm not trying to say that our current technique for all things is the end-all-be-all. That would deny the continuous improvement mindset (i.e., the relentless pursuit of perfection) that got us where we are. But, there are things that won't change. Smaller batches will always be (generally) easier to manage than larger. Taking something that has high afferent coupling and adding specializations to it will always be a mistake (See Fat Model, Skinny Controller). The reality is that there are software design mistakes that are knowably mistakes. They've been known for some time as well and can be measured. Those are the things that we avoid and those are the principles that have guided where we are. I know this sounds highfalutin, but there is someone on our team that has been doing this for over 30 years (by "this", I mean continuous improvement/seeking perfection) and has studied under many of the greats in our industry. We aren't just making this stuff up.