8 ms·

If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be r

by hmeh 3y ago

If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. The only reason we have Docker on our machine is to test the images we deploy (just to make sure they build and run). You have experienced what happens when web developers cosplay as software architects. That's when you get "microservices". It's a travesty, and it has ruined an entire generation of software developers who now believe that partitioning and autonomy are just myths. So, as a blunt statement - your experience is typical. That is, it's average. That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the bell curve doing what it always does.

kamikaz1k 3y ago
Just to clarify, what are components referring to? Are they independent web services? And what is the total size of the developers you worked with? Are the 12 just your team or the entire engineering org? Edit: found from another comment > We don’t call them microservices, because we aren’t web developers pretending to do architecture and recreating the mistakes of DCOM, CORBA, and Web Services. Most of them are autonomous event sourced components. The others are stand alone web applications that are stitched together with Nginx routing and SSI.
- hmeh 3y ago
  Components are either autonomous event sourced back end components (using Eventide) or independent web applications that are combined with Nginx routes and SSI. The UI is (mostly) server-rendered Rails. The users have no idea they are hitting 20 or so different web applications, but the developers sure feel the productivity boost of every application being a small application that is (relatively) independent from one another. A dozen or so is the entire software development organization. Edit: Indeed! If you have any other questions, ask away. Most people think what we do isn't actually possible, but it is.
irjustin 3y ago
> So, as a blunt statement - your experience is typical. That is, it's average. Definitely a fair judgement and would agree that this constitutes poor architecting on many many companies' parts including ones that I've been a part of. > Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team? More directly, what's wrong with a monolith at the dozen person team size?
- hmeh 3y ago
  > Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team? > More directly, what's wrong with a monolith at the dozen person team size? Great question. It's honestly a hard question to answer because the real answer requires tacit knowledge, but I'll try anyway because you asked. I'll link a couple articles of longer form writing that may help to answer as well. The benefit ultimately is productivity. That productivity comes from autonomy, which comes from partitioning. Each component can be worked, tested, and deployed on its own. We never need to spin up more than one at a time, which means any time we are working on our code, we are typically working on 1/400th of it, and we know we are because we have 400 separate GitHub repositories. We can't accidentally get peanut butter in our mayonaise. Each part is independently built and tested. When it's time to upgrade Rails, we can upgrade one application. This is 1/20th of our Rails code. That's a smaller batch. We know from basic flow principles that (see: Principles of Product Development Flow, Lean, etc.) that small batches are better for productivity. We rarely have merge conflicts. We don't do pull requests. We branch when we need to. We are often working on a brand new, never been deployed project. We don't have our tests running in CI. Our tests run in less than a second in most projects on our laptops. With those that have UI, our tests typically run in less than 30 seconds which include thorough Capybara (UI Interaction) testing. When we are working in any application, we only need to consider its direct efferents when making a change to any interfaces it exposes. We can trace those changes through easily when we need to make them. Because we practice software design, we don't often need to make large sweeping changes, but when we do, we can do them methodically without disruption. The "worst" part is that some things are tedious. But we can automate tedious things. We can invent around tedious things. We used to have to deploy 15 web applications any time we had a style change. Now we deploy one application that supplies the CSS for all our applications via SSI (server-side-include). If we need to deploy 70 back end components, we can script that (with manual verification at each step). We can even spread that out across a couple team members and do it as an ensemble. It takes about an hour to deploy everything. We maybe do this once a month or so (when something highly afferent changes -- and guess what, we structure our design so that afferent things change as infrequently as possible). Most things we work on we work until they are "complete". That is, they rarely change after that. We have components that have been in production for 3 years and haven't been substantially touched aside from Ruby upgrades and the like. They are still perfectly in control. The list goes on and on. Here are a few articles, feel free to poke around the others and ask any follow up questions. Partitions and Compositions: https://github.com/aaronjensen/software-development/blob/master/partitions-and-compositions.md https://github.com/aaronjensen/software-development/blob/mas... The Mythical Monolith: https://github.com/aaronjensen/software-development/blob/master/monoliths.md https://github.com/aaronjensen/software-development/blob/mas...

by hmeh 3y ago

kamikaz1k 3y ago

Just to clarify, what are components referring to? Are they independent web services? And what is the total size of the developers you worked with? Are the 12 just your team or the entire engineering org? Edit: found from another comment > We don’t call them microservices, because we aren’t web developers pretending to do architecture and recreating the mistakes of DCOM, CORBA, and Web Services. Most of them are autonomous event sourced components. The others are stand alone web applications that are stitched together with Nginx routing and SSI.

hmeh 3y ago
Components are either autonomous event sourced back end components (using Eventide) or independent web applications that are combined with Nginx routes and SSI. The UI is (mostly) server-rendered Rails. The users have no idea they are hitting 20 or so different web applications, but the developers sure feel the productivity boost of every application being a small application that is (relatively) independent from one another. A dozen or so is the entire software development organization. Edit: Indeed! If you have any other questions, ask away. Most people think what we do isn't actually possible, but it is.

irjustin 3y ago

> So, as a blunt statement - your experience is typical. That is, it's average. Definitely a fair judgement and would agree that this constitutes poor architecting on many many companies' parts including ones that I've been a part of. > Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team? More directly, what's wrong with a monolith at the dozen person team size?

hmeh 3y ago
> Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team? > More directly, what's wrong with a monolith at the dozen person team size? Great question. It's honestly a hard question to answer because the real answer requires tacit knowledge, but I'll try anyway because you asked. I'll link a couple articles of longer form writing that may help to answer as well. The benefit ultimately is productivity. That productivity comes from autonomy, which comes from partitioning. Each component can be worked, tested, and deployed on its own. We never need to spin up more than one at a time, which means any time we are working on our code, we are typically working on 1/400th of it, and we know we are because we have 400 separate GitHub repositories. We can't accidentally get peanut butter in our mayonaise. Each part is independently built and tested. When it's time to upgrade Rails, we can upgrade one application. This is 1/20th of our Rails code. That's a smaller batch. We know from basic flow principles that (see: Principles of Product Development Flow, Lean, etc.) that small batches are better for productivity. We rarely have merge conflicts. We don't do pull requests. We branch when we need to. We are often working on a brand new, never been deployed project. We don't have our tests running in CI. Our tests run in less than a second in most projects on our laptops. With those that have UI, our tests typically run in less than 30 seconds which include thorough Capybara (UI Interaction) testing. When we are working in any application, we only need to consider its direct efferents when making a change to any interfaces it exposes. We can trace those changes through easily when we need to make them. Because we practice software design, we don't often need to make large sweeping changes, but when we do, we can do them methodically without disruption. The "worst" part is that some things are tedious. But we can automate tedious things. We can invent around tedious things. We used to have to deploy 15 web applications any time we had a style change. Now we deploy one application that supplies the CSS for all our applications via SSI (server-side-include). If we need to deploy 70 back end components, we can script that (with manual verification at each step). We can even spread that out across a couple team members and do it as an ensemble. It takes about an hour to deploy everything. We maybe do this once a month or so (when something highly afferent changes -- and guess what, we structure our design so that afferent things change as infrequently as possible). Most things we work on we work until they are "complete". That is, they rarely change after that. We have components that have been in production for 3 years and haven't been substantially touched aside from Ruby upgrades and the like. They are still perfectly in control. The list goes on and on. Here are a few articles, feel free to poke around the others and ask any follow up questions. Partitions and Compositions: https://github.com/aaronjensen/software-development/blob/master/partitions-and-compositions.md https://github.com/aaronjensen/software-development/blob/mas... The Mythical Monolith: https://github.com/aaronjensen/software-development/blob/master/monoliths.md https://github.com/aaronjensen/software-development/blob/mas...