5 ms·

I feel like there are competent, competing visions talking past each other on the subject. There's kind of a spectrum: 1. Everything is a monolith. Frontend, b

I feel like there are competent, competing visions talking past each other on the subject. There's kind of a spectrum: 1. Everything is a monolith. Frontend, backend, dataplane, processing, whatever: it's all one giant, tightly coupled vertically-scaled ball of mud. (This is insane.) 2. Everything is a monolith, but parts are horizontally scaled. Imagine a big Flask app where there are M frontend servers, and N backend async task queue processors, all running the same codebase but with different configurations for each kind of deployment. (This is perfectly reasonable.) 3. There are a small number of separate services. That frontend Flask server talks to a Go or Rust or Node or whatever backend, each appropriate to the task at hand. (This is perfectly reasonable.) 4. Everything is a separate service. There are N engineers and N+50% servers written in N languages, and a web page load hits 8 different internal servers that do 12 different things. The site currently handles 23 requests per day, but it's meant to vertically scale to Google size once it becomes popular. Also, everything is behind a single load balancer, but the principle engineer (who interned at Netflix) handwaves it away a "basically infinitely scalable". (This is insane.) These conversations seem to devolve into fans of 1 and 4 arguing that the other is wrong. People in 2 and 3 make eye contact with each other, shrug, and get back to making money.

383toast 1y ago
you can get pretty far with (2) and (3), haven't really understood the need for (4) unless you're FAANG
- yearolinuxdsktp 1y ago
  Facts! #1 is not insane as long as you keep your internals modular (all-in-one deployment doesn't mean ball of mud... you can avoid putting service calls into your domain objects or your data plane code). And you can go from #1 to #2 once you see the need and slice the services out that need it (such as decoupling async batch processing into a 2nd service that shares the domain and the data plane code and does not include the front-end).
  - kstrauser 1y ago
    In fairness, it being a tightly coupled ball of mud is part of my definition of #1 here. What you describe is basically #2 waiting to happen, just that no one's needed to do it yet.
    - yearolinuxdsktp 1y ago
      Makes sense, I agree then that #1 as you exactly define it is insane.
- amarant 1y ago
  4 doesn't really work for serious big load applications either: nobody will be able to understand the codebase. You want 3. During my days at mojang we did some variation of 3. We had ~250k requests/second, and handled it just fine (we had 4 nines availability forever chasing the fifth, and sub 20ms response times) I think even among those who do see big loads, few see as much malicious traffic as we did. This was one of the arguments for a micro service architecture. If a DDOS took down our login service, already logged in players would be unaffected (until their tokens expired anyway) Well, that was a long winded way to say, 3 is about as micro as you want to go. I've only seen 4 done once, and that site actually went under whenever they had more than 30 request per minute. (Admittedly they had made a bunch of other really bad decisions not covered in the above description, but having ~30 services on a team of 12, in order to handle a handful of requests per hour was certainly their biggest mistake.)
notfed 1y ago
Why is #1 insane (no horizontal scaling) if you only have 23 requests per day?
- Spivak 1y ago
  I feel like people forget just how fast a single machine can be. If your database is SQLite the app will be able to burn down requests faster than you ever thought possible. You can handle much more than 23 req/day.
  - kstrauser 1y ago
    In the not-too-distant past I was handling many thousands of DB-backed requests per hour in a Python app running plain PostgreSQL. You can get really, really far with a decent machine if you mind the bottlenecks. Getting into swap? Add RAM. Blocked by IO? Throw in some more NVMe. Any reasonable CPU can process a lot more data than it's popular to think.
    - bluGill 1y ago
      Anytime someone talks about scale I remember just how much data the low mhz CPUs used to process. Sure the modern stuff has nicer UIs, but the UIs of the past where not bad, and we a lot of data was processed. Almost nobody has more data than what the busiest 200mhz CPU on 1999 used to handle alone, so if you can't do it that isn't a scaling problem it is a people problem. (don't get me wrong, this might be a good trade off to make - but don't say you couldn't do it on a single computer)
- bkanuka 1y ago
  Don't know if this is sarcasm or not. If you have 23 req/day, then there's no tech problem to solve. Whatever you have is good enough, and increasing traffic will come from solving problems outside tech (marketing, etc)
- kstrauser 1y ago
  It's not. It's kind of bonkers to pursue that when you have a lot of traffic, but it's a perfectly sane starting point until you know where the pain points are. In general, the vast number of small shops chugging away with a tractably sized monolith aren't really participating in the conversation, just idly wondering what approach they'd take if they suddenly needed to scale up.
Spivak 1y ago
Once you open the door to 3 it's a lot harder to stop the slide into 4. Devs love greenfield codebases and once that's an acceptable way to solve a problem they'll reach for it more and more. Especially if your setup has low friction to "just" spinning up a new service.
- kstrauser 1y ago
  True, and some org-level discipline is important here. I'm all for polyglot architectures, but only for reasonably small values of poly. Like if you wrote the frontend in Python, but really some performance critical Rust, fine. Or maybe you merged with another shop that already had a bunch of Go, far out. Resist the urge to roll out that Prolog-driven inference service that your VP Eng vibe coded after reading an article about cool and strange programming languages.
  - bluGill 1y ago
    I generally think the right rule of thumbs is about 10 languages - but that includes the build system, CI system... Not everyone needs to know all 10, but you need a good set of options for that that need it. Most people should be using the same 2 or 3 of that 10, but a few teams need to do something weird and so need the other options that are overall rarely used.
    - kstrauser 1y ago
      That's very scale-dependent. I can imagine why a FAANG might need 10 languages, between Python APIs and Java services and Rust and Go things and a Node system and a smattering of Perl and some ETL in Scala and BI in C#, etc. A startup with less than 50 employees almost certainly does not. Maybe I'd compromise on no more than, say, 2-3 language per department. If you're small enough that "engineering" is 1 department, then there you go: choose wisely. If you have a whole department for frontend, and one for backend, and one for ops, and one for analytics, etc., then you can somewhat treat those as encapsulation boundaries and have more flexibility.
      - 1y ago
mytailorisrich 1y ago
"Tightly coupled" is the sticking point. Tight coupling is bad architecture whether you have a monolith or microservices, which is the general point of the article.
nine_k 1y ago
Logical separation, the modules, is what allows to preserve developer sanity. Physical separation, the (micro-)services is what allows you to ship things flexibly. Somewhere on the distant high end, microservices also play a role in enabling scalability to colossal scales, only needed by relatively few very huge companies. The key problem of developing a large system is allowing many people to work on it, without producing a gridlock. A monolith, by its nature, produces a gridlock easily once a sufficient number of people need to work on it in parallel. Hence modules, "narrow waist" interfaces, etc. But the key thing is that "you ship your org chart" [1]. Modules allow different teams to work in parallel and independently, as long as the interfaces are clearly defined. Modules deployed separately, aka services, allow different teams to ship their results independently, as long as they remain compatible with the rest of the system. Solving these organizational problems is much more important for any large company than overcoming any technical hurdles. [1]: https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
- bluGill 1y ago
  > Physical separation, the (micro-)services is what allows you to ship things flexibly. If you are willing to pay a price. Once you allow things to ship separately are are locked into the API Hyrum's law mistakes of the past until you can be 100% that all uses of the old thing are gone in the real world. This is often hard. By shipping things together you can verify they all work together before you ship, and more importantly when you realize something was a mistake there is a set time where you can say "that is no longer allowed" (this is a political problem - sometimes they will tell you no we won't change, but at least you have options). Everything else is spot on, but I am feeling enough pain from mistakes made 15 years ago (that looked reasonable at the time!) such that I feel the need to point it out.
- xorcist 1y ago
  Scale? At what "scale" runs Linux, perhaps the most well known monolith software ever? (And also the subject of perhaps the most well studied flamew^Wdiscussion about mono- versus microservice architecture.) It only runs most of all servers and most of all the mobile terminals in the world. Where is that distant higher end, where microservices unlock colossal scale, exactly? Architecture matters. Just not always the way you think. But it serves as catnip for everyone who loves a good debate. Anyone who gets tired of writing code can always make a good living writing about software architecture. And good for them. There's a certain artistry to it, and they have diagrams and everything.
ivanjermakov 1y ago
I also laugh at 30+ microservice projects where almost every service connects to a single oracle database with no sharding/partitioning. Inb4 aws dynamodb outage.
- connicpu 1y ago
  The number of services at my job that are just grpc wrappers for a database with endpoints that are only accessed by services that have their own connections open to the same database has been driving me insane.
- 4ndrewl 1y ago
  Isn't that a microservices 101? No shared databases?
  - kstrauser 1y ago
    An important and common exception would be for performance critical reasons, as mentioned elsewhere. Say most of your code's in Python, but you rewrote parts in a faster language. There you're not necessaryily separating them for isolation or encapsulation purposes. It'd be as much of a mistake to force them to hit separate DBs.
  - xorcist 1y ago
    Wasn't it two pizza teams and using the right tools for the right job?
solumos 1y ago
1 Could maybe make sense for a proof of concept, but realize that you're probably throwing it away (sooner better than later). Start with 2, and think about the separation of deployable targets and shared libraries/plugins. You can eventually carve out separate infra for deployable targets when resource contention becomes a problem (e.g. DB load is affecting unrelated services) 3 is rarely the right first step for a small team 4 is never the right first step for a small team
abalone 1y ago
Or there's 3.5: separate services where it makes sense, but not necessarily small in number. "Makes sense" would entail things like, does it have distinct resource utilization or scaling characteristics, or do you want to enable your service to more gracefully degrade if that module becomes unavailable. (This is basically the definition of 3 without the implication that it will be rare.) As opposed to 4 which is about proactively breaking everything down into the smallest possible units on the expectation that the added complexity is always worth it.
Sammi 1y ago
Lots and lots and lots of people make no. 1 work. The issue isn't whether it is a monolith or not. The issue is whether it has good or poor internal architecture. No 1 can scale to 1000 requests per second. One machine, one db, one application. It is totally doable because so many people do it. It's just not sexy and doesn't pad you resume. It's boring stuff that just works.
- foobarian 1y ago
  I wish our app only got 1krps. Maybe these conversations should be bucketed by the traffic volume people are handling - it really leads to completely different designs.

383toast 1y ago

you can get pretty far with (2) and (3), haven't really understood the need for (4) unless you're FAANG

yearolinuxdsktp 1y ago
Facts! #1 is not insane as long as you keep your internals modular (all-in-one deployment doesn't mean ball of mud... you can avoid putting service calls into your domain objects or your data plane code). And you can go from #1 to #2 once you see the need and slice the services out that need it (such as decoupling async batch processing into a 2nd service that shares the domain and the data plane code and does not include the front-end).
- kstrauser 1y ago
  In fairness, it being a tightly coupled ball of mud is part of my definition of #1 here. What you describe is basically #2 waiting to happen, just that no one's needed to do it yet.
  - yearolinuxdsktp 1y ago
    Makes sense, I agree then that #1 as you exactly define it is insane.
amarant 1y ago
4 doesn't really work for serious big load applications either: nobody will be able to understand the codebase. You want 3. During my days at mojang we did some variation of 3. We had ~250k requests/second, and handled it just fine (we had 4 nines availability forever chasing the fifth, and sub 20ms response times) I think even among those who do see big loads, few see as much malicious traffic as we did. This was one of the arguments for a micro service architecture. If a DDOS took down our login service, already logged in players would be unaffected (until their tokens expired anyway) Well, that was a long winded way to say, 3 is about as micro as you want to go. I've only seen 4 done once, and that site actually went under whenever they had more than 30 request per minute. (Admittedly they had made a bunch of other really bad decisions not covered in the above description, but having ~30 services on a team of 12, in order to handle a handful of requests per hour was certainly their biggest mistake.)

notfed 1y ago

Why is #1 insane (no horizontal scaling) if you only have 23 requests per day?

Spivak 1y ago
I feel like people forget just how fast a single machine can be. If your database is SQLite the app will be able to burn down requests faster than you ever thought possible. You can handle much more than 23 req/day.
- kstrauser 1y ago
  In the not-too-distant past I was handling many thousands of DB-backed requests per hour in a Python app running plain PostgreSQL. You can get really, really far with a decent machine if you mind the bottlenecks. Getting into swap? Add RAM. Blocked by IO? Throw in some more NVMe. Any reasonable CPU can process a lot more data than it's popular to think.
  - bluGill 1y ago
    Anytime someone talks about scale I remember just how much data the low mhz CPUs used to process. Sure the modern stuff has nicer UIs, but the UIs of the past where not bad, and we a lot of data was processed. Almost nobody has more data than what the busiest 200mhz CPU on 1999 used to handle alone, so if you can't do it that isn't a scaling problem it is a people problem. (don't get me wrong, this might be a good trade off to make - but don't say you couldn't do it on a single computer)
bkanuka 1y ago
Don't know if this is sarcasm or not. If you have 23 req/day, then there's no tech problem to solve. Whatever you have is good enough, and increasing traffic will come from solving problems outside tech (marketing, etc)
kstrauser 1y ago
It's not. It's kind of bonkers to pursue that when you have a lot of traffic, but it's a perfectly sane starting point until you know where the pain points are. In general, the vast number of small shops chugging away with a tractably sized monolith aren't really participating in the conversation, just idly wondering what approach they'd take if they suddenly needed to scale up.

Spivak 1y ago

Once you open the door to 3 it's a lot harder to stop the slide into 4. Devs love greenfield codebases and once that's an acceptable way to solve a problem they'll reach for it more and more. Especially if your setup has low friction to "just" spinning up a new service.

kstrauser 1y ago
True, and some org-level discipline is important here. I'm all for polyglot architectures, but only for reasonably small values of poly. Like if you wrote the frontend in Python, but really some performance critical Rust, fine. Or maybe you merged with another shop that already had a bunch of Go, far out. Resist the urge to roll out that Prolog-driven inference service that your VP Eng vibe coded after reading an article about cool and strange programming languages.
- bluGill 1y ago
  I generally think the right rule of thumbs is about 10 languages - but that includes the build system, CI system... Not everyone needs to know all 10, but you need a good set of options for that that need it. Most people should be using the same 2 or 3 of that 10, but a few teams need to do something weird and so need the other options that are overall rarely used.
  - kstrauser 1y ago
    That's very scale-dependent. I can imagine why a FAANG might need 10 languages, between Python APIs and Java services and Rust and Go things and a Node system and a smattering of Perl and some ETL in Scala and BI in C#, etc. A startup with less than 50 employees almost certainly does not. Maybe I'd compromise on no more than, say, 2-3 language per department. If you're small enough that "engineering" is 1 department, then there you go: choose wisely. If you have a whole department for frontend, and one for backend, and one for ops, and one for analytics, etc., then you can somewhat treat those as encapsulation boundaries and have more flexibility.
    - 1y ago

mytailorisrich 1y ago

"Tightly coupled" is the sticking point. Tight coupling is bad architecture whether you have a monolith or microservices, which is the general point of the article.

nine_k 1y ago

Logical separation, the modules, is what allows to preserve developer sanity. Physical separation, the (micro-)services is what allows you to ship things flexibly. Somewhere on the distant high end, microservices also play a role in enabling scalability to colossal scales, only needed by relatively few very huge companies. The key problem of developing a large system is allowing many people to work on it, without producing a gridlock. A monolith, by its nature, produces a gridlock easily once a sufficient number of people need to work on it in parallel. Hence modules, "narrow waist" interfaces, etc. But the key thing is that "you ship your org chart" [1]. Modules allow different teams to work in parallel and independently, as long as the interfaces are clearly defined. Modules deployed separately, aka services, allow different teams to ship their results independently, as long as they remain compatible with the rest of the system. Solving these organizational problems is much more important for any large company than overcoming any technical hurdles. [1]: https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law

bluGill 1y ago
> Physical separation, the (micro-)services is what allows you to ship things flexibly. If you are willing to pay a price. Once you allow things to ship separately are are locked into the API Hyrum's law mistakes of the past until you can be 100% that all uses of the old thing are gone in the real world. This is often hard. By shipping things together you can verify they all work together before you ship, and more importantly when you realize something was a mistake there is a set time where you can say "that is no longer allowed" (this is a political problem - sometimes they will tell you no we won't change, but at least you have options). Everything else is spot on, but I am feeling enough pain from mistakes made 15 years ago (that looked reasonable at the time!) such that I feel the need to point it out.
xorcist 1y ago
Scale? At what "scale" runs Linux, perhaps the most well known monolith software ever? (And also the subject of perhaps the most well studied flamew^Wdiscussion about mono- versus microservice architecture.) It only runs most of all servers and most of all the mobile terminals in the world. Where is that distant higher end, where microservices unlock colossal scale, exactly? Architecture matters. Just not always the way you think. But it serves as catnip for everyone who loves a good debate. Anyone who gets tired of writing code can always make a good living writing about software architecture. And good for them. There's a certain artistry to it, and they have diagrams and everything.

ivanjermakov 1y ago

I also laugh at 30+ microservice projects where almost every service connects to a single oracle database with no sharding/partitioning. Inb4 aws dynamodb outage.

connicpu 1y ago
The number of services at my job that are just grpc wrappers for a database with endpoints that are only accessed by services that have their own connections open to the same database has been driving me insane.
4ndrewl 1y ago
Isn't that a microservices 101? No shared databases?
- kstrauser 1y ago
  An important and common exception would be for performance critical reasons, as mentioned elsewhere. Say most of your code's in Python, but you rewrote parts in a faster language. There you're not necessaryily separating them for isolation or encapsulation purposes. It'd be as much of a mistake to force them to hit separate DBs.
- xorcist 1y ago
  Wasn't it two pizza teams and using the right tools for the right job?

solumos 1y ago

1 Could maybe make sense for a proof of concept, but realize that you're probably throwing it away (sooner better than later). Start with 2, and think about the separation of deployable targets and shared libraries/plugins. You can eventually carve out separate infra for deployable targets when resource contention becomes a problem (e.g. DB load is affecting unrelated services) 3 is rarely the right first step for a small team 4 is never the right first step for a small team

abalone 1y ago

Or there's 3.5: separate services where it makes sense, but not necessarily small in number. "Makes sense" would entail things like, does it have distinct resource utilization or scaling characteristics, or do you want to enable your service to more gracefully degrade if that module becomes unavailable. (This is basically the definition of 3 without the implication that it will be rare.) As opposed to 4 which is about proactively breaking everything down into the smallest possible units on the expectation that the added complexity is always worth it.

Sammi 1y ago

Lots and lots and lots of people make no. 1 work. The issue isn't whether it is a monolith or not. The issue is whether it has good or poor internal architecture. No 1 can scale to 1000 requests per second. One machine, one db, one application. It is totally doable because so many people do it. It's just not sexy and doesn't pad you resume. It's boring stuff that just works.

foobarian 1y ago
I wish our app only got 1krps. Maybe these conversations should be bucketed by the traffic volume people are handling - it really leads to completely different designs.