20 ms·

Microservices are less efficient, but are still more scalable. Servers can only get so big. If your monolith needs more resources than a single server can prov

Microservices are less efficient, but are still more scalable. Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.

KronisLV 4y ago
> Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself). Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity. Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices. Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong. However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere): SINGLETONS - a monolithic application that can ever only have a single instance running, for example, when business processes are stored in memory for a bit, or have user sessions or something like that stored locally as well; these will ONLY ever scale VERTICALLY, unless you re-architect them MONOLITHS - applications that contain all of your project's logic in a single codebase, although multiple instances can be launched in parallel, depending on your needs; can be scaled BOTH VERTICALLY and HORIZONTALLY; they have more overhead though and observability can be a bit challenging MICROSERVICES - applications that contain a part of the total project's logic, typically across multiple separate codebases, possibly with shared library code, pieces of your project can be scaled separately, BOTH VERTICALLY and HORIZONTALLY; they are operationally more complex, can involve more network chatter and while you can observe how services perform, now you need to deal with distributed tracing Of course, there can be more nuance to it, like modular monoliths, that still have one codebase, but can have certain groups of functionality enabled or disabled. I actually wrote about that approach a while back, calling them "moduliths": https://blog.kronis.dev/articles/modulith-because-we-need-to-scale-but-we-also-cannot-afford-micro-services https://blog.kronis.dev/articles/modulith-because-we-need-to... I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.
rlt 4y ago
You can horizontally scale a monolith without going full microservices.
spion 4y ago
You can run multiple instances of most stateless monoliths
- bcrosby95 4y ago
  It's also way easier to design for. The web is an embarrassingly parallel problem once you remove the state. That's a big reason why you offload state to databases - they've done the hardest bits for you.
catlifeonmars 4y ago
Running a single server often can’t meet availability expectations that users have. This is orthogonal to scalability. You almost always need multiple copies.
- ElectricalUnion 4y ago
  You can run most applications on a single server - IBM S/390 big iron has very high reliability, redundancy and I/O bandwidth.
  - catlifeonmars 4y ago
    Yes but it’s literally a single point of failure. You probably want at least two servers in separate physical locations. Also how do you do deployments without interruption of service on a single server?
    - SgtBastard 4y ago
      Blue/green deployments and request draining like with any other deployment topology.
    - ElectricalUnion 4y ago
      > it’s literally a single point of failure. Those machines are made very redundant. All components in a Parallel Sysplex configuration can operate in redundant mode. Redundant power. Redundant central processor complex. Redundant multichip module (what would be a chipset on microcomputer platforms). Redundant RAM Memory. Redundant Storage. Redundant internal interconnects. Redundant network adapters. That's why corporations that require high availability to not get sued for millions/billions use them.
      - catlifeonmars 4y ago
        Redundant physical location. That means different power dependencies and different cities.
cogman10 4y ago
You know, I don't really think microservices are fundamentally more scalable. Rather, they expose scaling issues more readily. When you have a giant monolith with the "load the world" endpoint, it can be tricky to pinpoint the the "load the world" endpoint (or, as is often the case, endpoint*s*) is what's causing issues. Instead, everyone just tends to think of it as "the x app having problems." When you bust the monolith into the x/y/z, and x and z got the "load the world" endpoints, that starts the fires of "x is constantly killing things and it's only doing this one thing. How do we do that better?" That allows you to better prioritize fixing those scaling problems.
- zbobet2012 4y ago
  Microservices enable independent scale _domains_. This means they can be more scaleable and dense, but it's not necessarily true.
- deleted 4y ago
  [deleted]
- jacobsimon 4y ago
  That’s a really interesting point - something that could probably be addressed by module-level logging and metrics. That said, even as a pro-monolith advocate, I can see why it’s preferable to not allow any one module/service to consume all the resources for your service in the first place. The service boundaries in microservice architectures can help enforce resource limits that otherwise go unchecked in a larger application.
  - cogman10 4y ago
    It's one I've ran into a few times in my company (which has a large number of these types of endpoints). The silly thing is that something like the JVM (which we use) really wants to host monoliths. It's really the most efficient way to use resources if you can swing it. The problem is when you give the JVM 512GB of ram, it hides the fact that you have a module needlessly loading up 64gb of ram for a few seconds. Something that only comes to a head when that module is ran concurrently.
goodpoint 4y ago
> Microservices are less efficient, but are still more scalable. Not at all. You can run your monolith on multiple nodes just as you can do with microservices. If anything, it scales much better as you reduce network interactions.
locutous 4y ago
> Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices 200 threads, 12TB of RAM, with a pipe upwards of 200GB/s. This isn't even as big as you can go, this is a reasonable off the shelf item. If your service doesn't need more than this, maybe don't break it up. :) I believe that this level of service can no longer accurately be described as "micro".
- Xeoncross 4y ago
  Helps to have a language that natively uses more CPU cores and/or training for the devs. Ruby, Python, PHP and Node.js startups have to figure out how to use the cores while C++, Rust, Erlang and Go have no issues running a single process that maxes out all 64 CPU cores.
  - grogenaut 4y ago
    we have all sorts of problems maxing out all the cores on a c6.24xl if youre memory or network conatrained. Even if CPU constrained it can be hard.
  - NorwegianDude 4y ago
    Doesn't really matter that much if a language is using all cores from one process or if there is multiple processes.
  - FpUser 4y ago
    This is exactly what I do. When it comes to your regular backend business server I write stateful multithreaded monolith in C++ sitting on the same computer as the database hosted on some multicore server with gobbles of RAM (those are cheap now). Performance is insane and is enough to serve any normal business for years or decades to come. So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;)
    - noobermin 4y ago
      >So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;) As long as hype chasers in middle management don't get in the way after convincing themselves they too must be like FAANG with a few orders of magnitude less of a consumer base.
throwaway894345 4y ago
Yeah, it's bin packing, not straight efficiency. Also, people seem to exaggerate latency for RPC calls. I often get the feeling that people who make these latency criticisms have been burned by some nominal "microservices" architecture in which an API call is made in a hot loop or something. Network latency is real, and some systems really do run on a tight latency budget, but most sane architectures will just do a couple of calls to a database (same performance as monolith) and maybe a transitive service call or something.
- funnymony 4y ago
  Couple of calls is normal. But if you make everything a microservice. And there are dependencies between them, then by design its destined that some hotter loop eventually will contain rpc.
Sparkyte 4y ago
It is about striking a balance. Little big concept or mono-micro people like the call it is where it is at. Spending too much time making a complex environment benefits noone. Spending too much time making a complex application benefits noone. Breaking down monoliths into purposed tasks and then creating pod groups small containers is where it is at. Managing a fleet on kubernetes is easier than managing one in some configuration management stack. Be it puppet or salt. You have too many dependencies. Only your core infrastructure kubernetes should be a product of the configuration management.
greenhatman 4y ago
You can have more than one server per monolith. I don't think you actually understand what microservices are. You don't put a load balancer to load balance between different services. A load balancer balances trafic between servers of the same service or monolith. Microservices mean the servers of different services run different code. A load balancer only works together with servers running the same code.
- SgtBastard 4y ago
  >A load balancer only works together with servers running the same code. Uh - what? >A load balancer balances traffic between servers Correct. > of the same service or mononlith Incorrect. Load balancers used to work solely at Layer 4, in which case you’d be correct that any 80/443 traffic would be farmed across servers that would necessarily need to run the same code base. But modern load balancers (NGINX et al) especially in a service mesh / Kubernetes context, balance load at endpoint level. A high traffic endpoint might have dozens of pods running on dozens of K8s hosts having traffic routes to them by their ingress controller. A low traffic endpoint’s pod might only be running on few or one (potentially different) hosts. The load balancer internally makes these decisions. I hope this clears up your misunderstanding.
  - closeparen 4y ago
    Each set of upstream hosts in nginx is a single instance of load balancing. You aren't load balancing across services, you're splitting traffic by service and then load balancing across instances of that service. The split is inessential. You can just as easily have homogeneous backends & one big load balancing pool. Instances within that pool can even have affinity for or ownership of particular records! The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices.
    - SgtBastard 4y ago
      > The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices. Are you replying to the right comment? I made no such claim.

KronisLV 4y ago

> Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself). Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity. Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices. Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong. However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere): SINGLETONS - a monolithic application that can ever only have a single instance running, for example, when business processes are stored in memory for a bit, or have user sessions or something like that stored locally as well; these will ONLY ever scale VERTICALLY, unless you re-architect them MONOLITHS - applications that contain all of your project's logic in a single codebase, although multiple instances can be launched in parallel, depending on your needs; can be scaled BOTH VERTICALLY and HORIZONTALLY; they have more overhead though and observability can be a bit challenging MICROSERVICES - applications that contain a part of the total project's logic, typically across multiple separate codebases, possibly with shared library code, pieces of your project can be scaled separately, BOTH VERTICALLY and HORIZONTALLY; they are operationally more complex, can involve more network chatter and while you can observe how services perform, now you need to deal with distributed tracing Of course, there can be more nuance to it, like modular monoliths, that still have one codebase, but can have certain groups of functionality enabled or disabled. I actually wrote about that approach a while back, calling them "moduliths": https://blog.kronis.dev/articles/modulith-because-we-need-to-scale-but-we-also-cannot-afford-micro-services https://blog.kronis.dev/articles/modulith-because-we-need-to... I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.

rlt 4y ago

You can horizontally scale a monolith without going full microservices.

spion 4y ago

You can run multiple instances of most stateless monoliths

bcrosby95 4y ago
It's also way easier to design for. The web is an embarrassingly parallel problem once you remove the state. That's a big reason why you offload state to databases - they've done the hardest bits for you.

catlifeonmars 4y ago

Running a single server often can’t meet availability expectations that users have. This is orthogonal to scalability. You almost always need multiple copies.

ElectricalUnion 4y ago
You can run most applications on a single server - IBM S/390 big iron has very high reliability, redundancy and I/O bandwidth.
- catlifeonmars 4y ago
  Yes but it’s literally a single point of failure. You probably want at least two servers in separate physical locations. Also how do you do deployments without interruption of service on a single server?
  - SgtBastard 4y ago
    Blue/green deployments and request draining like with any other deployment topology.
  - ElectricalUnion 4y ago
    > it’s literally a single point of failure. Those machines are made very redundant. All components in a Parallel Sysplex configuration can operate in redundant mode. Redundant power. Redundant central processor complex. Redundant multichip module (what would be a chipset on microcomputer platforms). Redundant RAM Memory. Redundant Storage. Redundant internal interconnects. Redundant network adapters. That's why corporations that require high availability to not get sued for millions/billions use them.
    - catlifeonmars 4y ago
      Redundant physical location. That means different power dependencies and different cities.

cogman10 4y ago

You know, I don't really think microservices are fundamentally more scalable. Rather, they expose scaling issues more readily. When you have a giant monolith with the "load the world" endpoint, it can be tricky to pinpoint the the "load the world" endpoint (or, as is often the case, endpoint*s*) is what's causing issues. Instead, everyone just tends to think of it as "the x app having problems." When you bust the monolith into the x/y/z, and x and z got the "load the world" endpoints, that starts the fires of "x is constantly killing things and it's only doing this one thing. How do we do that better?" That allows you to better prioritize fixing those scaling problems.

zbobet2012 4y ago
Microservices enable independent scale _domains_. This means they can be more scaleable and dense, but it's not necessarily true.
deleted 4y ago
[deleted]
jacobsimon 4y ago
That’s a really interesting point - something that could probably be addressed by module-level logging and metrics. That said, even as a pro-monolith advocate, I can see why it’s preferable to not allow any one module/service to consume all the resources for your service in the first place. The service boundaries in microservice architectures can help enforce resource limits that otherwise go unchecked in a larger application.
- cogman10 4y ago
  It's one I've ran into a few times in my company (which has a large number of these types of endpoints). The silly thing is that something like the JVM (which we use) really wants to host monoliths. It's really the most efficient way to use resources if you can swing it. The problem is when you give the JVM 512GB of ram, it hides the fact that you have a module needlessly loading up 64gb of ram for a few seconds. Something that only comes to a head when that module is ran concurrently.

goodpoint 4y ago

> Microservices are less efficient, but are still more scalable. Not at all. You can run your monolith on multiple nodes just as you can do with microservices. If anything, it scales much better as you reduce network interactions.

locutous 4y ago

> Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices 200 threads, 12TB of RAM, with a pipe upwards of 200GB/s. This isn't even as big as you can go, this is a reasonable off the shelf item. If your service doesn't need more than this, maybe don't break it up. :) I believe that this level of service can no longer accurately be described as "micro".

Xeoncross 4y ago
Helps to have a language that natively uses more CPU cores and/or training for the devs. Ruby, Python, PHP and Node.js startups have to figure out how to use the cores while C++, Rust, Erlang and Go have no issues running a single process that maxes out all 64 CPU cores.
- grogenaut 4y ago
  we have all sorts of problems maxing out all the cores on a c6.24xl if youre memory or network conatrained. Even if CPU constrained it can be hard.
- NorwegianDude 4y ago
  Doesn't really matter that much if a language is using all cores from one process or if there is multiple processes.
- FpUser 4y ago
  This is exactly what I do. When it comes to your regular backend business server I write stateful multithreaded monolith in C++ sitting on the same computer as the database hosted on some multicore server with gobbles of RAM (those are cheap now). Performance is insane and is enough to serve any normal business for years or decades to come. So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;)
  - noobermin 4y ago
    >So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;) As long as hype chasers in middle management don't get in the way after convincing themselves they too must be like FAANG with a few orders of magnitude less of a consumer base.

throwaway894345 4y ago

Yeah, it's bin packing, not straight efficiency. Also, people seem to exaggerate latency for RPC calls. I often get the feeling that people who make these latency criticisms have been burned by some nominal "microservices" architecture in which an API call is made in a hot loop or something. Network latency is real, and some systems really do run on a tight latency budget, but most sane architectures will just do a couple of calls to a database (same performance as monolith) and maybe a transitive service call or something.

funnymony 4y ago
Couple of calls is normal. But if you make everything a microservice. And there are dependencies between them, then by design its destined that some hotter loop eventually will contain rpc.

Sparkyte 4y ago

It is about striking a balance. Little big concept or mono-micro people like the call it is where it is at. Spending too much time making a complex environment benefits noone. Spending too much time making a complex application benefits noone. Breaking down monoliths into purposed tasks and then creating pod groups small containers is where it is at. Managing a fleet on kubernetes is easier than managing one in some configuration management stack. Be it puppet or salt. You have too many dependencies. Only your core infrastructure kubernetes should be a product of the configuration management.

greenhatman 4y ago

You can have more than one server per monolith. I don't think you actually understand what microservices are. You don't put a load balancer to load balance between different services. A load balancer balances trafic between servers of the same service or monolith. Microservices mean the servers of different services run different code. A load balancer only works together with servers running the same code.

SgtBastard 4y ago
>A load balancer only works together with servers running the same code. Uh - what? >A load balancer balances traffic between servers Correct. > of the same service or mononlith Incorrect. Load balancers used to work solely at Layer 4, in which case you’d be correct that any 80/443 traffic would be farmed across servers that would necessarily need to run the same code base. But modern load balancers (NGINX et al) especially in a service mesh / Kubernetes context, balance load at endpoint level. A high traffic endpoint might have dozens of pods running on dozens of K8s hosts having traffic routes to them by their ingress controller. A low traffic endpoint’s pod might only be running on few or one (potentially different) hosts. The load balancer internally makes these decisions. I hope this clears up your misunderstanding.
- closeparen 4y ago
  Each set of upstream hosts in nginx is a single instance of load balancing. You aren't load balancing across services, you're splitting traffic by service and then load balancing across instances of that service. The split is inessential. You can just as easily have homogeneous backends & one big load balancing pool. Instances within that pool can even have affinity for or ownership of particular records! The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices.
  - SgtBastard 4y ago
    > The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices. Are you replying to the right comment? I made no such claim.