11 ms·

I just want to point out that for the second problem (scalability of CPU/memory/io), microservices almost always make things worse. Making an RPC necessarily im

I just want to point out that for the second problem (scalability of CPU/memory/io), microservices almost always make things worse. Making an RPC necessarily implies serialization and deserialization of data, and nearly always also means sending data over a socket. Plus the fact that most services have some constant overhead of the footprint to run the RPC code and other things (healthchecking, stats collection, etc.) that is typically bundled into each service. And even if the extra CPU/memory isn't a big deal for you, doing RPCs is going to add latency, and if you get too many microservices the latency numbers can start really adding up and be very difficult to fix later. Running code in a single process is MUCH lower overhead because you don't need to transit a network layer and you're generally just passing pointers to data around rather than serializing/deserializing it. There are definitely some cases where using microservices does make things more CPU/memory efficient, but it's much rarer than people think. An example where you'd actually get efficiency would be something like a geofence service (imagine Uber, Doordash, etc.) where the geofence definitions are probably large and have to be stored in memory. Depending on how often geofence queries happen, it might be more efficient to have a small number of geofence service instances with the geofence definitions loaded in memory rather than having this logic as a module that many workers need to have loaded. But again, cases like this are much less common than the cases where services just lead to massive bloat. I was working at Uber when they started transitioning from monolith to microservices, and pretty much universally splitting logic into microservices required provisioning many more servers and were disastrous for end-to-end latency times.

jupp0r 4y ago
I think all your points are correct but none of them address scalability. What do you do if your one efficient machine is too small for your problem? To be fair, not a lot of problems reach that point in practice, definitely not every problem a microservice is written for. It might still be completely viable to rewrite something so that it's only 10% as efficient but you can scale it horizontally easily.
spullara 4y ago
I do think using something like Flatbuffers or CapnProto where the SerDe is removed (aside from the travel over the network) could be a huge win. Another thing I have always wanted to try is to develop a system using microservices with a RPC layer that uses immutable objects so you can later combine the services and convert RPCs to function calls.
winternett 4y ago
I really miss the old days where when an app was down it was obvious... Now apps just go unresponsive because of connectivity issues between services... We used to have frameworks that could communicate errors in one log, it was a beautiful thing. As cloud infrastructure became more powerful to better host so called "monolithic" systems, it was all scrapped to move to microservices, which were made for less powerful infrastructure. Everything was refactored to the point where it's expensive to go back... Pretty crazy how much money the industry wastes on refactoring... On top of that, the tendency to get complacent with unstructured data in a lot of systems is really creating a very complicated lock-in when systems are developed for unique services on each cloud provider... Bowls of spaghetti. Hire Solutions Architects for dev projects, make your apps future proof... I warn you. Too much microservice customization leads to vendor lock in and expensive operational costs... This is why a lot of apps get sunsetted early.
dahfizz 4y ago
Microservices are less efficient, but are still more scalable. Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.
- KronisLV 4y ago
  > Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself). Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity. Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices. Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong. However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere): SINGLETONS - a monolithic application that can ever only have a single instance running, for example, when business processes are stored in memory for a bit, or have user sessions or something like that stored locally as well; these will ONLY ever scale VERTICALLY, unless you re-architect them MONOLITHS - applications that contain all of your project's logic in a single codebase, although multiple instances can be launched in parallel, depending on your needs; can be scaled BOTH VERTICALLY and HORIZONTALLY; they have more overhead though and observability can be a bit challenging MICROSERVICES - applications that contain a part of the total project's logic, typically across multiple separate codebases, possibly with shared library code, pieces of your project can be scaled separately, BOTH VERTICALLY and HORIZONTALLY; they are operationally more complex, can involve more network chatter and while you can observe how services perform, now you need to deal with distributed tracing Of course, there can be more nuance to it, like modular monoliths, that still have one codebase, but can have certain groups of functionality enabled or disabled. I actually wrote about that approach a while back, calling them "moduliths": https://blog.kronis.dev/articles/modulith-because-we-need-to-scale-but-we-also-cannot-afford-micro-services https://blog.kronis.dev/articles/modulith-because-we-need-to... I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.
lamontcg 4y ago
I was working at Amazon when they started transitioning from monolith to microservices, and the big win there was locality of data and caching. All of the catalog data was moved to a service which only served catalog data so its cache was optimized for catalog data and the load balancers in front of it could optimize across that cache with consistent hashing. This was different from the front end web tier which used consistent hashing to pin customers to individual webservers. For stuff like order history or customer data, those services sat in front of their respective databases and provided consistent hashing along with availability (providing a consistent write-through cache in front of the SQL databases used at the time). I wouldn't call those areas where it makes things more efficient rare, but actually common, and it comes from letting your data dictate your microservices rather than letting your organiation dictate your microservices (although if teams own data, then they should line up).
- jbergens 4y ago
  I think Amazon (and Google and other FAANG entities) work at another scale than 99.9% of the rest of the world. The problems they face are different, including the impact of data locality. I've seen many systems where there are a few hundred to a few thousand users, not hundred of millions or billions of users. There can also be teams of 5-10 developers who manages +20 microservices. I still don't think those projects have the same needs and could have done something else.
  - lamontcg 4y ago
    I started at Amazon in 2001 when we had around 400 servers running the website, it was large, but it was around the scale that most Fortune 500 companies operate at. I'll agree that a team of developers with 20 microservices sounds obviously wrong (which is really my point that the site should be divided around data and the orgs should probably reflect the needs of the data--not the other way around). And Amazon back then didn't have that problem.
- jayde2767 4y ago
  Thank you for pointing out caching. I was going to reply, at some point, that architecture in a well-designed distributed system considers both locality of data and the ability to cache data as a means of improving both latency and throughput and often does appreciably. But I don't have to now.
jayd16 4y ago
So tell me, who is making microservices for calls that could be done wholly in memory? How are you handling rollbacks and such for this in memory logic? Aren't you still writing out to external data stores? If that's the case then its really not a comparison of in process to a RPC, it just two RPC hops now, no?
- dagss 4y ago
  When you move to microservices -- or rather, when you split your data across several DBs -- you sometimes end up basically reimplementing DB synchronization in business logic layers that could (sometimes, and depending on scalability constraints) be solved by using DB transactions. So instead of a single DB transaction, often with a single SQL roundtrip, microservices can sometimes force through coordination of separate transactions to several DBs.

jupp0r 4y ago

I think all your points are correct but none of them address scalability. What do you do if your one efficient machine is too small for your problem? To be fair, not a lot of problems reach that point in practice, definitely not every problem a microservice is written for. It might still be completely viable to rewrite something so that it's only 10% as efficient but you can scale it horizontally easily.

spullara 4y ago

I do think using something like Flatbuffers or CapnProto where the SerDe is removed (aside from the travel over the network) could be a huge win. Another thing I have always wanted to try is to develop a system using microservices with a RPC layer that uses immutable objects so you can later combine the services and convert RPCs to function calls.

winternett 4y ago

I really miss the old days where when an app was down it was obvious... Now apps just go unresponsive because of connectivity issues between services... We used to have frameworks that could communicate errors in one log, it was a beautiful thing. As cloud infrastructure became more powerful to better host so called "monolithic" systems, it was all scrapped to move to microservices, which were made for less powerful infrastructure. Everything was refactored to the point where it's expensive to go back... Pretty crazy how much money the industry wastes on refactoring... On top of that, the tendency to get complacent with unstructured data in a lot of systems is really creating a very complicated lock-in when systems are developed for unique services on each cloud provider... Bowls of spaghetti. Hire Solutions Architects for dev projects, make your apps future proof... I warn you. Too much microservice customization leads to vendor lock in and expensive operational costs... This is why a lot of apps get sunsetted early.

dahfizz 4y ago

Microservices are less efficient, but are still more scalable. Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.

KronisLV 4y ago
> Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers. I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself). Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity. Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices. Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong. However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere): SINGLETONS - a monolithic application that can ever only have a single instance running, for example, when business processes are stored in memory for a bit, or have user sessions or something like that stored locally as well; these will ONLY ever scale VERTICALLY, unless you re-architect them MONOLITHS - applications that contain all of your project's logic in a single codebase, although multiple instances can be launched in parallel, depending on your needs; can be scaled BOTH VERTICALLY and HORIZONTALLY; they have more overhead though and observability can be a bit challenging MICROSERVICES - applications that contain a part of the total project's logic, typically across multiple separate codebases, possibly with shared library code, pieces of your project can be scaled separately, BOTH VERTICALLY and HORIZONTALLY; they are operationally more complex, can involve more network chatter and while you can observe how services perform, now you need to deal with distributed tracing Of course, there can be more nuance to it, like modular monoliths, that still have one codebase, but can have certain groups of functionality enabled or disabled. I actually wrote about that approach a while back, calling them "moduliths": https://blog.kronis.dev/articles/modulith-because-we-need-to-scale-but-we-also-cannot-afford-micro-services https://blog.kronis.dev/articles/modulith-because-we-need-to... I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.

lamontcg 4y ago

I was working at Amazon when they started transitioning from monolith to microservices, and the big win there was locality of data and caching. All of the catalog data was moved to a service which only served catalog data so its cache was optimized for catalog data and the load balancers in front of it could optimize across that cache with consistent hashing. This was different from the front end web tier which used consistent hashing to pin customers to individual webservers. For stuff like order history or customer data, those services sat in front of their respective databases and provided consistent hashing along with availability (providing a consistent write-through cache in front of the SQL databases used at the time). I wouldn't call those areas where it makes things more efficient rare, but actually common, and it comes from letting your data dictate your microservices rather than letting your organiation dictate your microservices (although if teams own data, then they should line up).

jbergens 4y ago
I think Amazon (and Google and other FAANG entities) work at another scale than 99.9% of the rest of the world. The problems they face are different, including the impact of data locality. I've seen many systems where there are a few hundred to a few thousand users, not hundred of millions or billions of users. There can also be teams of 5-10 developers who manages +20 microservices. I still don't think those projects have the same needs and could have done something else.
- lamontcg 4y ago
  I started at Amazon in 2001 when we had around 400 servers running the website, it was large, but it was around the scale that most Fortune 500 companies operate at. I'll agree that a team of developers with 20 microservices sounds obviously wrong (which is really my point that the site should be divided around data and the orgs should probably reflect the needs of the data--not the other way around). And Amazon back then didn't have that problem.
jayde2767 4y ago
Thank you for pointing out caching. I was going to reply, at some point, that architecture in a well-designed distributed system considers both locality of data and the ability to cache data as a means of improving both latency and throughput and often does appreciably. But I don't have to now.

jayd16 4y ago

So tell me, who is making microservices for calls that could be done wholly in memory? How are you handling rollbacks and such for this in memory logic? Aren't you still writing out to external data stores? If that's the case then its really not a comparison of in process to a RPC, it just two RPC hops now, no?

dagss 4y ago
When you move to microservices -- or rather, when you split your data across several DBs -- you sometimes end up basically reimplementing DB synchronization in business logic layers that could (sometimes, and depending on scalability constraints) be solved by using DB transactions. So instead of a single DB transaction, often with a single SQL roundtrip, microservices can sometimes force through coordination of separate transactions to several DBs.