5 ms·

Note that your example seems to only matter if I use differently spec'd instances for different services. If they all have the same ratio of e.g. memory to cpu

by sbov 10y ago

Note that your example seems to only matter if I use differently spec'd instances for different services. If they all have the same ratio of e.g. memory to cpu to disk, I'm not sure what slicing my instances across services really gets me. If you slice it across services, you add a few instances, and your system chugs along. If you don't, you still add a few instances, and it still chugs along. I could maybe see if you have some services that take huge amounts of memory and others that take huge amounts of CPU. If you have a standard "monolithic app instance", and had to scale up your CPU by 10x but memory only by 1x because of your pdf renderer, you will likely be wasting large amounts of memory. But unless you have huge disparities in memory vs cpu use (and services don't vary in use together) I don't really know what sort of cost savings you can get here - wouldn't this be the actual value add of being able to independently scale - less cost because you can more accurately hit your resource (cpu vs memory vs disk and so on) requirements for your project? In contrast, from my experience, separating your web/app/database/cache layer from eachother tends to be extremely beneficial for independent scaling because they almost always vary widely in how they consume which resources (memory vs cpu vs disk and so on). They also tend to be written with this in mind and so it is basically free to do so. An aside, but many of your downsides apply not to just services, but also having to scale an application horizontally. If the name of the game is scaling, then many of these you will pay for regardless of the question of monolith vs. services. All that above aside, I definitely feel the other benefits. But I really don't get it - every article I read about services seems to mention independent scaling, when it seems like a fairly suspect benefit. Maybe I just haven't worked on the correct project.

dlubarov 10y ago
Coming from Square (which is mostly SOA, but with an old monolithic service), we had quite a few services which really needed to be separated for performance reasons: - One needed a large in-process cache in order to deliver good performance; it would have consumed to much memory on each instance of the monolith. - Some services used large ML models, which also would have consumed too much of memory on monolith instances. - A lot of our payment-related services had hourly or daily batch jobs. Anything with big resource spikes probably shouldn't share a machine with latency-sensitive code (like online payment processing or just web handlers). - Related to the above, some jobs had to be done by a master instance. If the monolith did them, they would have disproportionately affected a single instance of the monolith.
greenhatman 10y ago
I'm new to SOA apps, but I have the exact same questions. Even failure isolation seems less useful that it's touted to be.
jfoutz 10y ago
Each thing takes some resources. Application performance tends to get worse over time, usually because the focus is on adding features, rather than improving efficiency (sweeping generalization). So eventually you'll find yourself in a spot where your monolithic app doesn't have much headroom on existing machines. You switch to a new auth system that's 20% more expensive, and the whole system is running out of memory. This is a contrived example. I mostly agree with you. I think you do hit a point at the application level where the big monolithic app is a little bit to big, and that's a tough spot to be in. If you can keep things small, light and efficient mono will work forever. But always keep in mind that it can outgrow your instance sizes. So prepare to either move up to the next size instance (tough if you're running your own hardware), or start thinking about splitting out parts of the app.
- closeparen 10y ago
  Resource utilization is generally a function of traffic. Why is adding more machines to a specific service's load balancer pool better than adding more machines to the monolith's load balancer pool? You're saving maybe a little bit of memory by not loading unnecessary code, but that seems inconsequential given OS level caching. One possible argument is that you can reserve capacity by service, so a computationally expensive but unimportant endpoint doesn't accidentally gobble up the resources you added to alleviate the starvation that was slowing down a different endpoint. But you could also do that with smarter load balancing - deploy a monolith to all hosts, but partition traffic between separate pools depending on endpoint at the load balancer level. I don't think performance isolation is a good argument for microservices. I don't think failure isolation is either - the interactions are likely to create more, trickier failures than the isolation will prevent. The real argument is about scaling the organization. Much easier to work on and frequently deploy small codebases with small numbers of commits and commiters per day, communicating across team boundaries via Thrift IDL files, than to have thousands of engineers on one codebase and thousands of potentially breaking changes introduced between every deploy.
  - phamilton 10y ago
    Scaling your app isn't just about turning knobs. It's about making design tradeoffs based on your usage pattern. Most apps are very framework dependent and mixing and matching frameworks within a monolith is a nightmare. Having more flexibility around platform allows you to scale independently.
phamilton 10y ago
CPU vs memory is one area, but also being blocked on IO and network saturation. An endpoint that is essentially a glorified proxy is going to scale differently than one that does real CPU work. Doing all of that in a synchronous platform us going to require massive memory for the threaded IO, or it's going to be really expensive on an event loop. So build two separate systems, put them on specialized hardware (make sure you have 10G network on the proxy).

by sbov 10y ago

dlubarov 10y ago

Coming from Square (which is mostly SOA, but with an old monolithic service), we had quite a few services which really needed to be separated for performance reasons: - One needed a large in-process cache in order to deliver good performance; it would have consumed to much memory on each instance of the monolith. - Some services used large ML models, which also would have consumed too much of memory on monolith instances. - A lot of our payment-related services had hourly or daily batch jobs. Anything with big resource spikes probably shouldn't share a machine with latency-sensitive code (like online payment processing or just web handlers). - Related to the above, some jobs had to be done by a master instance. If the monolith did them, they would have disproportionately affected a single instance of the monolith.

greenhatman 10y ago

I'm new to SOA apps, but I have the exact same questions. Even failure isolation seems less useful that it's touted to be.

jfoutz 10y ago

Each thing takes some resources. Application performance tends to get worse over time, usually because the focus is on adding features, rather than improving efficiency (sweeping generalization). So eventually you'll find yourself in a spot where your monolithic app doesn't have much headroom on existing machines. You switch to a new auth system that's 20% more expensive, and the whole system is running out of memory. This is a contrived example. I mostly agree with you. I think you do hit a point at the application level where the big monolithic app is a little bit to big, and that's a tough spot to be in. If you can keep things small, light and efficient mono will work forever. But always keep in mind that it can outgrow your instance sizes. So prepare to either move up to the next size instance (tough if you're running your own hardware), or start thinking about splitting out parts of the app.

closeparen 10y ago
Resource utilization is generally a function of traffic. Why is adding more machines to a specific service's load balancer pool better than adding more machines to the monolith's load balancer pool? You're saving maybe a little bit of memory by not loading unnecessary code, but that seems inconsequential given OS level caching. One possible argument is that you can reserve capacity by service, so a computationally expensive but unimportant endpoint doesn't accidentally gobble up the resources you added to alleviate the starvation that was slowing down a different endpoint. But you could also do that with smarter load balancing - deploy a monolith to all hosts, but partition traffic between separate pools depending on endpoint at the load balancer level. I don't think performance isolation is a good argument for microservices. I don't think failure isolation is either - the interactions are likely to create more, trickier failures than the isolation will prevent. The real argument is about scaling the organization. Much easier to work on and frequently deploy small codebases with small numbers of commits and commiters per day, communicating across team boundaries via Thrift IDL files, than to have thousands of engineers on one codebase and thousands of potentially breaking changes introduced between every deploy.
- phamilton 10y ago
  Scaling your app isn't just about turning knobs. It's about making design tradeoffs based on your usage pattern. Most apps are very framework dependent and mixing and matching frameworks within a monolith is a nightmare. Having more flexibility around platform allows you to scale independently.

phamilton 10y ago

CPU vs memory is one area, but also being blocked on IO and network saturation. An endpoint that is essentially a glorified proxy is going to scale differently than one that does real CPU work. Doing all of that in a synchronous platform us going to require massive memory for the threaded IO, or it's going to be really expensive on an event loop. So build two separate systems, put them on specialized hardware (make sure you have 10G network on the proxy).