9 ms·

Somewhere down the line, developers will start to realize that single applications running in the same server memory and CPU cache will run much faster than Mic

by acd 6y ago

Somewhere down the line, developers will start to realize that single applications running in the same server memory and CPU cache will run much faster than Micro services applications that makes api calls over the network. Depending on your organization scale of course, but for many smaller shops, there is a significant performance overhead of going over the network. Ie a Single binary running Rust/Go/Java/NodeJs vs distributed Microservices applications.

scarmig 6y ago
Within the same data center, a network call doesn't need to have significant performance overhead; it can be on the order of a read from main memory. Still overhead, but for many SLOs it's irrelevant. The inherent cost of microservices is conceptual and organizational complexity, not runtime performance (though you can certainly build slow microservices, you can also build slow monoliths).
- tleb_ 6y ago
  You might get similar throughput but the latency isn't comparable.
vergessenmir 6y ago
I'd like to point out that the article does mention that their system is not single process. They have a database, caching layer and web servers at the very least which could be run on a single box. The number of API calls is a matter of degree when compared with a microservices architecture. A microservices approach does not mean that they should run on different hosts and an early design decision should be that the entire architecture can run on a single host and that the scale out is a feature of the system. It's worrying that this assumption is often made because it does lead to the maintenance and velocity issues that teams encounter when naively adopting microservices.
pbowyer 6y ago
I like RDS because I don't have to think about backups, replication etc but if I colocate the app server (web server, application code) and the database server on the same machine I see 1-2 orders of magnitude performance improvement in database-intensive apps. Replication is the same as usual; have another machine setup to replicate onto. The restriction is you can't fan out hundreds of application servers and half a dozen database servers if you need that - but I don't need to. I haven't done this with clients but I'm doing it with my own (cost-sensitive) side projects and appreciate removing the internal network latency.
- petercooper 6y ago
  I like RDS for the same reason, but I wonder if there's a way you could run Postgres locally, as you suggest, while still using RDS for all the replication/backup side of things? That way the local Postgres management could be pretty minimal and outsource the "annoying bits" to RDS as before. I need to look into the best way to set up something like that.. perhaps pgPool-II.
  - pbowyer 6y ago
    Would this cope with needing to replicate the web server onto another machine as well, for cloud-level reliability? Rather than web + local db <-----> rds you end up with web + local db <-\ |--> rds web + local db <-/ At which point the two masters plus a RDS slave is something I don't know if it's feasible.
- nlittlepoole 6y ago
  You might want to check this out then if you are on AWS https://litestream.io/ https://litestream.io/
anon1253 6y ago
My hope is that the same realization will eventually come to web developers as well. A well written desktop application is much faster, and typically more stable, than a web app that's hosted ... well who can tell where. This isn't to say that web applications don't offer tremendous value, but it is definitely a trade-off worth considering sometimes.
- ivanbakel 6y ago
  >My hope is that the same realization will eventually come to web developers as well. What makes you think that they don't already realise this? What suggests to you that web developers work on webapps for performance reasons, rather than familiarity with the technology or ease of development (or cross-platform support, or lack of user barriers, etc)?
  - FridgeSeal 6y ago
    The insane proliferation of SPA’s, electron apps and the general attitude of “dev time is more important than app performance” of the front-end community suggests performance isn’t a real concern.
- jugg1es 6y ago
  I love building desktop applications but they come with their own set of complications related to installation and DLL-hell.
bottled_poe 6y ago
I don’t think anyone is claiming that microservices are inherently faster. Instead it is a strategy to keep software loosely coupled and enable very large systems to be built by parallel teams while maintaining productivity and quality both technically and organisationally. That said, I’m a strong proponent of monolithic architectures until scale becomes a problem.
- klodolph 6y ago
  It’s also a strategy to isolate failures and to use resources efficiently. I’ve had a very positive experience working on a team that deployed a collection of microservices. That said, it was large scale, and most of the “micro” services were only conceptually micro; they were large in terms of resource usage. We also had excellent tooling.
  - josephg 6y ago
    There are strategies other than microservices to break a problem up into small pieces, manage resources and isolate failures. I've consistently heard good things about Erlang/OTP.
    - klodolph 6y ago
      Not resource management. Resource efficiency. If you have a monolith, you have less control over the service’s shape—the CPU, RAM, IO footprints. If you break it up into smaller microservices, the footprints are smaller and you have more options for how you schedule them. There is a tradeoff here—services that are too small have high overhead. Services that are too large are inefficiently scheduled. Services that are way too large force you to spend money buying the massive machines needed to run them.
      - jandrewrogers 6y ago
        This assertion is making a lot of unwarranted assumptions about the software architecture of a monolith. You can have a monolith and total control over the resources at the scale of an entire machine. This can be extraordinarily resource efficient in a way microservices never are, and is often the default case when designing systems like databases or data infrastructure. A monolith easily achieves 10x throughput of what microservices will do on the same hardware. Which is why people build monoliths, they can be extremely efficient. Microservices are usually about trading resource efficiency for organizational convenience.
kodah 6y ago
Architectures and scalability, although related, are two different conversations. You could build a monolith with an internal messaging queue, this obviously limits scaling possibilities horizontally but may be able to scale vertically. If you externalize the queue and use something like Rabbit, Redis, or NATS then you have more options but you also have more/new problems. Not all things need to scale, and not all things need to scale horizontally. There are ways of shifting out of monoliths too, but this gets trickier with some older, ill-defined codebases in my experience. Microservices can be advantageous but I don't think I've ever cited speed, unless I was talking about deployments and build times (which are definitely part of the conversation since architecture affects the entire SDLC). Usually my strongest points about microservices are about maintainability from the team perspective, reliability (if done right), and earlier parts of the SDLC like testing and build times. The cons are that you are usually having to maintain (and test against) very strong external contracts, inner-service knowledge is strong due to a smaller codebase but often times developers have very little intra-service knowledge (they don't understand the big picture, which is important), and many times codesharing is tricky or just not possible.
- rightbyte 6y ago
  > scaling possibilities horizontally but may be able to scale vertically Let me guess. Vertically, bigger computer, horizontally, more computers? These cloud business speaks is getting to my head.
  - chrisandchris 6y ago
    Yes. But these terms are way older than any cloud-computing and go back many many years.
- threeseed 6y ago
  Scaling vertically is problematic in the cloud where it’s very common to have instances and even entire regions come and go at any time. It’s one of the big reasons why horizontally architected applications have thrived in the cloud era.
jtsiskin 6y ago
That’s true, but also networks can be fast. https://colin-scott.github.io/personal_website/research/interactive_latency.html https://colin-scott.github.io/personal_website/research/inte... It can be faster to read over the network than to read from a hard drive.
- marcosdumay 6y ago
  Well, yes, it can, and people exploit this in large datacenters all the time. That's one of the reasons people get huge, expensive machines full of disks that do nothing else. But none of this applies to your run of the mill REST web service.
- yen223 6y ago
  ... as long as the server isn't itself reading from a hard drive!
Cthulhu_ 6y ago
What do you mean "somewhere down the line"? This is already well-known, but what is also known is that vertical scaling, e.g. how much one machine can do, is limited. When you get to a global, millions-of-requests-per-second system like e.g. Twitter, you've gone well beyond vertical scaling. Know your problem, then pick a solution. I'll admit that the microservices architecture is overused and I'm sure that's what you're aiming at, and I fully agree with that. But I also feel like you cannot understand the scale of true 'web scale' companies and their application. I know I can't.
mcv 6y ago
This was never a surprise to anyone. The goal here isn't speed, but maintainability.
ehutch79 6y ago
Netflix is doing it! So it must be right for my 1 developer internal app with a dozen users.
driverdan 6y ago
General statements like this help no one. The best architecture depends on the application. For example, running a CMS as a monolith makes sense but a video transcoding service doesn't. Good developers choose the architecture that works best for what they're building.

by acd 6y ago

scarmig 6y ago

Within the same data center, a network call doesn't need to have significant performance overhead; it can be on the order of a read from main memory. Still overhead, but for many SLOs it's irrelevant. The inherent cost of microservices is conceptual and organizational complexity, not runtime performance (though you can certainly build slow microservices, you can also build slow monoliths).

tleb_ 6y ago
You might get similar throughput but the latency isn't comparable.

vergessenmir 6y ago

I'd like to point out that the article does mention that their system is not single process. They have a database, caching layer and web servers at the very least which could be run on a single box. The number of API calls is a matter of degree when compared with a microservices architecture. A microservices approach does not mean that they should run on different hosts and an early design decision should be that the entire architecture can run on a single host and that the scale out is a feature of the system. It's worrying that this assumption is often made because it does lead to the maintenance and velocity issues that teams encounter when naively adopting microservices.

pbowyer 6y ago

I like RDS because I don't have to think about backups, replication etc but if I colocate the app server (web server, application code) and the database server on the same machine I see 1-2 orders of magnitude performance improvement in database-intensive apps. Replication is the same as usual; have another machine setup to replicate onto. The restriction is you can't fan out hundreds of application servers and half a dozen database servers if you need that - but I don't need to. I haven't done this with clients but I'm doing it with my own (cost-sensitive) side projects and appreciate removing the internal network latency.

petercooper 6y ago
I like RDS for the same reason, but I wonder if there's a way you could run Postgres locally, as you suggest, while still using RDS for all the replication/backup side of things? That way the local Postgres management could be pretty minimal and outsource the "annoying bits" to RDS as before. I need to look into the best way to set up something like that.. perhaps pgPool-II.
- pbowyer 6y ago
  Would this cope with needing to replicate the web server onto another machine as well, for cloud-level reliability? Rather than web + local db <-----> rds you end up with web + local db <-\ |--> rds web + local db <-/ At which point the two masters plus a RDS slave is something I don't know if it's feasible.
nlittlepoole 6y ago
You might want to check this out then if you are on AWS https://litestream.io/ https://litestream.io/

anon1253 6y ago

My hope is that the same realization will eventually come to web developers as well. A well written desktop application is much faster, and typically more stable, than a web app that's hosted ... well who can tell where. This isn't to say that web applications don't offer tremendous value, but it is definitely a trade-off worth considering sometimes.

ivanbakel 6y ago
>My hope is that the same realization will eventually come to web developers as well. What makes you think that they don't already realise this? What suggests to you that web developers work on webapps for performance reasons, rather than familiarity with the technology or ease of development (or cross-platform support, or lack of user barriers, etc)?
- FridgeSeal 6y ago
  The insane proliferation of SPA’s, electron apps and the general attitude of “dev time is more important than app performance” of the front-end community suggests performance isn’t a real concern.
jugg1es 6y ago
I love building desktop applications but they come with their own set of complications related to installation and DLL-hell.

bottled_poe 6y ago

I don’t think anyone is claiming that microservices are inherently faster. Instead it is a strategy to keep software loosely coupled and enable very large systems to be built by parallel teams while maintaining productivity and quality both technically and organisationally. That said, I’m a strong proponent of monolithic architectures until scale becomes a problem.

klodolph 6y ago
It’s also a strategy to isolate failures and to use resources efficiently. I’ve had a very positive experience working on a team that deployed a collection of microservices. That said, it was large scale, and most of the “micro” services were only conceptually micro; they were large in terms of resource usage. We also had excellent tooling.
- josephg 6y ago
  There are strategies other than microservices to break a problem up into small pieces, manage resources and isolate failures. I've consistently heard good things about Erlang/OTP.
  - klodolph 6y ago
    Not resource management. Resource efficiency. If you have a monolith, you have less control over the service’s shape—the CPU, RAM, IO footprints. If you break it up into smaller microservices, the footprints are smaller and you have more options for how you schedule them. There is a tradeoff here—services that are too small have high overhead. Services that are too large are inefficiently scheduled. Services that are way too large force you to spend money buying the massive machines needed to run them.
    - jandrewrogers 6y ago
      This assertion is making a lot of unwarranted assumptions about the software architecture of a monolith. You can have a monolith and total control over the resources at the scale of an entire machine. This can be extraordinarily resource efficient in a way microservices never are, and is often the default case when designing systems like databases or data infrastructure. A monolith easily achieves 10x throughput of what microservices will do on the same hardware. Which is why people build monoliths, they can be extremely efficient. Microservices are usually about trading resource efficiency for organizational convenience.

kodah 6y ago

Architectures and scalability, although related, are two different conversations. You could build a monolith with an internal messaging queue, this obviously limits scaling possibilities horizontally but may be able to scale vertically. If you externalize the queue and use something like Rabbit, Redis, or NATS then you have more options but you also have more/new problems. Not all things need to scale, and not all things need to scale horizontally. There are ways of shifting out of monoliths too, but this gets trickier with some older, ill-defined codebases in my experience. Microservices can be advantageous but I don't think I've ever cited speed, unless I was talking about deployments and build times (which are definitely part of the conversation since architecture affects the entire SDLC). Usually my strongest points about microservices are about maintainability from the team perspective, reliability (if done right), and earlier parts of the SDLC like testing and build times. The cons are that you are usually having to maintain (and test against) very strong external contracts, inner-service knowledge is strong due to a smaller codebase but often times developers have very little intra-service knowledge (they don't understand the big picture, which is important), and many times codesharing is tricky or just not possible.

rightbyte 6y ago
> scaling possibilities horizontally but may be able to scale vertically Let me guess. Vertically, bigger computer, horizontally, more computers? These cloud business speaks is getting to my head.
- chrisandchris 6y ago
  Yes. But these terms are way older than any cloud-computing and go back many many years.
threeseed 6y ago
Scaling vertically is problematic in the cloud where it’s very common to have instances and even entire regions come and go at any time. It’s one of the big reasons why horizontally architected applications have thrived in the cloud era.

jtsiskin 6y ago

That’s true, but also networks can be fast. https://colin-scott.github.io/personal_website/research/interactive_latency.html https://colin-scott.github.io/personal_website/research/inte... It can be faster to read over the network than to read from a hard drive.

marcosdumay 6y ago
Well, yes, it can, and people exploit this in large datacenters all the time. That's one of the reasons people get huge, expensive machines full of disks that do nothing else. But none of this applies to your run of the mill REST web service.
yen223 6y ago
... as long as the server isn't itself reading from a hard drive!

Cthulhu_ 6y ago

What do you mean "somewhere down the line"? This is already well-known, but what is also known is that vertical scaling, e.g. how much one machine can do, is limited. When you get to a global, millions-of-requests-per-second system like e.g. Twitter, you've gone well beyond vertical scaling. Know your problem, then pick a solution. I'll admit that the microservices architecture is overused and I'm sure that's what you're aiming at, and I fully agree with that. But I also feel like you cannot understand the scale of true 'web scale' companies and their application. I know I can't.

mcv 6y ago

This was never a surprise to anyone. The goal here isn't speed, but maintainability.

ehutch79 6y ago

Netflix is doing it! So it must be right for my 1 developer internal app with a dozen users.

driverdan 6y ago

General statements like this help no one. The best architecture depends on the application. For example, running a CMS as a monolith makes sense but a video transcoding service doesn't. Good developers choose the architecture that works best for what they're building.