4 ms·

> scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it). I gave a talk [1] about scalability of Java s

> scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it). I gave a talk [1] about scalability of Java systems on Kubernetes, and one of my recommendations is that Java-based systems - or any system on a runtime similar to the JVM, like CLR or even Go - should be scaled diagonally. Efficiency is the word. While horizontal scaling can easily address most performance issues and load demand, in most cases it is the least efficient way for Java (and again, I risk saying .NET and Go), as these systems struggle with CPU throttling and garbage collectors. In short, and exemplifying: one container with 2 CPUs and 2GB of RAM will allow the JVM to perform better, in general, than 2 containers with 1 CPU/1GB RAM each. That said, customers shouldn't be scaling horizontally to any amount more than what is adequately reasonable for resiliency, or unless the bottleneck is somewhere like disk access. For performance on the other hand, customers should be scaling vertically. And Kubernetes VPA is already available. People just need to use it properly and smartly. If a Kubernetes admin believes a particular pod should double in number of replicas, the admin should consider: "would this microservice benefit even more from 1.5x more CPU/Memory than 2x more replicas?" and I bet to say that, in general, yes. [1] https://www.youtube.com/watch?v=wApqCjHWF8Q https://www.youtube.com/watch?v=wApqCjHWF8Q

kevincox 4y ago
Basically any software, even something in C or C++ with manual memory management will scale more efficiently vertically than horizontally. Basically it would be better to run one twice as big as it would be to run two instances on the same machine. There is almost always some overhead or cache that will make the performance of fewer instances better. The only exception is resource contention but it is generally not too hard to fix the important cases. My default recommendation has always been to make instances "as big as possible, but no bigger". You may need 3 instances for redundancy, fault tolerance and graceful updates but past that you should probably scale up to near the size of your machine. There are obviously lots of complications and exceptions (for example maybe using most 3 instances uses 90% of your machines so you can't bin pack other processes there so it is better to use 4 instances at 70% as the machines will be used more efficiently) but bigger by default is generally a good option.
- JAlexoid 4y ago
  It's really not a big surprise, tbh. Making data available to a thread on the same core has exceptionally high bandwidth. Just that alone will help with scaling.
- brunoborges 4y ago
  > make instances "as big as possible, but no bigger" Exactly. But I am seeing more and more of the opposite: make instances as small as possible, and scale out (not up) if/when needed.

kevincox 4y ago

Basically any software, even something in C or C++ with manual memory management will scale more efficiently vertically than horizontally. Basically it would be better to run one twice as big as it would be to run two instances on the same machine. There is almost always some overhead or cache that will make the performance of fewer instances better. The only exception is resource contention but it is generally not too hard to fix the important cases. My default recommendation has always been to make instances "as big as possible, but no bigger". You may need 3 instances for redundancy, fault tolerance and graceful updates but past that you should probably scale up to near the size of your machine. There are obviously lots of complications and exceptions (for example maybe using most 3 instances uses 90% of your machines so you can't bin pack other processes there so it is better to use 4 instances at 70% as the machines will be used more efficiently) but bigger by default is generally a good option.

JAlexoid 4y ago
It's really not a big surprise, tbh. Making data available to a thread on the same core has exceptionally high bandwidth. Just that alone will help with scaling.
brunoborges 4y ago
> make instances "as big as possible, but no bigger" Exactly. But I am seeing more and more of the opposite: make instances as small as possible, and scale out (not up) if/when needed.