4 ms·

Reminds me of a frequent problem I observe with developers and operations teams moving Java workloads to Kubernetes. They focus on scalability, but miss the who

Reminds me of a frequent problem I observe with developers and operations teams moving Java workloads to Kubernetes. They focus on scalability, but miss the whole point of how the runtime (and not exclusive to the JVM) behaves - and prefers - when given proper resources. Suddenly, you see Kubernetes clusters of 2 to 4 vCPUs VMs/Nodes and containers with limits to 1,000 milicore (or 1 vCPU), and then the team solves the performance problem with dozens, sometimes hundreds of replicas for one particular microservice. Many developers don't even understand the impact the JVM has when running on a single core (yeap... you get Stop the World GC - aka Serial GC - by default). And then, the dev team decides to move to a new language because of performance and cost issues. And that by itself just brings many other problems. All they had to do was keep the same amount of CPU and memory, but less VMs and replicas, with more vertical resources. Depending on the workloads and the system, it is even possible to reduce the cost. All this is caused by the push in the industry for companies to go Cloud Native.

zhughes3 5y ago
So you can’t scale java horizontally?
- lvice 5y ago
  I don't know java, but that's not the message I got from the comment. It looks like with java, 3 VMs with 2 cores are better than 6 VMs with 1 core, which may be forgotten when configuring equivalent kubernetes services.
  - brunoborges 5y ago
    Exactly that.
- hvidgaard 5y ago
  I don't think you quite understood what (s)he wrote. The problem is using JVM without understanding some basic properties of it. Scaling is a matter of how you programmed it, not the language used.
- eklavya 5y ago
  Their point is that you can and probably even reduce cost, but you need to decide on the atomic resource unit and whether that makes sense (like having a measly single core, low ram instance/container).
- stepbeek 5y ago
  Any stateless service can be scaled horizontally. But the fixed cost of running the JVM is higher than the runtimes in other ecosystems (say a rust application as an extreme example). The JVM has had an outrageous amount of effort put into having it scale well for large heaps with high CPU counts. As the OP points out, GC in java-land isn't worth writing home about on 1vCPU. When you throw a 16GiB heap at it though, with 16vCPUs then you'll unlock some of the really interesting optimizations and garbage collectors that the showcase the JVM in an advantageous light. So it's not that the JVM can't scale horizontally - that's a matter of system architecture - it's that scaling it as 100 tiny nodes is far less impactful than having 5 large nodes.
dgb23 5y ago
I have amateur level JVM knowledge, but what you describe here seems intuitive. It feels like a waste to split a JVM application like this. I‘ve read a bit about new advancements in GC design, they’ve become highly parallel and layered. And then there are all sorts of JIT optimizations that may happen in a more comprehensive, larger service, right?
Joeri 5y ago
From seeing the difficulties containerizing java application servers at my own workplace I would say the hard part is making the application server stateless without sacrificing performance. Much of the older java (EE) ecosystem seems to assume a stateful application server that scales up, instead of a stateless one that scales out. Not that I think stateless is inherently superior, just that containerizing and scaling out sort of assumes moving state out of the application server and into redis, postgres, mongo, elastic, kafka, rabbitmq, …

zhughes3 5y ago

So you can’t scale java horizontally?

lvice 5y ago
I don't know java, but that's not the message I got from the comment. It looks like with java, 3 VMs with 2 cores are better than 6 VMs with 1 core, which may be forgotten when configuring equivalent kubernetes services.
- brunoborges 5y ago
  Exactly that.
hvidgaard 5y ago
I don't think you quite understood what (s)he wrote. The problem is using JVM without understanding some basic properties of it. Scaling is a matter of how you programmed it, not the language used.
eklavya 5y ago
Their point is that you can and probably even reduce cost, but you need to decide on the atomic resource unit and whether that makes sense (like having a measly single core, low ram instance/container).
stepbeek 5y ago
Any stateless service can be scaled horizontally. But the fixed cost of running the JVM is higher than the runtimes in other ecosystems (say a rust application as an extreme example). The JVM has had an outrageous amount of effort put into having it scale well for large heaps with high CPU counts. As the OP points out, GC in java-land isn't worth writing home about on 1vCPU. When you throw a 16GiB heap at it though, with 16vCPUs then you'll unlock some of the really interesting optimizations and garbage collectors that the showcase the JVM in an advantageous light. So it's not that the JVM can't scale horizontally - that's a matter of system architecture - it's that scaling it as 100 tiny nodes is far less impactful than having 5 large nodes.

dgb23 5y ago

I have amateur level JVM knowledge, but what you describe here seems intuitive. It feels like a waste to split a JVM application like this. I‘ve read a bit about new advancements in GC design, they’ve become highly parallel and layered. And then there are all sorts of JIT optimizations that may happen in a more comprehensive, larger service, right?

Joeri 5y ago

From seeing the difficulties containerizing java application servers at my own workplace I would say the hard part is making the application server stateless without sacrificing performance. Much of the older java (EE) ecosystem seems to assume a stateful application server that scales up, instead of a stateless one that scales out. Not that I think stateless is inherently superior, just that containerizing and scaling out sort of assumes moving state out of the application server and into redis, postgres, mongo, elastic, kafka, rabbitmq, …