4 ms·

I remember working in Java where we'd have huge threadpools that sat idle 90% of the time. It feels like you can eliminate most of this problem in other langua

I remember working in Java where we'd have huge threadpools that sat idle 90% of the time. It feels like you can eliminate most of this problem in other languages by using a much smaller pool and then leveraging userland concurrency/ scheduling. You probably don't want to have N cores and N + K threads, but in some languages you don't have much choice. Java has options for userland concurrency but they're pretty ugly and I don't think you'll find a lot of integration. Containers make this a bit harder, and the Linux kernel sounds like it had a pretty silly default behavior, but how much of this is also just Java?

le-mark 5y ago
I don’t think blaming JVM is productive, but identifying “JVM” as a proxy for “language runtime designed to optimize multi processor machines” is the core element here. One could imagine a vm or runtime that is async and multiprocess that also enforces quotas on cycles and heap such that these types of “noisy neighbor” events aren’t a problem. In this direction there have been solutions that haven’t caught on; a multi tenant jvm existed at one time, and at least one js implementation has this ability. I’ve often thought Lua would be ideal for this.
- staticassertion 5y ago
  Yeah to be clear I'm not saying all fault lies with the JVM here. But a lack of concurrency primitives exacerbates the problem by encouraging very large threadpools.
  - neerajsi 5y ago
    I haven't used it in anger, but it looks to me like the C# async compiler and library support helps reduce the need for large threadpools. But it also looks like the GC was a major contributor, so that would not be as influenced by the differences between dotnet and Java.
- yencabulator 5y ago
  > identifying “JVM” as a proxy for “language runtime designed to optimize multi processor machines” That'd be the wrong thing to pick as the proxy destination. Go is also a “language runtime designed to optimize multi processor machines” and, as the article explains, doesn't trigger this the same way. Try something like "total number of threads in all the pools used by the application overwhelms the CPU".

le-mark 5y ago

I don’t think blaming JVM is productive, but identifying “JVM” as a proxy for “language runtime designed to optimize multi processor machines” is the core element here. One could imagine a vm or runtime that is async and multiprocess that also enforces quotas on cycles and heap such that these types of “noisy neighbor” events aren’t a problem. In this direction there have been solutions that haven’t caught on; a multi tenant jvm existed at one time, and at least one js implementation has this ability. I’ve often thought Lua would be ideal for this.

staticassertion 5y ago
Yeah to be clear I'm not saying all fault lies with the JVM here. But a lack of concurrency primitives exacerbates the problem by encouraging very large threadpools.
- neerajsi 5y ago
  I haven't used it in anger, but it looks to me like the C# async compiler and library support helps reduce the need for large threadpools. But it also looks like the GC was a major contributor, so that would not be as influenced by the differences between dotnet and Java.
yencabulator 5y ago
> identifying “JVM” as a proxy for “language runtime designed to optimize multi processor machines” That'd be the wrong thing to pick as the proxy destination. Go is also a “language runtime designed to optimize multi processor machines” and, as the article explains, doesn't trigger this the same way. Try something like "total number of threads in all the pools used by the application overwhelms the CPU".