5 ms·

Not just the JVM, lot of libraries build thread pools based on number of processors and some of them even hard code the multipliers (X the number of processors

by r4um 9y ago

Not just the JVM, lot of libraries build thread pools based on number of processors and some of them even hard code the multipliers (X the number of processors etc). Setting up each one becomes a lot of work fast, so we wrote a LD_PRELOAD[1] hook that overrides sysconf[1] _NC_PROCESSORS* call to get number of processors availabale/online to a certain specified value and is baked in the docker image by default during builds. [1] http://man7.org/linux/man-pages/man3/sysconf.3.html http://man7.org/linux/man-pages/man3/sysconf.3.html

_pmf_ 9y ago
That's horrible and irresponsible; I love it.
MaxBarraclough 9y ago
Can we expect that kind of horror to fade away as Java evolves? In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) As I understand it, the JVM is rather behind in this department. (Not to mention async/await.)
- jacques_chester 9y ago
  Java has these concepts too (ExecutorService since at least Java 5, circa 2004). The problem is not with the JDK libraries, it's been in the JVM's assumption that it's running on bare metal or a VM. Linux containers leak information about the "true" environment in a way that upset JVM assumptions before 9 and 10.
- badpun 9y ago
  > In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). > (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) Java had thread pools and tasks since 2004 (https://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html https://docs.oracle.com/javase/1.5.0/docs/api/java/util/conc...). For typical Java programs, no one is writing their own thread pool functionality as far as I know.
- pjmlp 9y ago
  java.util.concurrent is actually more advanced that TPL, with contributions from Doug Lea. Java just lacks async/await primitives.
  - bunderbunder 9y ago
    java.util.concurrent certainly has more buttons and levers you can fiddle with. I could probably burn at least a week trying to grok the whole thing.
- bunderbunder 9y ago
  .NET has the same problem: Each process would create one thread pool thread per CPU core, and you could end up with excessive context switching. Java possibly makes it a bit easier to work around, really, in that Java forces you to initialize your thread pool, so it wouldn't feel quite so weird to add a "# of threads in thread pool" setting to an app config file for something. I'm guessing that's not the Docker way of doing it, though.

by r4um 9y ago

_pmf_ 9y ago

That's horrible and irresponsible; I love it.

MaxBarraclough 9y ago

Can we expect that kind of horror to fade away as Java evolves? In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) As I understand it, the JVM is rather behind in this department. (Not to mention async/await.)

jacques_chester 9y ago
Java has these concepts too (ExecutorService since at least Java 5, circa 2004). The problem is not with the JDK libraries, it's been in the JVM's assumption that it's running on bare metal or a VM. Linux containers leak information about the "true" environment in a way that upset JVM assumptions before 9 and 10.
badpun 9y ago
> In the .Net world there's the ThreadPool class to manage the single process-wide thread-pool, and the Task class to enqueue and orchestrate concurrent jobs (it uses ThreadPool and almost completely abstracts away the thread management). > (You could write your own thread-pool, of course, but for most code that wouldn't make sense.) Java had thread pools and tasks since 2004 (https://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html https://docs.oracle.com/javase/1.5.0/docs/api/java/util/conc...). For typical Java programs, no one is writing their own thread pool functionality as far as I know.
pjmlp 9y ago
java.util.concurrent is actually more advanced that TPL, with contributions from Doug Lea. Java just lacks async/await primitives.
- bunderbunder 9y ago
  java.util.concurrent certainly has more buttons and levers you can fiddle with. I could probably burn at least a week trying to grok the whole thing.
bunderbunder 9y ago
.NET has the same problem: Each process would create one thread pool thread per CPU core, and you could end up with excessive context switching. Java possibly makes it a bit easier to work around, really, in that Java forces you to initialize your thread pool, so it wouldn't feel quite so weird to add a "# of threads in thread pool" setting to an app config file for something. I'm guessing that's not the Docker way of doing it, though.