5 ms·

Thousands of Threads and Blocking I/O (2008) [pdf]

25 points by logn 11y ago

smegel 11y ago
Interesting all his graphs top out at 1000 threads. Wonder what they would look like if they went to 5000, or 50,000 threads.
- dboreham 11y ago
  5k would be fine, 50k would not, with current hardware and JVM, based on my experience. The pain point again in my experience, is not any of the issues cited in that presentation, but rather that GC gets bound up in a wad eventually when you have large (>10k) numbers of threads in a JVM (and any reasonably complex application). This is because the object graph becomes highly interconnected through the Thread and associated TLS objects. YMMV of course. Definitely agree that the "async is fast" myth needs debunking, though.
  - stefano 11y ago
    Do you think 10 JVMs running 5k threads each would perform better? Assuming you have enough memory for such a setup...
    - dboreham 11y ago
      Probably, but then you have to manage 10x the JVMs which is no picnic. Pick your poison. Better to re-write to not use zillions of threads.
RyanZAG 11y ago
Vert.x ( http://vertx.io/ http://vertx.io/ ) is an interesting point in this discussion that was released after the paper was written. It uses the async reactor pattern, but allows multiple reactors to run - 1 per CPU core. It also handles events and state so that each unit runs only within a single thread which completely eliminates the need to worry about threat synchronization. Basically a multithreaded node.js engine. Along with using javascript or Java8 lambdas and promises, it makes for very nice code. The async style also allows for faster code in fairly typical web server cases. Consider a request that needs to do an HTTP lookup and 3 database calls. In a blocking I/O case, this needs a memory heavy thread and each I/O operation must be done sequentially even if we need the result from all 4 operations to give the result. With async I/O, we can send off all 4 operations without requiring the memory usage of a thread, and all 4 operations complete much faster as they run in parallel.

25 points by logn 11y ago

Interesting all his graphs top out at 1000 threads. Wonder what they would look like if they went to 5000, or 50,000 threads.

dboreham 11y ago
5k would be fine, 50k would not, with current hardware and JVM, based on my experience. The pain point again in my experience, is not any of the issues cited in that presentation, but rather that GC gets bound up in a wad eventually when you have large (>10k) numbers of threads in a JVM (and any reasonably complex application). This is because the object graph becomes highly interconnected through the Thread and associated TLS objects. YMMV of course. Definitely agree that the "async is fast" myth needs debunking, though.
- stefano 11y ago
  Do you think 10 JVMs running 5k threads each would perform better? Assuming you have enough memory for such a setup...
  - dboreham 11y ago
    Probably, but then you have to manage 10x the JVMs which is no picnic. Pick your poison. Better to re-write to not use zillions of threads.

RyanZAG 11y ago

Vert.x ( http://vertx.io/ http://vertx.io/ ) is an interesting point in this discussion that was released after the paper was written. It uses the async reactor pattern, but allows multiple reactors to run - 1 per CPU core. It also handles events and state so that each unit runs only within a single thread which completely eliminates the need to worry about threat synchronization. Basically a multithreaded node.js engine. Along with using javascript or Java8 lambdas and promises, it makes for very nice code. The async style also allows for faster code in fairly typical web server cases. Consider a request that needs to do an HTTP lookup and 3 database calls. In a blocking I/O case, this needs a memory heavy thread and each I/O operation must be done sequentially even if we need the result from all 4 operations to give the result. With async I/O, we can send off all 4 operations without requiring the memory usage of a thread, and all 4 operations complete much faster as they run in parallel.