4 ms·

"Most systems can handle 1K-10K threads efficiently. If you need more connections than that, buy another server, a cheap one cost about $500." Terrible advice.

"Most systems can handle 1K-10K threads efficiently. If you need more connections than that, buy another server, a cheap one cost about $500." Terrible advice. Use NIO and do it properly. Creating 1 thread per connection is for unskilled programmers. In my experience once you get up to about 1k threads in Java you'll be wasting most of the CPU up switching between them. For comparison, it's trivial for any skilled programmer to use NIO to get up 100k+ connections without any real CPU usage.

sehugg 15y ago
Fer sure. Netty (http://www.jboss.org/netty http://www.jboss.org/netty) is your friend for handling massive numbers of NIO connections.
papaf 15y ago
Creating 1 thread per connection is for unskilled programmers. Its not as clear cut or obvious as it first seems: http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-... Edit: The original presentation is more interesting: http://www.mailinator.com/tymaPaulMultithreaded.pdf http://www.mailinator.com/tymaPaulMultithreaded.pdf
- speckledjim 15y ago
  The article is from back in 2008 so I'd firstly dispute its accuracy with modern JVMs. It's not just speed (The speed obviously changes as you change number of connections - 100k threads isn't going to beat 100k NIO connections). It's also that it's more scalable, better code, more maintainable, less prone to bugs, and you don't have to worry as much about concurrency issues. I'd argue that the lines of code is likely to be less also. And memory usage is likely to be less.
  - spullara 15y ago
    And performance suffers terribly when you don't need to scale the number of connections. Like anything, use it when it is needed.
  - gojomo 15y ago
    For better or worse, JVMs don't change that fast. (In 2008, the latest official release was JDK6, reaching 'update 12' in December. In July 2011, until the official release of JDK7 next week, the latest official JVM is still JDK6, 'update 26'.) Or do you know of specific NIO performance improvements between JDK6 in 2008 and JDK6u26, or in JDK7? 100k threads isn't going to beat 100k NIO connections That may be true but I wouldn't assume it with certainty without evidence. Threading has been improving, too. Tyma's 2008 presentation mentioned a JVM limit at the time of 16000 threads, but that may have changed as well. A single-threaded NIO implementation, using only one core, could very plausibly lose to a 100K-threaded implementation on a multicore machine. And once you split your NIO over a few threads to use cores effectively, you have most of the same concurrency worries as with connection-per-thread approaches.
deleted 15y ago
[deleted]
deleted 15y ago
[deleted]
cube13 15y ago
>Terrible advice. Use NIO and do it properly. Creating 1 thread per connection is for unskilled programmers. Depends on the application. If the connection is handling real-time financial data(for example), there might be enough heavy lifting on the data stream to warrant one thread per connection. A good programmer wouldn't have too many of these threads running on a given machine, however, because one thread can easily consume a full CPU core, and still need more. For connections that aren't doing much work, it's definitely better to aggregate connections onto a much smaller number of threads to avoid context switches. >In my experience once you get up to about 1k threads in Java you'll be wasting most of the CPU up switching between them. I agree. Thread context switching is extremely expensive. I've seen 10 thread applications consume full CPU time with context switches alone. There are a lot of tricks that can be done to mitigate it, like binding threads to certain CPU cores, making sure that threads that are communicating with each other are on the same CPU to avoid L2/L3 cache thrashing, etc. It's a lot of work, and requires a lot of knowledge about the code and system, but the results are staggeringly good. We've seen performance boosts on the order of 25% with single threaded applications, and boosts around 30-40% for multithreaded apps. Almost all of this is done outside the application code, however. I can't imagine any reason why you'd have 1k threads running on the same box in one process, to be honest. 1k+ threads seems to indicate a lot of threads doing very little work. If they're doing basically nothing, consuming less than 1% of the CPU each, it's better to try to consolidate the work they're doing into a much smaller number of threads. It's a lot of work, but will get pretty considerable performance gains.
peter_lawrey 15y ago
> Terrible advice. Use NIO and do it properly. Creating 1 thread per connection is for unskilled programmers. Sometimes the simple solutions are the best. Developing a complex and clever solution may your feel more skilled. Using a dispacther pattern often doesn't provides a real advantage and or the advantage it provides isn't needed. I would agree with the suggestion that if you are going to use a dispatcher pattern with NIO, use a library like netty which does this for you. (In any case you don't write it youself)
- speckledjim 15y ago
  It's like 100 lines of code. Write it yourself.
  - pkl 15y ago
    I have and if you use blocking IO or NIO its like twenty lines, code just about any one can read and maintain. Why make it more complicated than it needs to be?

sehugg 15y ago

Fer sure. Netty (http://www.jboss.org/netty http://www.jboss.org/netty) is your friend for handling massive numbers of NIO connections.

papaf 15y ago

Creating 1 thread per connection is for unskilled programmers. Its not as clear cut or obvious as it first seems: http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-... Edit: The original presentation is more interesting: http://www.mailinator.com/tymaPaulMultithreaded.pdf http://www.mailinator.com/tymaPaulMultithreaded.pdf

speckledjim 15y ago
The article is from back in 2008 so I'd firstly dispute its accuracy with modern JVMs. It's not just speed (The speed obviously changes as you change number of connections - 100k threads isn't going to beat 100k NIO connections). It's also that it's more scalable, better code, more maintainable, less prone to bugs, and you don't have to worry as much about concurrency issues. I'd argue that the lines of code is likely to be less also. And memory usage is likely to be less.
- spullara 15y ago
  And performance suffers terribly when you don't need to scale the number of connections. Like anything, use it when it is needed.
- gojomo 15y ago
  For better or worse, JVMs don't change that fast. (In 2008, the latest official release was JDK6, reaching 'update 12' in December. In July 2011, until the official release of JDK7 next week, the latest official JVM is still JDK6, 'update 26'.) Or do you know of specific NIO performance improvements between JDK6 in 2008 and JDK6u26, or in JDK7? 100k threads isn't going to beat 100k NIO connections That may be true but I wouldn't assume it with certainty without evidence. Threading has been improving, too. Tyma's 2008 presentation mentioned a JVM limit at the time of 16000 threads, but that may have changed as well. A single-threaded NIO implementation, using only one core, could very plausibly lose to a 100K-threaded implementation on a multicore machine. And once you split your NIO over a few threads to use cores effectively, you have most of the same concurrency worries as with connection-per-thread approaches.

deleted 15y ago

[deleted]

deleted 15y ago

[deleted]

cube13 15y ago

>Terrible advice. Use NIO and do it properly. Creating 1 thread per connection is for unskilled programmers. Depends on the application. If the connection is handling real-time financial data(for example), there might be enough heavy lifting on the data stream to warrant one thread per connection. A good programmer wouldn't have too many of these threads running on a given machine, however, because one thread can easily consume a full CPU core, and still need more. For connections that aren't doing much work, it's definitely better to aggregate connections onto a much smaller number of threads to avoid context switches. >In my experience once you get up to about 1k threads in Java you'll be wasting most of the CPU up switching between them. I agree. Thread context switching is extremely expensive. I've seen 10 thread applications consume full CPU time with context switches alone. There are a lot of tricks that can be done to mitigate it, like binding threads to certain CPU cores, making sure that threads that are communicating with each other are on the same CPU to avoid L2/L3 cache thrashing, etc. It's a lot of work, and requires a lot of knowledge about the code and system, but the results are staggeringly good. We've seen performance boosts on the order of 25% with single threaded applications, and boosts around 30-40% for multithreaded apps. Almost all of this is done outside the application code, however. I can't imagine any reason why you'd have 1k threads running on the same box in one process, to be honest. 1k+ threads seems to indicate a lot of threads doing very little work. If they're doing basically nothing, consuming less than 1% of the CPU each, it's better to try to consolidate the work they're doing into a much smaller number of threads. It's a lot of work, but will get pretty considerable performance gains.

peter_lawrey 15y ago

> Terrible advice. Use NIO and do it properly. Creating 1 thread per connection is for unskilled programmers. Sometimes the simple solutions are the best. Developing a complex and clever solution may your feel more skilled. Using a dispacther pattern often doesn't provides a real advantage and or the advantage it provides isn't needed. I would agree with the suggestion that if you are going to use a dispatcher pattern with NIO, use a library like netty which does this for you. (In any case you don't write it youself)

speckledjim 15y ago
It's like 100 lines of code. Write it yourself.
- pkl 15y ago
  I have and if you use blocking IO or NIO its like twenty lines, code just about any one can read and maintain. Why make it more complicated than it needs to be?