3 ms·
"The architecture uses two thread pools: one to run queries and one to get record values. Applications must carefully tune the number of threads in each pool so
by axod 19y ago
"The architecture uses two thread pools: one to run queries and one to get record values. Applications must carefully tune the number of threads in each pool so the queries to overwhelm the gets. Using a query thread pool with 2 threads and a get thread pool with 32 threads it was possible to perform 300 TPS on a small EC2 instances. "
This seems horrible design to me. Surely it must be possible to make it asynchronous and event driven - getting rid of the need for threads?
- st3fan 19y agoWhat is the problem with threads? I keep hearing that they are so difficult to work with but yet people ar successfully building very high performance Java applications that take advantage of many CPU cores. Simply by using the proven patterns and concurrency abstractions available to them.
- axod 19y agoHaving more threads than you have CPUs is just sloppy inefficient overcomplicated programming :/ If you have control of all the pieces, and don't need to interface to things that block, there's no reason to use more threads than cpus IMHO.
- Tamerlin 19y agoThere are times when it makes sense to use more threads than CPU's though; one good (and common) example is web application servers. Applications that use resources like databases that are I/O rather than compute bound can be significantly more efficient with additional threads beyond the number of CPU's available. Of course there's a limit; overkill doesn't do anyone any good.
- st3fan 19y ago"""If you have control of all the pieces, and don't need to interface to things that block, there's no reason to use more threads than cpus IMHO""" I only partially agree with this. It certainly makes no sense to have many threads running if you're talking about very CPU intensive application. We (Polar Rose) do very intense image processing (Java combined with native C++ code) and never have more than NCPU worker threads running at the same time. We do have a whole bunch of threads running around that for network services like JMS, JMX, etc. But you can basically ignore the overhead of that. It's like having idle UNIX processes. Or are you also saying that having more then NCPU processes running is sloppy? S.
- andrewf 19y agoAvoiding >NCPU threads is basically doing your context switching explicitly in code, rather than letting your OS and/or VM handle it. http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-... Seems it isn't necessarily more efficient in many cases.
- axod 19y agoIt's certainly more tasteful IMHO, which should count for a lot. Spamming the CPU/OS with thousands of threads is just ugly. How can it not be? Networking should be event driven. It's like having a thread for each pixel on the display or each button...
- st3fan 19y agoYou speak in emotional terms like 'Tasteful' and 'Ugly'. Which makes a technical discussion about threading really difficult.
- axod 19y agoWell in technical terms, using threads can never be as efficient as not using threads. It can be equal, but not better. A lot of these things are a matter of taste though. If you think it's a thing of beauty to create 5,000 threads, then good luck to you :)