6 ms·
The moral is that threads and asynchronous programming are not mutually exclusive. No matter how much you can do with a single thread, in a highly parallel env
by dumbfounder 9y ago
The moral is that threads and asynchronous programming are not mutually exclusive.
No matter how much you can do with a single thread, in a highly parallel environment (such as a web server), on a cpu with multiple cores (the norm), you can (almost always) do more with multiple threads.
- brianwawok 9y agoSure for threads up to cores / 2. Past that it is likely a throughput and latency loss.
- virmundi 9y agoDoes anyone have numbers to back this claim? Seems like for IO focused software you'll have a lot of dead CPU time. If you have more threads than cores, you can take advantage of that.
- ma2rten 9y agoAsynchronous programming is also solving that problem by using an event loop.
- jandrewrogers 9y agoNo dead CPU time in well-designed software. If you care about I/O performance then you are managing that in user space asynchronously. No thread is waiting for I/O to complete or anything like that unless there is literally nothing else to do (in which case more threads won't help). There is no performance benefit to adding more threads, they would increase CPU state/cache thrashing which will slow things down. The main benefit of using a lot of threads is that more programmers know how to design software that way because it was taught for so many years; it isn't good for performance on modern hardware.
- jorangreef 9y ago"Seems like for IO focused software you'll have a lot of dead CPU time. If you have more threads than cores, you can take advantage of that." Yes, see here for more discussion on this: https://github.com/ronomon/crypto-async#adjust-threadpool-size-and-control-concurrency https://github.com/ronomon/crypto-async#adjust-threadpool-si...
- dboreham 9y agoMore like <cores> * 2
- brianwawok 9y agoNot for busy spinning threads. Source - spent a year performance tuning a matching engine.
- frankpf 9y agoCan you elaborate on why cores / 2 is the "ideal" number for throughput and latency?
- brianwawok 9y agoWell HT is generally crap. In most cases if you go up to a random server, it is going to have HT. So cores / 2 will get you ballpark. You really should go in the bios and shut down HT if you are into latency at all. Then you can just do core or core - 2 threads. So once you are there, the next step is to busy spin at a Thread Per Core (TPC). You have 10 cores, you find 10 (or more realistically 8-9 to leave the OS some spare cores to muck with) threads, and busy spin them at 100% doing work. You never let the linux scheduler touch them. If this stuff interests you, there are some cool papers by lmax and the disruptor and gil tene that talk about this stuff.. The amount of work you can do with a single 10-20 core CPU is amazing.
- dboreham 9y agoBusy spinning threads are a bug that needs to be fixed first..
- reza_n 9y agoCompletely depends on if these are actual "worker threads" or just dumb threads which have been over scheduled. If these are "worker threads", new workers are scheduled when all other workers are blocked and work is available. This guarantees that idle CPU will always be scheduled with work. When workers unblock, they continue execution on their work and start picking up new work. The worker scheduler will stop scheduling new workers and the extra workers will go back to the pool of inactive workers. This is the idea behind a worker or thread pool, they get scheduled when its possible to do more work. A poor implementation of this idea will just hand work off to threads and then expect them to all execute in parallel, which will lead to negative performance impact. The problem with (some implementations of) async workers is that just scheduling more will result in the latter problem, work split across too many workers. This is especially true for workers implemented as entire processes. The problem this post brings up is that 1 to 1 mapping of workers to cores suffers from idle CPU due to unanticipated blocking. The general solution is to smartly distribute work over a pool of workers, and not blindly schedule a fixed number workers.
- solotronics 9y agoI think it should make a difference if the threads end up being IO limited or CPU limited.