6 ms·
Threads, Threads, and More Threads
- nhoughto 5y agoHmm feels like there is a bit of a misunderstanding on the meaning and cost of threads and how the OS schedules work from threads onto logical CPUs and what state they are in. It is expected that processes might use more than 1 thread, and it is expected that you would have more processes than logical CPUs. Can not-dead process exist without a thread at all? I'd expect not. 7000+ threads does seem like a lot though and agree there is the general lack of caring about resource consumption in all forms, eg Electron apps. At this point having lots of threads seems almost expected by various OSs and something they solve for to reduce the cost of it so the user doesn't have to worry about it.
- jayd16 5y ago>Can not-dead process exist without a thread at all? I'd expect not. Isn't this the case for some older cooperative threading OSes? It's possible in theory, anyway.
- astrange 5y agoIf all the threads are idle then you could exit all of them; this makes sense if they're using async system calls. The OS would just create a new thread in the process to resume it again. Of course, there's a lot of tradeoffs between that and just keeping an idle thread around.
- mwcremer 5y ago> Most threads are waiting for something – this is not the ideal activity for a thread. A thread should run (useful) code. Seems like that is an ideal activity for a thread, and in fact why most people use them? Combining all those waits into a single thread/process means also creating a dispatcher which then either spins up a thread to do work (ahem), or blocks all other useful work. If the cost is low (and for thread it usually is), having lots of blocked threads often makes the code easier to write and maintain.
- macintux 5y agoAgreed, I don’t understand this post at all. This is where Moore’s Law has brought us: if we want better performance, we need to take advantage of parallelism. Thus more threads.
- tcfhgj 5y agoyou can't have more parallelism by simply increasing #threads. Parallelism is bound by CPU cores. Threads need to scale with CPU cores, however simply increasing #threads doesn't help. You need to split tasks of your program such that more tasks can be executed in parallel while not adding more overhead than you gain by parallelism. Then you can distribute those tasks on as many threads as possible (while still being bound by #cores).
- astrange 5y ago> you can't have more parallelism by simply increasing #threads. Parallelism is bound by CPU cores. It's bound by the slowest shared resource in the system (Amdahl's law). That could be disk I/O, memory bandwidth, or power budget.
- Icathian 5y agoThat isn't what Amdahl's law says. Amdahl's law is an equation describing the theoretical paralellizability of a given task. It doesn't have anything to do with systems bottlenecking on one resource vs another.
- Terry_Roll 5y ago> Seems like that is an ideal activity for a thread Agree, because spawning a new thread and killing a thread carries its own overhead so better to have the thread sleep or be paused by the OS and cpu scheduling. Its all about getting the balance right, but I have noticed dont net apps run their gui on a single thread instead of a window(s) linked to a thread, like in a multi document interface (MDI) setup.
- 5y ago
- marginalia_nu 5y ago> In an ideal world, the number of threads in a system would be the same as the number of logical processors – any more and threads might fight over processors, any less and you’re not using the full power of the machine. Obviously, each “normal” process must have at least one thread running whatever main function is available in the executable, so on my system 521 threads would be the minimum number of threads. Still – we have over 7000 Seems to not account for the fact that threads will sleep while waiting for I/O.
- tcfhgj 5y agoin what way? According to my understanding you don't need additional threads for this.
- marginalia_nu 5y agoIf a thread is waiting for I/O or a mutex, it will yield and let another thread use that CPU time to do something. This doesn't work if you don't have more threads than logical cores, because then there is no job that can take over. We've had this type of cpu time sharing far longer than we've had multicore CPUs.
- tcfhgj 5y agoDoes this still hold if you use async/await which simply would push a different task on the thread while the other one is waiting?
- marginalia_nu 5y agoAs far as the operating system is concerned, a thread is a thread. Async/await is a pattern for hiding the implementation details of multitasking or faking it in languages with limited support, it still largely works the same, except you see it as the thread being yielded to rather than from in explicit threading.
- 5y ago
- colanderman 5y ago> In an ideal world, the number of threads in a system would be the same as the number of logical processors ... no. Not if you rely on blocking system calls. Or wait for anything that isn't the CPU. Or run more than one application per CPU on your system. Or manage concurrent code flows at any level higher than source code.
- tcfhgj 5y agoin an ideal world you don't use blocking system calls?
- colanderman 5y agoYep, that's a great goal, and I/O-intesive applications do this, managing concurrent workflows with custom user-mode schedulers. But now you have more than one application on the machine? Or even, code from multiple vendors which all need to issue I/O concurrently? How do you integrate their work with your custom scheduler? You need some centralized, system-wide scheduler everyone agrees on. That's what threads are.
- tcfhgj 5y agoDoes it make sense to ask for a system wide scheduler that manages tasks (tasks in terms of the state machine behind async await) instead of threads? Currently every app that uses tasks has it's own executor with a corresponding thread pool, which may be more efficiently used if it was shared by using a single async executor across all processes. of course, it requires to agree on one executor which doesn't sound realistic, but would it make sense in general?
- colanderman 5y agoThis is essentially how an Erlang system works. There's a single executor with one "CPU" thread per CPU, and a bunch of "I/O" threads, the sum of which are shared among all Erlang applications executing under the runtime. Erlang can get away with this because it's interpreted. But also, this is essentially what threads are... the main advantage of threads being, they coordinate raw assembly code, instead of something interpreted or JITted. OS schedulers are pretty well optimized by this point. The major overheads relate to maintaining security boundaries (e.g. swapping out the TLBs on context switch) and the cost of starting up a thread. Just having threads sitting around doesn't really cost anything.
- chrisseaton 5y agoWhat did this guy think was the purpose of threads before multi-processors were common? What does he think is the point of green threads? Fibres? Why does he think some languages only allow one thread to run at a time but still have threads? Answer to all these questions is: he’s mistaken about the original purpose of threads.
- astrange 5y agoAs he says, the ~only cost of an idle thread is memory. In that case, I wonder why GVFS.Mount.exe in his screenshot is using 1GB.
- IgorPartola 5y agoProbably a bunch of in-RAM cache?
- IgorPartola 5y agoThe comments here remind me of MongoDB is web scale. You can have thousands of idle threads on a machine with one or a few CPU cores. Sure if you have several of them competing for CPU time you will lose efficiency. But at the point where the total duty cycle is less than capacity - context switch overhead you will not see any I’ll effects. I know because I have tried this. 4096 threads on a dual core $5/month VPS did beautifully for IO-bound tasks. Also the author is examining a desktop machine. Of course it will have a bunch of idle processes. In fact I would argue that if the machine was running 100% CPU usage by a few threads all day long the author would be quite upset at the electric bill, fan noise, and the heat generated. Desktop applications are not throughput bound. They are latency bound and having a bunch of threads on standby is normal.
- asdfasgasdgasdg 5y agoIn the modern era, threads are more of a memory problem than a compute problem. If you spawn a thread per request (or several) it can be an issue by maintaining too much stack space. But I don't know of any reliable benchmark showing that threads or events outperform in real systems, except in cases of extremely high numbers of simultaneous contexts (i.e. very high number of ongoing requests/very low cost per request of operation).
- staticassertion 5y agoYou could just spawn threads with smaller stacks. Also afaik that memory won't ever get paged in if it isn't used? So I don't know if it matters much. Also, idle threads aren't a problem, even on a system under load. Especially on a desktop. Also also, there's no global threadpool, which means if you have K programs on N codes and K > N it's a moot point trying to save threads. Any desktop has K > N. So reducing thread usage in all of your processes is unlikely to help.
- IgorPartola 5y agoYou said what I would have. Stack size of 100k is plenty for lightweight threads. 100MB for 1000 threads isn’t much for a modern system.
- yuliyp 5y agoThis post has the thesis that threads are expensive in some way. Modern OS's schedulers don't do work that's O(blocked threads) to schedule. They require a small amount of memory to maintain some data structures to track it in the kernel, and some memory for a stack in userspace. Threads as a mechanism for separating out the concerns that can run independent of each other is perfectly fine. While eventloops and thread pools are good, allowing for things like, say, a stats aggregator to not need to share a work queue with an application's work just makes for an easier-to-reason-about system.
- perryizgr8 5y ago> And since a negative thread count is not possible (even zero is almost impossible) I wonder why zero is only "almost" impossible. Don't you always need one thread to be called a process?