11 ms·
Cost of a thread in C++ under Linux
- saagarjha 7y agoIs a std::thread a thin wrapper around pthreads on Linux?
- fwsgonzo 7y agoYep. And all the serialization is futex wait/wake.
- foo101 7y agoA related question if anyone knows good answers here. What programming languages' de-facto thread implementations are not wrappers around pthreads? I think Go has its own thread implementation? Or am I mistaken?
- saagarjha 7y agoRight, Go uses green (userspace) threads.
- daurnimator 7y agozig optionally uses pthreads (depending on if you link against libc or not)
- signa11 7y agoerlang has its own process/thread implementation with, iirc, 64b per process.
- toast0 7y agoThe docs [1] say: > A newly spawned Erlang process uses 309 words of memory in the non-SMP emulator without HiPE support. (SMP support and HiPE support both add to this size.) And a word is the native register size, so 4 or 8 bytes these days, so fairly small, but not 64 bytes small. [1] http://erlang.org/doc/efficiency_guide/processes.html http://erlang.org/doc/efficiency_guide/processes.html
- ghostwriter 7y agoGHC Haskell's runtime has a "default" light-weight thread system (forkIO) that schedules logical threads on the available operating system threads and parallelises them across available CPUs: - https://wiki.haskell.org/Parallelism#Multicore_GHC https://wiki.haskell.org/Parallelism#Multicore_GHC - https://stackoverflow.com/a/41485705 https://stackoverflow.com/a/41485705 - https://www.aosabook.org/en/posa/warp.html https://www.aosabook.org/en/posa/warp.html
- deleted 7y ago[deleted]
- pjmlp 7y agoJava does not specify the actual threading model, so you can get green threads (user space) or red threads (kernel threads). The upcoming Project Loom, intends to make it so that green threads become the default (aka virtual threads on Loom), but you can still ask for kernel threads, given that is what most JVM implementations have converged into.
- signa11 7y agoyes.
- abjKT26nO8 7y agoWith the caveat that the destructor crashes your program if you neither join nor explicitly detach the thread.
- ncmncm 7y agoBut, don't detach the thread.
- pjmlp 7y agoC++20 has apparently a fix for it with std::jthread, though. With all possible the learnings from Java, .NET, Erlang, TBB, Concurrency Runtime, and yet ISO C++ did not manage to get a proper concurrency story, and it full of traps like the one you mention. Another one is std::async, which might actually be synchronous, depending on a set of factors.
- isatty 7y agoWhy is there such a big difference in timing between Skylake and Rome? Something compiler specific? The number of steps required to create a thread should be identical. I’ll also be interested to see the same benchmark but using pthread_create directly.
- deleted 7y ago[deleted]
- thedance 7y agoCould be as basic as clock speed differences.
- sys_64738 7y agoBenchmarking in C++. Who knew!
- saagarjha 7y agoDaniel does most of his benchmarks in C++; it’s fairly well-suited for the task.
- shin_lao 7y agoGreat reminder. Even if you pre-create a thread (thread pool), when the task is small enough (less than 1,000 cycles), it is less expensive to do it in place (for example, with fibers), because of the cost of context switching.
- mac01021 7y agoWhy is the cost of switching threads so much higher than the cost of switching fibers?
- fwsgonzo 7y agoBecause threads are traditionally created and scheduled by the OS, so it inevitably involves a costly context switch both first into the kernel and then back again into the next thread, if one is ready. Userspace threads are more light-weight, but probably still worse than just using fibers and co-routines. Depends on your needs, I suppose.
- gpderetta 7y agoSwitching threads require entering the kernel which costs from a few hundreds to thousands of clock cycles (and it got worse from all the spectre/meltdown mitigations). A fiber switch can be done in less than 10 clock cycles.
- iforgotpassword 7y agoAgree. A few years ago I noticed a C program we used in production spawned a new thread for each incoming connection. Since the vast majority of these just served two small requests (think two HTTP gets) I tried adding a very simple thread pool that would keep up to four idle threads around. To make a thread wait for work I used an eventfd (Linux). I tried a linked list and an array for the idle threads. I tried protecting the get/return code with a mutex and spin lock, and then made it lock free with C11s atomics. Two days later I still couldn't get this to be faster than just spawning a new thread every time, so I gave up this experiment. It seems at least the Linux folks optimized the crap out of clone() over the last years.
- signa11 7y agoimho, if _cost_ of thread creation is where the bottleneck is, then more likely than not, you are doing things wrong.
- Ensorceled 7y agoThis is just another way of saying what the article just said.
- bluetomcat 7y agoFor CPU-bound tasks, it is best to pre-create a number of threads whose count roughly corresponds to the number logical execution cores. Every thread is then a worker with a main loop and not just spawn on-demand. Pin their affinity to a specific core and you are as close as possible to the “perfect” arrangement with minimized context switches and core-local cache data being there most of the time.
- fwsgonzo 7y agoYep. We do this on real hardware and never preempt. There is nothing more performant than that. Using IPIs (inter-processor interrupts) you can trigger events like "more work has been added" on each CPUs queue. Additionally, when you put all interrupts of a device solely on one specific CPU you won't have to lock anything. Some other things: pthreads generally have high cost, and that means C++ threads do too. pthreads have quite a few features that you regularly don't use, which you can skip completely using fibers and coroutines.
- Koshkin 7y ago> never preempt I don’t think this is a realistic expectation on Linux and, especially, Windows which runs hundreds of threads of its own you don’t want to know about. (Besides, we must remember that multithreading was invented and found quite useful in the era of ”single-core” processors.)
- emidln 7y agoLinux/CFS has an isolcpus feature that can be used to tell the kernel to never schedule things on a given CPU. This is useful when latency matters.
- ncmncm 7y agoI learned recently that even with isolcpu, and nohz, and interrupts directed elsewhere, the kernel will still pause the thread on the isolcpu if it has mmapped a file (e.g., to report stats) and the kernel decides it's time to copy the bits to disk. If you don't want stalls, only map writable files on a tmpfs volume. To snapshot the file, copy it to another file on the same volume, and then snapshot the copy.
- brainscdf 7y agoMy personal best practice is to always create a thread pool on program startup and distribute your tasks among the thread pool. I use the same best practice in all other languages too. Is this best practice sound or can it lead to problems in some corner cases?
- hinoki 7y agoThere are lots of details that might cause problems: * Do your tasks block? How many threads do you need to make sure you can use all your CPUs. * Do your tasks access different sets of memory? Would keeping similar tasks on the same CPUs reduce cache misses. * Do your tasks have different priorities? You might need a pool for each priority. For a UI program that isn’t doing anything really intensive or real-time, having a common thread pool makes a lot of sense, and can reduce resource use (stacks add up once you get to many 10s or 100s of threads...), and improve latency (a work queue with many threads will get more CPU than another with the same amount of work but fewer threads)
- londons_explore 7y agoCase in point: I used nodejs for a project, and assumed that "it's all javascript on one thread" would leave threading issues behind. My application curiously stopped responding whenever I had 5 or more users. Connected users could continue to do anything, but new users couldn't connect, and existing users sessions would hang when executing any code that wrote to a logfile, making debugging even harder. Using the nodejs debugger, the internals of write(...., cb) were just never calling the done callback. After hours of head scratching I found that most IO from nodejs is not asynchronous and callback based as the docs suggest, but is in fact blocking IO done from worker threads. My process was using pipes to communicate with other processes, and those pipes were doing blocking writes, and when blocked, the worker thread was blocked. There are 4 worker threads by default, so whenever 5 users were using the system, all worker threads were tied up and it would fail. It would have been nice for nodejs to at least have printed to the console "All worker threads busy for >1000ms. See nodejs.com/troubleshooting/blockingfileio.htm" or something.
- maayank 7y agoWhy the relative high cost of threads on ARM? If anything, I'd imagine it is more geared towards "massive parallel" scenarios (i.e. dozens of cores).
- Koshkin 7y agoIntel’s excellent TBB library is the answer to all your worries about threads in C++. (IMHO it should be made part of the standard library.)
- ncmncm 7y agoAll your worries, if throughput is all you worry about, and not latency. Or, if you have interaction between threads. Or, if you might need to run on other archs. An equivalent to TBB or GCD will be in C++23 std libraries, but you can often do better with coroutines, in 20. TBB and GCD still need to sychronize sometimes, and they randomize workload assignment, which is bad for cache locality (i.e. bad). If you can arrange static assignment and avoid need to synchronize, you can do better, sometimes much better.
- Koshkin 7y ago> other archs See, for instance, https://www.theimpossiblecode.com/blog/intel-tbb-on-raspberry-pi/ https://www.theimpossiblecode.com/blog/intel-tbb-on-raspberr....
- ncmncm 7y agoInteresting, thank you.
- pjmlp 7y agoThe problem with C++23, is that it will be mostly usable around 2025, and C++20 co-routines still don't have a co-routine aware standard library, right?
- ncmncm 7y agoExecutors, which abstract threads, coroutines, fibers, vector units, GPUs, and very possibly even spin-polling isolated cores, will be integrated in 23. In the meantime you have the core language features, if you don't need or want abstraction, and Boost asio, which will get everything ahead of time. But std algorithms integrated to use the abstract executors optimally may not surface until 23, IIUC. I don't know why you would need to wait for 2025.
- known 7y agoOn any architecture, you may need to reduce the amount of stack space allocated for each thread to avoid running out of virtual memory http://www.kegel.com/c10k.html#limits.threads http://www.kegel.com/c10k.html#limits.threads
- Koshkin 7y agoNot making a jab at what you are saying, but to me “running out of virtual memory” has always sounded like a crazy thing, like running out of address space. Sure, given enough disk space, your program might get (quite) a bit slower, but it should still chug along just fine. Yet, running out of virtual memory is indeed still a thing, especially in Windows (a workaround being using memory-mapped files).
- CJefferson 7y agoIs this even possible on a 64 bit architexture? The default stack size is, I think, 2mb, and i have previously allocated terabytes of VM space without issues.
- wbkang 7y agoNo this is more of a 32bit issue.
- nurettin 7y agoI can't believe this link is still relevant after more than 15 years.
- drmeister 7y agoThreads are very expensive if you start throwing C++ exceptions within them in parallel. You see the overall time to join the threads increases with each thread you add. There is a mutex in the unwinding code and as the threads grab the mutex they invalidate each other's cache line. I wrote a demo to illustrate the problem https://github.com/clasp-developers/ctak https://github.com/clasp-developers/ctak MacOS doesn't have this problem but Linux and FreeBSD do.
- wwarner 7y agoWow, I wouldn't have guessed that! Nice data!
- drmeister 7y agoThank you. We have developed a Common Lisp implementation that uses LLVM and interoperates with C++ and uses C++ exception handling to unwind the stack. Common Lisp code relies on stack unwinding a fair bit. Imagine my surprise when my fancy multi-threaded compiler can't get out of first gear (tops out at ~150% cpu) on Linux. Sheesh.
- gumby 7y agoI’ve been really interested in clasp; what’s the current state? Can’t see any recent posts that summarize it.
- drmeister 7y agoIt's going well. I'll post something soon. We've just been working on it quietly. We have multithreading, unicode, cffi etc, good debugging support, cross-language profiling and more.
- ajross 7y agoDid gcc/libstdc++ have the same problem? The report you link is for clang, it looks like.
- boulos 7y agoI find Eli Bendersky’s writeup [1] more useful as it actually goes closer to the details. For readers less familiar, it also makes it more clear what the time spent will depend on (how much state there is to copy). Eli’s post is actually a sub-post of his “cost of context switching” post [2] which is more often applicable (and helps answer all the questions below about threadpools). [1] https://eli.thegreenplace.net/2018/launching-linux-threads-and-processes-with-clone/ https://eli.thegreenplace.net/2018/launching-linux-threads-a... [2] https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/ https://eli.thegreenplace.net/2018/measuring-context-switchi...
- amiga_500 7y agoThanks both of these were excellent.
- hrgiger 7y agoUsing taskset pinning my numbers improves: $taskset --cpu-list 8 ./costofthread avg: 11000~ $taskset --cpu-list 8,11 ./costofthread avg: 33000~ $./costofthread avg: 60000~
- sproketboy 7y agoJVM Fibers FTW