7 ms·

Posix Threads Programming

165 points by dvaun 6y ago

intc 6y ago
Nice share - Very clear and informative. Wrote some time ago a simple threaded implementation of Sieve of Eratosthenes utilizing POSIX threads (in C): https://github.com/intc/c-notes/blob/master/threads/4-thrd_eratosthenes/erat.c https://github.com/intc/c-notes/blob/master/threads/4-thrd_e...
zerof1l 6y ago
Pthreads will always have special place in my heart. It was my first serious introduction to the parallelism concepts.
- senderista 6y ago
  Ditto, via David Butenhof's excellent book _Programming with Posix Threads_. I didn't know what a mutex or condition variable was before I read that book.
  - gautamdivgi 6y ago
    I still have that book on my shelf.
  - johnisgood 6y ago
    Woah, it is that old! Been some time. Now I feel old.
  - jjav 6y ago
    Lives on my prime shelf space, along with TAOCP and the Stevens books.
- cylde_frog 6y ago
  I was going to comment the exact same thing. I have fond memories of learning about parallelism in university and using pThreads. It was such a mind-blowing and novel thing at the time when all I had been exposed to was sequential single threaded programming.
rramadass 6y ago
Good succinct tutorial. For people interested in studying Concurrency (Threading is just a subset of the model) i highly suggest the following book; Foundations of Multithreaded, Parallel, and Distributed Programming by Gregory Andrews. On the low-level OS/Hardware side (i.e. SMP/Caches etc.) UNIX Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers by Curt Schimmel is excellent.
- Daho0n 6y ago
  Thank you for those. Do you happen to know a good one on C#? Trying to learn more about the Windows side.
  - dauchande 6y ago
    C# should use Async and Await. Steven Cleary's book is the standard for C#: Concurrency in C# Cookbook: Asynchronous, Parallel, and Multithreaded Programming For Windows, I liked: Concurrent Programming on Windows by Duffy
37ef_ced3 6y ago
Here is an example: NN-512 uses pthreads. It generates a tiny, standalone, highly scalable C99 work-stealing thread pool in every C file. The work items are simply coordinates, similar to the threadIdx/blockIdx coordinates in Nvidia's CUDA. For example, all the structs and functions starting with "Example32Threader" in this file: https://nn-512.com/example/32#3 https://nn-512.com/example/32#3 The implementation ends at Example32ThreaderDo1 The key is to avoid writing to shared cache lines whenever possible, and to keep a bitmap in the central struct (the "hub") that avoids rediscovery of the fact that a thread has no remaining work to be stolen (thereby avoiding a "thundering herd" type problem: https://en.m.wikipedia.org/wiki/Thundering_herd_problem https://en.m.wikipedia.org/wiki/Thundering_herd_problem) Another key is to never take any short-cuts in synchronization. Rather than doing something subtle/clever to avoid mutexes (etc.), instead make sure the units of work are big enough so that synchronization costs are negligible with respect to the overall computation
- gary_0 6y ago
  > Rather than doing something subtle/clever to avoid mutexes (etc.), instead make sure the units of work are big enough so that synchronization costs are negligible with respect to the overall computation This seems to be working well in my current project, even though I originally planned to refactor things to be "cleverer". A momentary Big Stupid Mutex lock followed by a good chunk of independent work is a nicely boring design for worker threads that seems to work fine. Another trick I previously used was just leaving work involving contended resources aside until after all the worker threads finish, instead of trying to resolve contention with waits or more complex code. If the leftovers from each thread are <1% of the total work, it can just be cleaned up single-threaded style later. Sometimes it's enough to just detect potential contention and leave the trickier work to simpler code.
screwgoth 6y ago
Wow !! This is the same page from where I started learning about Pthreads way back in 2004. 16 freaking years ago. I loved it then, love it now.
pantulis 6y ago
This is a great resource, and it's also well written. I did a lot of C thread programming 22 years ago (plus some CORBA), and this would have been very helpful, basically resumes a whole semester of university classes in one web page! Lazy question: are there any details of how pthreads are implemented for each operating system?
MisterTea 6y ago
Concurrency is easy if the library is well thought out and clean. So I much prefer the Plan 9 approach using the CSP thread(2) library: http://man.postnix.pw/9front/2/thread http://man.postnix.pw/9front/2/thread, Example program: https://github.com/mischief/9problems/blob/master/sys/src/libthread/example.c https://github.com/mischief/9problems/blob/master/sys/src/li... Wonderful easy to use library which is built on top of rfork and uses channels and alternates to synchronize the procs. Heap memory can be shared across processes (e.g. malloc'd memory or a segment(3)) so passing data around is just passing pointers to objects in shared memory. Further reading which demonstrates the library and how it influenced Go can be found here: https://seh.dev/go-legacy/ https://seh.dev/go-legacy/
albmoriconi 6y ago
This wonderful tutorial is the same I used to learn Pthreads around 10 years ago. The training section of the LLNL website [1] offers a lot of solid tutorials; I used the OpenMP and MPI ones for my Parallel Computing class some years ago and they helped me to sistematically learn (and learn to apply) the material. I love them! [1] https://hpc.llnl.gov/training/tutorials https://hpc.llnl.gov/training/tutorials
marmight 6y ago
For folks interested in threads in C, I also recommend reading "Threads Cannot be Implemented as a Library" [1]. The summary is that Pthreads mostly works correctly as a library (1) because its functions execute memory fence instructions and (2) because its functions are treated as opaque functions by the C compiler, i.e., functions that might read or write from any global variable. However, these properties do not generate thread-safe code under a number of conditions such as under the presence of certain compiler optimizations. Thus, the paper argues that the compiler must be aware of threads and that threads cannot be implemented purely as a library. [1] https://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf https://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf
- jcranmer 6y ago
  The language basis for threading is now the C11/C++11 memory model [1], even if you don't directly use it--it's baked into the compiler IR. I would argue that in the 2020s, you should be using C11/C++11 threading instead of direct pthreads if you're writing new code. (And yes, Hans Boehm was one of the people instrumental in getting the threaded memory model adopted.) [1] If you really want to be pedantic, it's the current verbiage that rules, independent of the actual language version you request. This is one of those situations where trying to describe the semantics more formally is challenging, so a lot of the changes in the specification aren't supposed to actually be changing how things are compiled but how things are described in the text.
  - wahern 6y ago
    In some ways the POSIX API is better. For example, the POSIX API permits you to statically define and initialize pthread_mutex_t, pthread_once_t, and similar objects. Without static initialization you're stuck in a catch-22 because to initialize such objects at runtime you have to rely on some other implicitly serialized control flow, such as by invoking initializers from main(), which is particularly cumbersome and error prone for libraries. C++ has it a little easier because, AFAIU, the language itself supports static constructors--i.e. the ability to run [mostly] ad hoc code to initialize static objects at runtime (specifically load time). The C11/C++11 threading API lacks static initializers because when the specification was being developed Windows light-weight threading primitives didn't support such a capability. Basically, the C11/C++11 API had to take a lowest common denominator approach, which in most cases meant dropping useful features from the POSIX API so Microsoft didn't have to reimplement their primitives. (The hope was that C11- and C++11-compliant implementations would become widely available immediately if the standard made it easy to wrap existing OS primitives, but unfortunately implementations were--and arguably still are, especially for C--many years late.) Also, the memory model is mostly distinct from the threading APIs. The memory model was more important for nailing down atomics in C11 and C++11. The threading APIs are much higher level, and because they rely on opaque types and function calls, their implementation and the necessary compiler support (ISA memory barriers, compiler code motion barriers, etc) was largely transparent.
bch 6y ago
I recently got excited about OpenMP[0], which seems to let one take advantage of threads while handling some of the drudgery of setup in common patterns. [0] https://en.wikipedia.org/wiki/OpenMP https://en.wikipedia.org/wiki/OpenMP Edit: add link
turminal 6y ago
Every threading tutorial should come with a warning. This one doesn't, so I'm adding one here. https://shouldiusethreads.com https://shouldiusethreads.com

165 points by dvaun 6y ago

intc 6y ago

Nice share - Very clear and informative. Wrote some time ago a simple threaded implementation of Sieve of Eratosthenes utilizing POSIX threads (in C): https://github.com/intc/c-notes/blob/master/threads/4-thrd_eratosthenes/erat.c https://github.com/intc/c-notes/blob/master/threads/4-thrd_e...

zerof1l 6y ago

Pthreads will always have special place in my heart. It was my first serious introduction to the parallelism concepts.

senderista 6y ago
Ditto, via David Butenhof's excellent book _Programming with Posix Threads_. I didn't know what a mutex or condition variable was before I read that book.
- gautamdivgi 6y ago
  I still have that book on my shelf.
- johnisgood 6y ago
  Woah, it is that old! Been some time. Now I feel old.
- jjav 6y ago
  Lives on my prime shelf space, along with TAOCP and the Stevens books.
cylde_frog 6y ago
I was going to comment the exact same thing. I have fond memories of learning about parallelism in university and using pThreads. It was such a mind-blowing and novel thing at the time when all I had been exposed to was sequential single threaded programming.

rramadass 6y ago

Good succinct tutorial. For people interested in studying Concurrency (Threading is just a subset of the model) i highly suggest the following book; Foundations of Multithreaded, Parallel, and Distributed Programming by Gregory Andrews. On the low-level OS/Hardware side (i.e. SMP/Caches etc.) UNIX Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers by Curt Schimmel is excellent.

Daho0n 6y ago
Thank you for those. Do you happen to know a good one on C#? Trying to learn more about the Windows side.
- dauchande 6y ago
  C# should use Async and Await. Steven Cleary's book is the standard for C#: Concurrency in C# Cookbook: Asynchronous, Parallel, and Multithreaded Programming For Windows, I liked: Concurrent Programming on Windows by Duffy

37ef_ced3 6y ago

Here is an example: NN-512 uses pthreads. It generates a tiny, standalone, highly scalable C99 work-stealing thread pool in every C file. The work items are simply coordinates, similar to the threadIdx/blockIdx coordinates in Nvidia's CUDA. For example, all the structs and functions starting with "Example32Threader" in this file: https://nn-512.com/example/32#3 https://nn-512.com/example/32#3 The implementation ends at Example32ThreaderDo1 The key is to avoid writing to shared cache lines whenever possible, and to keep a bitmap in the central struct (the "hub") that avoids rediscovery of the fact that a thread has no remaining work to be stolen (thereby avoiding a "thundering herd" type problem: https://en.m.wikipedia.org/wiki/Thundering_herd_problem https://en.m.wikipedia.org/wiki/Thundering_herd_problem) Another key is to never take any short-cuts in synchronization. Rather than doing something subtle/clever to avoid mutexes (etc.), instead make sure the units of work are big enough so that synchronization costs are negligible with respect to the overall computation

gary_0 6y ago
> Rather than doing something subtle/clever to avoid mutexes (etc.), instead make sure the units of work are big enough so that synchronization costs are negligible with respect to the overall computation This seems to be working well in my current project, even though I originally planned to refactor things to be "cleverer". A momentary Big Stupid Mutex lock followed by a good chunk of independent work is a nicely boring design for worker threads that seems to work fine. Another trick I previously used was just leaving work involving contended resources aside until after all the worker threads finish, instead of trying to resolve contention with waits or more complex code. If the leftovers from each thread are <1% of the total work, it can just be cleaned up single-threaded style later. Sometimes it's enough to just detect potential contention and leave the trickier work to simpler code.

screwgoth 6y ago

Wow !! This is the same page from where I started learning about Pthreads way back in 2004. 16 freaking years ago. I loved it then, love it now.

pantulis 6y ago

This is a great resource, and it's also well written. I did a lot of C thread programming 22 years ago (plus some CORBA), and this would have been very helpful, basically resumes a whole semester of university classes in one web page! Lazy question: are there any details of how pthreads are implemented for each operating system?

MisterTea 6y ago

Concurrency is easy if the library is well thought out and clean. So I much prefer the Plan 9 approach using the CSP thread(2) library: http://man.postnix.pw/9front/2/thread http://man.postnix.pw/9front/2/thread, Example program: https://github.com/mischief/9problems/blob/master/sys/src/libthread/example.c https://github.com/mischief/9problems/blob/master/sys/src/li... Wonderful easy to use library which is built on top of rfork and uses channels and alternates to synchronize the procs. Heap memory can be shared across processes (e.g. malloc'd memory or a segment(3)) so passing data around is just passing pointers to objects in shared memory. Further reading which demonstrates the library and how it influenced Go can be found here: https://seh.dev/go-legacy/ https://seh.dev/go-legacy/

albmoriconi 6y ago

This wonderful tutorial is the same I used to learn Pthreads around 10 years ago. The training section of the LLNL website [1] offers a lot of solid tutorials; I used the OpenMP and MPI ones for my Parallel Computing class some years ago and they helped me to sistematically learn (and learn to apply) the material. I love them! [1] https://hpc.llnl.gov/training/tutorials https://hpc.llnl.gov/training/tutorials

marmight 6y ago

For folks interested in threads in C, I also recommend reading "Threads Cannot be Implemented as a Library" [1]. The summary is that Pthreads mostly works correctly as a library (1) because its functions execute memory fence instructions and (2) because its functions are treated as opaque functions by the C compiler, i.e., functions that might read or write from any global variable. However, these properties do not generate thread-safe code under a number of conditions such as under the presence of certain compiler optimizations. Thus, the paper argues that the compiler must be aware of threads and that threads cannot be implemented purely as a library. [1] https://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf https://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf

jcranmer 6y ago
The language basis for threading is now the C11/C++11 memory model [1], even if you don't directly use it--it's baked into the compiler IR. I would argue that in the 2020s, you should be using C11/C++11 threading instead of direct pthreads if you're writing new code. (And yes, Hans Boehm was one of the people instrumental in getting the threaded memory model adopted.) [1] If you really want to be pedantic, it's the current verbiage that rules, independent of the actual language version you request. This is one of those situations where trying to describe the semantics more formally is challenging, so a lot of the changes in the specification aren't supposed to actually be changing how things are compiled but how things are described in the text.
- wahern 6y ago
  In some ways the POSIX API is better. For example, the POSIX API permits you to statically define and initialize pthread_mutex_t, pthread_once_t, and similar objects. Without static initialization you're stuck in a catch-22 because to initialize such objects at runtime you have to rely on some other implicitly serialized control flow, such as by invoking initializers from main(), which is particularly cumbersome and error prone for libraries. C++ has it a little easier because, AFAIU, the language itself supports static constructors--i.e. the ability to run [mostly] ad hoc code to initialize static objects at runtime (specifically load time). The C11/C++11 threading API lacks static initializers because when the specification was being developed Windows light-weight threading primitives didn't support such a capability. Basically, the C11/C++11 API had to take a lowest common denominator approach, which in most cases meant dropping useful features from the POSIX API so Microsoft didn't have to reimplement their primitives. (The hope was that C11- and C++11-compliant implementations would become widely available immediately if the standard made it easy to wrap existing OS primitives, but unfortunately implementations were--and arguably still are, especially for C--many years late.) Also, the memory model is mostly distinct from the threading APIs. The memory model was more important for nailing down atomics in C11 and C++11. The threading APIs are much higher level, and because they rely on opaque types and function calls, their implementation and the necessary compiler support (ISA memory barriers, compiler code motion barriers, etc) was largely transparent.

bch 6y ago

I recently got excited about OpenMP[0], which seems to let one take advantage of threads while handling some of the drudgery of setup in common patterns. [0] https://en.wikipedia.org/wiki/OpenMP https://en.wikipedia.org/wiki/OpenMP Edit: add link

turminal 6y ago

Every threading tutorial should come with a warning. This one doesn't, so I'm adding one here. https://shouldiusethreads.com https://shouldiusethreads.com