4 ms·

Libcsp: a fast C concurrency library influenced by the CSP model

138 points by lime66 7y ago

grayed-down 7y ago
Hmm...nice examples. Let's give this a whirl. Thanks!
- grayed-down 7y ago
  Ha! Only on HN will a golem gray you down for wanting to give a guy's library a test. And I'm getting ready to do a PR in the next few days to help him out!
metroholografix 7y ago
There is also http://libdill.org/ http://libdill.org/ which has a different, better in my view, model and I have actually used in production.
- latenightcoding 7y ago
  I dont think libdill supports multiple CPUs, which is really hard to get right
  - armitron 7y ago
    Well it does not allow coroutines to migrate to different OS threads and handles to be shared across OS threads, but that does not mean it doesn't support multiple CPUs. It's more correct to say that it doesn't come with that support out of the box, but it certainly does allow for it and nudges you towards a specific direction (no shared mutable state).
    - throwaway17_17 7y ago
      That has been my experience using the library, I have found that it really encourages, by means of making it the most expedient option to have write once data structures. My use case is certainly not the common one though, so others may have different experiences.
  - ioquatix 7y ago
    You run multiple instances, one per CPU core.
    - latenightcoding 7y ago
      there is a huge difference between running an instance per CPU and letting a smart scheduler orchestrate things for you.
      - ignoramous 7y ago
        Reminds me of this epic discussion between KirinDave, Ryan Dahl, and Aphyr: https://news.ycombinator.com/item?id=4306241 https://news.ycombinator.com/item?id=4306241
      - zekrioca 7y ago
        Parent my have a StackOverFlow CV.. :)
giancarlostoro 7y ago
The side by side with Go is actually impressive: https://libcsp.com/ https://libcsp.com/
- pickdenis 7y ago
  I suspect it stops being so nice when you want to do something equivalent to go's `select` statement. Though that's purely speculation, and if I'm wrong it should definitely go in the examples!
  - dallbee 7y ago
    Your skepticism made me look through the library more carefully. I came across some limitations in comparison to what Go provides: 1. As you suggest, there's no equivalent to the `select` mechanism. To perform the equivalent, you must busy loop and call `csp_chan_try_pop`. 2. It appears you cannot nest coroutines. 3. The mutex implementation does not provide an RWMutex. 4. `csp_yield` seems somewhat necessary for practical applications. Normally I would not use another language as a basis for comparison to a C library, but that's precisely what the author of Libcsp appears to be aiming for. All that said, there's a really slick implementation of a lock free RB queue backing this library, and It's a neat project. I think it would stand on its own better than as a "go style in C" library.
    - alexchamberlain 7y ago
      RB queue?
      - erling 7y ago
        ring buffer queue [0] [0] https://github.com/shiyanhui/libcsp/blob/48782baf0e63cbf2914930c67adbb884604fc3b7/src/rbq.h#L33 https://github.com/shiyanhui/libcsp/blob/48782baf0e63cbf2914...
jagged-chisel 7y ago
The linked Wikipedia article[1] on CSP says > communicating sequential processes (CSP) is a formal language for describing patterns of interaction in concurrent systems The Libcsp project says it’s influenced by CSP. Does libcsp stick to this formal language? 1 - https://en.wikipedia.org/wiki/Communicating_sequential_processes https://en.wikipedia.org/wiki/Communicating_sequential_proce...
- wnoise 7y ago
  Of course not. It implements much (not all!) of the available semantics of CSP, but does so embedded in the C language with function calls and macros. Though macros can do truly disturbing things in C, these are not enough to truly implement the syntax of CSP.
deleted 7y ago
[deleted]
marios 7y ago
Does anyone have resources related to learning concurrency ? I've had a very short introduction to pthreads in a CS class, but it was too succinct to get any practical benefit IMO. Knowing that a mutex can be used to limit access to a shared variable is easy enough to understand. How to structure software that runs concurrent tasks is still a mystery to me. Code I've written that uses threads appears to work but I'm not confident I haven't introduced a deadlock in there that's just waiting the worst possible timing to trigger. For instance, the pthreads manpage does a good job at listing the available functions. I have no idea when they should be used though. If anyone can point me to learning resources around concurrency (books, presentations, talks, sample open source software that does good use of concurrency patterns, ... anything goes, really) I will be eternally grateful.
- percentcer 7y ago
  I found C++ Concurrency in Action to be pretty helpful. Also if you're able to run TSan it'll really shine a light on your code and will help you sleep more easily. (https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual https://github.com/google/sanitizers/wiki/ThreadSanitizerCpp..., currently built into clang I believe)
- macintux 7y ago
  This book give a broad overview of the problem space: https://pragprog.com/book/pb7con/seven-concurrency-models-in-seven-weeks https://pragprog.com/book/pb7con/seven-concurrency-models-in... Also because I have it, here’s a distributed systems meta-reading-list. It’s old and thus I’m sure there are some broken links, but it’s a deep rabbit hole if you’re interested in distributed systems, which is basically concurrency on steroids. https://gist.github.com/macintux/6227368 https://gist.github.com/macintux/6227368
- trevyn 7y ago
  Try learning Rust? It pretty much forces you to think about safe concurrency. https://doc.rust-lang.org/book/ch16-00-concurrency.html https://doc.rust-lang.org/book/ch16-00-concurrency.html
wbl 7y ago
People should check out the Plan 9 libthread if they are interested http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread
jwr 7y ago
I wish I had a minimal version of something like this for microcontrollers (ARM Cortex M0+ and M4). Implementing event-driven code with interrupts is always a pain, even with neatly structured state machines.
- snops 7y ago
  You can use Protothreads[1] for this which work on anything that runs C, as they are just using setjmp/longjmp. I would recommend FreeRTOS instead, as it has proper prempetive scheduling, context switching and wide debugger support. It's actually only 4 files[2] so not very heavyweight in either memory or stuff to learn, and has decent abstractions for cross thread communication (queues etc). Then you can write continuous code and use preemption, rather than have to code everything as a state machine and co-operatively schedule, which is painful sometimes. [1] http://dunkels.com/adam/pt/ http://dunkels.com/adam/pt/ [2] https://www.freertos.org/Creating-a-new-FreeRTOS-project.html https://www.freertos.org/Creating-a-new-FreeRTOS-project.htm...

138 points by lime66 7y ago

grayed-down 7y ago

Hmm...nice examples. Let's give this a whirl. Thanks!

grayed-down 7y ago
Ha! Only on HN will a golem gray you down for wanting to give a guy's library a test. And I'm getting ready to do a PR in the next few days to help him out!

metroholografix 7y ago

There is also http://libdill.org/ http://libdill.org/ which has a different, better in my view, model and I have actually used in production.

latenightcoding 7y ago
I dont think libdill supports multiple CPUs, which is really hard to get right
- armitron 7y ago
  Well it does not allow coroutines to migrate to different OS threads and handles to be shared across OS threads, but that does not mean it doesn't support multiple CPUs. It's more correct to say that it doesn't come with that support out of the box, but it certainly does allow for it and nudges you towards a specific direction (no shared mutable state).
  - throwaway17_17 7y ago
    That has been my experience using the library, I have found that it really encourages, by means of making it the most expedient option to have write once data structures. My use case is certainly not the common one though, so others may have different experiences.
- ioquatix 7y ago
  You run multiple instances, one per CPU core.
  - latenightcoding 7y ago
    there is a huge difference between running an instance per CPU and letting a smart scheduler orchestrate things for you.
    - ignoramous 7y ago
      Reminds me of this epic discussion between KirinDave, Ryan Dahl, and Aphyr: https://news.ycombinator.com/item?id=4306241 https://news.ycombinator.com/item?id=4306241
    - zekrioca 7y ago
      Parent my have a StackOverFlow CV.. :)

giancarlostoro 7y ago

The side by side with Go is actually impressive: https://libcsp.com/ https://libcsp.com/

pickdenis 7y ago
I suspect it stops being so nice when you want to do something equivalent to go's `select` statement. Though that's purely speculation, and if I'm wrong it should definitely go in the examples!
- dallbee 7y ago
  Your skepticism made me look through the library more carefully. I came across some limitations in comparison to what Go provides: 1. As you suggest, there's no equivalent to the `select` mechanism. To perform the equivalent, you must busy loop and call `csp_chan_try_pop`. 2. It appears you cannot nest coroutines. 3. The mutex implementation does not provide an RWMutex. 4. `csp_yield` seems somewhat necessary for practical applications. Normally I would not use another language as a basis for comparison to a C library, but that's precisely what the author of Libcsp appears to be aiming for. All that said, there's a really slick implementation of a lock free RB queue backing this library, and It's a neat project. I think it would stand on its own better than as a "go style in C" library.
  - alexchamberlain 7y ago
    RB queue?
    - erling 7y ago
      ring buffer queue [0] [0] https://github.com/shiyanhui/libcsp/blob/48782baf0e63cbf2914930c67adbb884604fc3b7/src/rbq.h#L33 https://github.com/shiyanhui/libcsp/blob/48782baf0e63cbf2914...

jagged-chisel 7y ago

The linked Wikipedia article[1] on CSP says > communicating sequential processes (CSP) is a formal language for describing patterns of interaction in concurrent systems The Libcsp project says it’s influenced by CSP. Does libcsp stick to this formal language? 1 - https://en.wikipedia.org/wiki/Communicating_sequential_processes https://en.wikipedia.org/wiki/Communicating_sequential_proce...

wnoise 7y ago
Of course not. It implements much (not all!) of the available semantics of CSP, but does so embedded in the C language with function calls and macros. Though macros can do truly disturbing things in C, these are not enough to truly implement the syntax of CSP.

deleted 7y ago

[deleted]

marios 7y ago

Does anyone have resources related to learning concurrency ? I've had a very short introduction to pthreads in a CS class, but it was too succinct to get any practical benefit IMO. Knowing that a mutex can be used to limit access to a shared variable is easy enough to understand. How to structure software that runs concurrent tasks is still a mystery to me. Code I've written that uses threads appears to work but I'm not confident I haven't introduced a deadlock in there that's just waiting the worst possible timing to trigger. For instance, the pthreads manpage does a good job at listing the available functions. I have no idea when they should be used though. If anyone can point me to learning resources around concurrency (books, presentations, talks, sample open source software that does good use of concurrency patterns, ... anything goes, really) I will be eternally grateful.

percentcer 7y ago
I found C++ Concurrency in Action to be pretty helpful. Also if you're able to run TSan it'll really shine a light on your code and will help you sleep more easily. (https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual https://github.com/google/sanitizers/wiki/ThreadSanitizerCpp..., currently built into clang I believe)
macintux 7y ago
This book give a broad overview of the problem space: https://pragprog.com/book/pb7con/seven-concurrency-models-in-seven-weeks https://pragprog.com/book/pb7con/seven-concurrency-models-in... Also because I have it, here’s a distributed systems meta-reading-list. It’s old and thus I’m sure there are some broken links, but it’s a deep rabbit hole if you’re interested in distributed systems, which is basically concurrency on steroids. https://gist.github.com/macintux/6227368 https://gist.github.com/macintux/6227368
trevyn 7y ago
Try learning Rust? It pretty much forces you to think about safe concurrency. https://doc.rust-lang.org/book/ch16-00-concurrency.html https://doc.rust-lang.org/book/ch16-00-concurrency.html

wbl 7y ago

People should check out the Plan 9 libthread if they are interested http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread

jwr 7y ago

I wish I had a minimal version of something like this for microcontrollers (ARM Cortex M0+ and M4). Implementing event-driven code with interrupts is always a pain, even with neatly structured state machines.

snops 7y ago
You can use Protothreads[1] for this which work on anything that runs C, as they are just using setjmp/longjmp. I would recommend FreeRTOS instead, as it has proper prempetive scheduling, context switching and wide debugger support. It's actually only 4 files[2] so not very heavyweight in either memory or stuff to learn, and has decent abstractions for cross thread communication (queues etc). Then you can write continuous code and use preemption, rather than have to code everything as a state machine and co-operatively schedule, which is painful sometimes. [1] http://dunkels.com/adam/pt/ http://dunkels.com/adam/pt/ [2] https://www.freertos.org/Creating-a-new-FreeRTOS-project.html https://www.freertos.org/Creating-a-new-FreeRTOS-project.htm...