41 ms·

Libmill: Go-Style Concurrency in C

233 points by Cieplak 7y ago

abbiya 7y ago
The out and in functions should be swapped to match the go example?
AceJohnny2 7y ago
There's was an episode of the This Week In Tech podcast from 2015 that interviewed LibMill's creator about its design concepts: https://twit.tv/shows/floss-weekly/episodes/358 https://twit.tv/shows/floss-weekly/episodes/358 And HN discussion from 4 years ago: https://news.ycombinator.com/item?id=10585505 https://news.ycombinator.com/item?id=10585505
jwr 7y ago
This is something that would be extremely useful on microcontrollers, where you do lots of event-driven programming. Any sane design ends up with a state machine (or a hierarchy of state machines) driven by events generated in interrupts. Coming from Clojure, I wished for something like core.async on microcontrollers for a long time — a way to convert most of my state machine into sequential code, with the complexity hidden, all while keeping everything in C (converted/generated during compile).
- microcolonel 7y ago
  > a way to convert most of my state machine into sequential code Isn't that the norm for state machines in C? while (cont) { switch (v) { case S_STOP: cont = 0; break; ... } } Or are you talking about inlining sequences of transitions into eachother? I've been avoiding core.async so I don't really know what it does that has to do with this.
  - jwr 7y ago
    With core.async (or go, for that matter), you can write code like: * do one thing * wait for the result of one thing (for example, an event from an interrupt) * take the result and feed it into another thing * wait for the result of another thing So the code looks sequential. It's easy to write and easy to understand. But the entire 'go' block returns immediately: the whole thing gets converted into a state machine (which you don't normally see). That way you don't have to manage the complexity of the state machine yourself. Note that this is not the same thing as threading. Also, it does not require threading at all, for example core.async works just as well in ClojureScript, even though Javascript has no threads.
- dpwm 7y ago
  You may be in for a treat if you haven't already come across protothreads [0]. They provide a form of stack-less, co-operative multi-threading and are aimed at microcontrollers. I came across them by chance, and have used them on an ESP8266 – it's really made everything feel much more clear and far less confusing. You do need to wrap your head around the trick, but that really shouldn't take long. You also need to put any state that you expect to be preserved between calls of the function into something more persistent – just like with state machines. The examples all use static variables, but I am happier using a struct pointer to maintain state. [0] http://dunkels.com/adam/pt/ http://dunkels.com/adam/pt/
- jupp0r 7y ago
  Have you considered C++ coroutines? Imho they struck a really nice tradeoff in making it a language feature and letting you swap out the underlying infrastructure.
  - creatornator 7y ago
    C++ coroutines, being a feature of C++17/C++20, are not present in most if not all micro-controller toolchains, so I'm not sure if it would be relevant for the parent
    - jupp0r 7y ago
      What processors are you referring to? Everything ARM is well supported by LLVM/Clang which implements the coroutines TS.
      - creatornator 7y ago
        AVR and PIC? Maybe things have improved since the last time I've explored compilers alternative to those supplied by manufacturers. But from what I understand avr-gcc is the standard for avr, and PIC is just targeted by the manufacturer's compilers.
        jupp0r 7y ago
        AVR support is pretty bad in LLVM, PIC32 is MIPS, which is well supported afaik. ARM market share is around 50% in embedded microcontrollers afaik [1], so it's definitely possible to use in embedded projects, but might be impossible for your specific platform. [1] https://www.electronicsweekly.com/news/business/arm-market-share-slipping-2018-05/ https://www.electronicsweekly.com/news/business/arm-market-s...
- Matthias247 7y ago
  If you are OK with stackful coroutines (that libmill provides) then an RTOS (like freertos) would mostly be the same thing. Context switching for those isn’t as expensive as for PC Threads and processes. If you can’t afford the memory for a dedicated callstack per thread then there are the other event driven models that had been mentioned here. However one loses out one real-time capabilities and priority handling by using those compared to a preemptive RTOS.
- rixed 7y ago
  Have you considered gnupth, a venerable, solid light-thread library ? I've never used it on a microcontroller and if you work from interrupts you might need to adapt its event loop, but that's still probably the first lib I would try, having successfully used it in large industrial projects in the past. Actually, there were no release since 2006, when the last bug was fixed I guess :+) https://www.gnu.org/software/pth/ https://www.gnu.org/software/pth/
coliveira 7y ago
Isn't this something already provided by libtask? Libtask was written by Russ Cox.
- Jtsummers 7y ago
  Based on a very cursory check, it seems libtask doesn't do channels. Libmill does, in the style of Go. So libmill seems to be more a superset of libtask.
  - p_l 7y ago
    Go channels et al are actually a syntax sugar for a C library that later got subsumed and rewritten in Go, but can still be used - libthread
  - aidenn0 7y ago
    libtask does have channels[1]. libdill (also by the creator of libmill) has some features that libtask does not. [edit] Also one should note that libtask predates Go, and so the API definitely feels very different. Libmill seems to specifically have a goal of making the usage feel "go-like" 1: https://swtch.com/libtask/channel.c https://swtch.com/libtask/channel.c
    - Jtsummers 7y ago
      I missed that, thanks. I did say it was a cursory check. I somehow overlooked it there at the top and bottom of the main libtask page.
    - nicklaf 7y ago
      It's amusing to note that the rewriting of a concurrent language as a C library has happened before in Bell Lab's history (by some of the same people who worked on Go and its precursors). Specifically, the same thing we're talking about here already happened with Alef[0], a CSP-based language, which was scrapped in favor of a C library[1] (man 2 thread) in the third edition of the Plan 9 OS[2]. [0] http://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ http://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ [1] http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread [2] http://doc.cat-v.org/plan_9/4th_edition/papers/release3 http://doc.cat-v.org/plan_9/4th_edition/papers/release3
kazinator 7y ago
The macro should be called select, since that's what Go calls it. (Of course I know there is a POSIX function (not C) by that name; so what). This "choose {" "end }" business is an awful way to design the macro. If you need nesting that involves a block, with prolog and epilog material on either or both ends, you just have to hide that into the macro: selbegin; out(ch, int, 42): foo(); in(ch, int, i): bar(i); otherwise: baz(); selend; I don't understand the downvotes. I have 30 years of (continous!) C experience. This is the best practice for doing a macro like this. This "end }" business has improper nesting, visually. The choose is outside of the braces, the end is within; it is scatter-brained. There is no error checking for a forgotten end. I'm looking at the implementation and it's looking quite weird! choose simply expands into mill_choose_init__, and end expands into mill_choose_end__. The definitions of these seem as if they should pair together: #define mill_choose_init__ \ {\ mill_choose_init_(MILL_HERE_);\ int mill_idx = -2;\ while(1) {\ if(mill_idx != -2) {\ if(0) #define mill_choose_end__ \ break;\ }\ }\ mill_idx = mill_choose_wait_();\ } But, oops, look at the obvious lack of balance in the braces. mill_choose_init__ { mill_choose_end__ } we are putting a brace after the if (0) which balances the one after break, and the final missing brace that balances the one opened internally in mill_choose_init__. There are best practices to design this sort of macrology that don't require the user put braces in weird places to balance the inconsistencies in macro expansions. Speaking as a professional, I would not pass this in a code review. That we have to use a flaky preprocessor in order to get a half decent syntax is already a significant regret. We must not compound the regret by allowing the macrology to be less than as good as it can be.
- frenchman99 7y ago
  As someone unfamiliar with this library or concurrency in C for that matter, I do not understand your comment. The code examples on libmill.org look simple enough that I'd want to use it. What is it that you think is not right about it ?
  - chrisseaton 7y ago
jjtheblunt 7y ago
Go is Go-style concurrency _almost_ in C.
oneplane 7y ago
How come neither libmill nor libdill sites have TLS. It's almost a chore to set up a site without it nowadays.
azhenley 7y ago
Is libmill safe with garbage collectors (like Boehm GC)?
gigatexal 7y ago
I actually really like the syntax for how they're emulating sending and receiving from channels. I like it better than Go's way of doing this with the <- operator. What would be nice would be a defer function for closing channels and things.
StreamBright 7y ago
What is Go style in this context? Isn’t it CSP?
snarfy 7y ago
It seems like development stopped a few years ago. There were steady releases almost monthly for a few years and then it just stops in 2017. Did the author lose interest? Are there any plans to support multicore? I noticed one of the examples addresses this by using fork(). Multicore is something goroutines > 1.5 support.
alexhutcheson 7y ago
How does this compare to boost:Fiber (other than the obvious "it's in C, not C++")?
yandrypozo 7y ago
who is using libmill by the way? like other open source projects using this library?

233 points by Cieplak 7y ago

The out and in functions should be swapped to match the go example?

There's was an episode of the This Week In Tech podcast from 2015 that interviewed LibMill's creator about its design concepts: https://twit.tv/shows/floss-weekly/episodes/358 https://twit.tv/shows/floss-weekly/episodes/358 And HN discussion from 4 years ago: https://news.ycombinator.com/item?id=10585505 https://news.ycombinator.com/item?id=10585505

jwr 7y ago

This is something that would be extremely useful on microcontrollers, where you do lots of event-driven programming. Any sane design ends up with a state machine (or a hierarchy of state machines) driven by events generated in interrupts. Coming from Clojure, I wished for something like core.async on microcontrollers for a long time — a way to convert most of my state machine into sequential code, with the complexity hidden, all while keeping everything in C (converted/generated during compile).

microcolonel 7y ago
> a way to convert most of my state machine into sequential code Isn't that the norm for state machines in C? while (cont) { switch (v) { case S_STOP: cont = 0; break; ... } } Or are you talking about inlining sequences of transitions into eachother? I've been avoiding core.async so I don't really know what it does that has to do with this.
- jwr 7y ago
  With core.async (or go, for that matter), you can write code like: * do one thing * wait for the result of one thing (for example, an event from an interrupt) * take the result and feed it into another thing * wait for the result of another thing So the code looks sequential. It's easy to write and easy to understand. But the entire 'go' block returns immediately: the whole thing gets converted into a state machine (which you don't normally see). That way you don't have to manage the complexity of the state machine yourself. Note that this is not the same thing as threading. Also, it does not require threading at all, for example core.async works just as well in ClojureScript, even though Javascript has no threads.
dpwm 7y ago
You may be in for a treat if you haven't already come across protothreads [0]. They provide a form of stack-less, co-operative multi-threading and are aimed at microcontrollers. I came across them by chance, and have used them on an ESP8266 – it's really made everything feel much more clear and far less confusing. You do need to wrap your head around the trick, but that really shouldn't take long. You also need to put any state that you expect to be preserved between calls of the function into something more persistent – just like with state machines. The examples all use static variables, but I am happier using a struct pointer to maintain state. [0] http://dunkels.com/adam/pt/ http://dunkels.com/adam/pt/
jupp0r 7y ago
Have you considered C++ coroutines? Imho they struck a really nice tradeoff in making it a language feature and letting you swap out the underlying infrastructure.
- creatornator 7y ago
  C++ coroutines, being a feature of C++17/C++20, are not present in most if not all micro-controller toolchains, so I'm not sure if it would be relevant for the parent
  - jupp0r 7y ago
    What processors are you referring to? Everything ARM is well supported by LLVM/Clang which implements the coroutines TS.
    - creatornator 7y ago
      AVR and PIC? Maybe things have improved since the last time I've explored compilers alternative to those supplied by manufacturers. But from what I understand avr-gcc is the standard for avr, and PIC is just targeted by the manufacturer's compilers.
      - jupp0r 7y ago
        AVR support is pretty bad in LLVM, PIC32 is MIPS, which is well supported afaik. ARM market share is around 50% in embedded microcontrollers afaik [1], so it's definitely possible to use in embedded projects, but might be impossible for your specific platform. [1] https://www.electronicsweekly.com/news/business/arm-market-share-slipping-2018-05/ https://www.electronicsweekly.com/news/business/arm-market-s...
Matthias247 7y ago
If you are OK with stackful coroutines (that libmill provides) then an RTOS (like freertos) would mostly be the same thing. Context switching for those isn’t as expensive as for PC Threads and processes. If you can’t afford the memory for a dedicated callstack per thread then there are the other event driven models that had been mentioned here. However one loses out one real-time capabilities and priority handling by using those compared to a preemptive RTOS.
rixed 7y ago
Have you considered gnupth, a venerable, solid light-thread library ? I've never used it on a microcontroller and if you work from interrupts you might need to adapt its event loop, but that's still probably the first lib I would try, having successfully used it in large industrial projects in the past. Actually, there were no release since 2006, when the last bug was fixed I guess :+) https://www.gnu.org/software/pth/ https://www.gnu.org/software/pth/

coliveira 7y ago

Isn't this something already provided by libtask? Libtask was written by Russ Cox.

Jtsummers 7y ago
Based on a very cursory check, it seems libtask doesn't do channels. Libmill does, in the style of Go. So libmill seems to be more a superset of libtask.
- p_l 7y ago
  Go channels et al are actually a syntax sugar for a C library that later got subsumed and rewritten in Go, but can still be used - libthread
- aidenn0 7y ago
  libtask does have channels[1]. libdill (also by the creator of libmill) has some features that libtask does not. [edit] Also one should note that libtask predates Go, and so the API definitely feels very different. Libmill seems to specifically have a goal of making the usage feel "go-like" 1: https://swtch.com/libtask/channel.c https://swtch.com/libtask/channel.c
  - Jtsummers 7y ago
    I missed that, thanks. I did say it was a cursory check. I somehow overlooked it there at the top and bottom of the main libtask page.
  - nicklaf 7y ago
    It's amusing to note that the rewriting of a concurrent language as a C library has happened before in Bell Lab's history (by some of the same people who worked on Go and its precursors). Specifically, the same thing we're talking about here already happened with Alef[0], a CSP-based language, which was scrapped in favor of a C library[1] (man 2 thread) in the third edition of the Plan 9 OS[2]. [0] http://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ http://doc.cat-v.org/plan_9/2nd_edition/papers/alef/ [1] http://man.cat-v.org/plan_9/2/thread http://man.cat-v.org/plan_9/2/thread [2] http://doc.cat-v.org/plan_9/4th_edition/papers/release3 http://doc.cat-v.org/plan_9/4th_edition/papers/release3

kazinator 7y ago

The macro should be called select, since that's what Go calls it. (Of course I know there is a POSIX function (not C) by that name; so what). This "choose {" "end }" business is an awful way to design the macro. If you need nesting that involves a block, with prolog and epilog material on either or both ends, you just have to hide that into the macro: selbegin; out(ch, int, 42): foo(); in(ch, int, i): bar(i); otherwise: baz(); selend; I don't understand the downvotes. I have 30 years of (continous!) C experience. This is the best practice for doing a macro like this. This "end }" business has improper nesting, visually. The choose is outside of the braces, the end is within; it is scatter-brained. There is no error checking for a forgotten end. I'm looking at the implementation and it's looking quite weird! choose simply expands into mill_choose_init__, and end expands into mill_choose_end__. The definitions of these seem as if they should pair together: #define mill_choose_init__ \ {\ mill_choose_init_(MILL_HERE_);\ int mill_idx = -2;\ while(1) {\ if(mill_idx != -2) {\ if(0) #define mill_choose_end__ \ break;\ }\ }\ mill_idx = mill_choose_wait_();\ } But, oops, look at the obvious lack of balance in the braces. mill_choose_init__ { mill_choose_end__ } we are putting a brace after the if (0) which balances the one after break, and the final missing brace that balances the one opened internally in mill_choose_init__. There are best practices to design this sort of macrology that don't require the user put braces in weird places to balance the inconsistencies in macro expansions. Speaking as a professional, I would not pass this in a code review. That we have to use a flaky preprocessor in order to get a half decent syntax is already a significant regret. We must not compound the regret by allowing the macrology to be less than as good as it can be.

frenchman99 7y ago
As someone unfamiliar with this library or concurrency in C for that matter, I do not understand your comment. The code examples on libmill.org look simple enough that I'd want to use it. What is it that you think is not right about it ?
- chrisseaton 7y ago

jjtheblunt 7y ago

Go is Go-style concurrency _almost_ in C.

oneplane 7y ago

How come neither libmill nor libdill sites have TLS. It's almost a chore to set up a site without it nowadays.

azhenley 7y ago

Is libmill safe with garbage collectors (like Boehm GC)?

gigatexal 7y ago

I actually really like the syntax for how they're emulating sending and receiving from channels. I like it better than Go's way of doing this with the <- operator. What would be nice would be a defer function for closing channels and things.

StreamBright 7y ago

What is Go style in this context? Isn’t it CSP?

snarfy 7y ago

It seems like development stopped a few years ago. There were steady releases almost monthly for a few years and then it just stops in 2017. Did the author lose interest? Are there any plans to support multicore? I noticed one of the examples addresses this by using fork(). Multicore is something goroutines > 1.5 support.

alexhutcheson 7y ago

How does this compare to boost:Fiber (other than the obvious "it's in C, not C++")?

yandrypozo 7y ago

who is using libmill by the way? like other open source projects using this library?