9 ms·
Libdill: Structured Concurrency for C (2016)
- jchw 9y ago>There is currently no support for Windows. Cygwin is very broken. It doesn't support AF_UNIX properly, and so no further development will be done for this platform. Boy do I have good news for whoever wrote this: https://blogs.msdn.microsoft.com/commandline/2017/12/19/af_unix-comes-to-windows/ https://blogs.msdn.microsoft.com/commandline/2017/12/19/af_u...
- TimMurnaghan 9y agoDoesn't sound great if a coroutine library has resorted to passing messages across sockets. I was expecting something that implemented multiple stacks and some abstraction of some synchronization.
- pritambaral 9y agoLooking at the source, AF_UNIX is used only for IPC and in DNS support, which is sensible.
- pjc50 9y agoIt appears to do that: https://github.com/sustrik/libdill/blob/master/stack.c https://github.com/sustrik/libdill/blob/master/stack.c
- rumcajz 9y agoCorrect. It does that. The IPC, on the other hand, is there for you to use if you want to talk to other processes on the same machine.
- scadge 9y agoI read that NodeJS is written in C with the same concept in its core - asynchronous on one core (thread) with interruptions. How does it compare to libdill?
- fleetfox 9y agoNode runs on top of libuv. https://nikhilm.github.io/uvbook/basics.html#hello-world https://nikhilm.github.io/uvbook/basics.html#hello-world
- scadge 9y agoSo it looks like libuv could use libdill, right?
- beagle3 9y agoCould have, possibly, but makes no sense at all to retrofit libuv with libdill at this point
- piscisaureus 9y agoShameless(-ish) plug: https://github.com/piscisaureus/wepoll https://github.com/piscisaureus/wepoll is not a fat abstraction layer but it could help with windows support.
- beagle3 9y agoLibuv supports windows and in some ways only exists because windows support is needed; on Unix/Linux where Node started, libev was sufficient; libuv was created to facilitate the windows port of node - though it is now standing in its own right.
- piscisaureus 9y agoI know - I am one of the authors of libuv. However libuv is pretty big and imposes its own asynchronous i/o model onto your application. Later I discovered that it is possible to do efficient epoll emulation on windows so then I wrote wepoll. With it you can just stick to the good ol' epoll/kqueue model and still support windows.
- 9y ago
- JepZ 9y ago> coroutine void worker(const char *text) { At first I read 'goroutine' and thought something is wrong with my mind and that I have coded too much Go. But then I saw this: > go(worker("Hello!")); Now I am pretty sure someone got inspired by Golang ;-) Nice to see some backporting of Go features to C. Note: That doesn't mean that Go is the only language that supports easy concurrent patterns, but that the creators of Go wanted to improve C and while not everybody agrees that all they have done are improvements, making concurrency easier most certainly is one.
- scadge 9y agoI think it's fair to say that Go was hardly the first language to implement the concept of coroutines[1]. Probably the only merit of Go here is the short and convenient name of the 'run' method. [1] https://en.wikipedia.org/wiki/Coroutine#Implementations https://en.wikipedia.org/wiki/Coroutine#Implementations
- teamhappy 9y ago> libmill was a project that aimed to copy Go's > concurrency model to C 1:1 [...] > libdill is a follow-up project that experiments with > structured concurrency and diverges from the Go model. http://libdill.org/documentation.html http://libdill.org/documentation.html
- rumcajz 9y agoI would say the main advantage is being able to pass couroutine arguments as if it was a simple function. In other coroutine implementations in C (e.g. libtask) you have to make a struct containing all the args and pass that to the run function.
- lucozade 9y agolibmill, the pre-cursor to libdill, was directly inspired by an implementation of nanomsg written in go (called mango I believe). Martin had struggled with the state machine approach he had been using and the coroutine approach of golang was a lot easier to reason about in practice. libdill was a development on libmill in two main ways: it was more idiomatically C and it supports structured concurrency. The latter is, in my opinion, very interesting. Martin has a blog post on it [0]. BTW I've got all this from following the nanomsg threads and reading Martin's blog. I'm not an active participant, so I may have some details wrong, but I think it's fairly accurate. [0] http://250bpm.com/blog:71 http://250bpm.com/blog:71
- majke 9y agoOk, I'll chime in here since all the comments are missing the point. If you are excited by golang-style in C, then Mr Sustrik's libmill is for you http://libmill.org/tutorial.html http://libmill.org/tutorial.html LibDill is a thin experiment "above" of libmill. The main problem it tries to solve is "cancellations". For the uninitiated, in normal programming languages there is no way to cancel/interrupt a tread/coroutine from outside. Look at the mess in pthread_cancel(3) [1]. In golang as well, there is no way to cancel a coroutine. Why this is important? Well, read Mr Sustrik's notes [2]. But basically - imagine you want to define the time limit for a completion of some task. Or - imagine what should happen if a HTTP request comes, initiates some actions (SQL quieries?) and then disconnects. Normal programming languages have NO way of expressing that - "client went away, stop processing its sql queries". Libdill is an attempt to solve this. I must admit, I'm not fully on board with the "structured concurrency" train of thought [3] , but the whole prospect of defining semantics of killing/cancelling coroutines is absolutely amazing. Also, review the golang "context" mess, which AFAIU tries to work around the same problem. [4] [1] http://man7.org/linux/man-pages/man3/pthread_cancel.3.html http://man7.org/linux/man-pages/man3/pthread_cancel.3.html [2] http://250bpm.com/blog:71 http://250bpm.com/blog:71 [3] http://libdill.org/structured-concurrency.html http://libdill.org/structured-concurrency.html [4] https://golang.org/pkg/context https://golang.org/pkg/context
- rumcajz 9y agoAuthor here. Thanks for the nice summary. You've expressed it better then I would. I am not 100% sure about the structured concurrency concept myself, but comparing it to structured programming feels reassuring. Basically, a tree structure (whether a syntactic tree or the tree of running coroutines) is probably the most complex way to structuring stuff that's still tractable by a human being. Once you go beyond that, you'll eventually get lost. Thus, tools enforcing a tree-like structure are likely to be helpful to keep the program maintainable.
- igravious 9y agoI like the idea of structured tree-like concurrency, fits my brains. About your API though. Why is there a load of inet/ip/socket functions, shouldn't they be in a separate lib? Why is the opposite of the fn int go(expr) the fn int hclose(int) and not the fn int stop(int) or something like it. open/close – go/stop – and so on …
- MobiusHorizons 9y agoVery nice!! I look forward to reading through all the structured concurrency stuff later.
- kaeluka 9y agoI was confused -- how can a library add a keyword like "coroutine" to C. It didn't look like macros were powerful enough to do that (I expected to at least some context having to be passed to the worker, or so). It turns out that a coroutine is just a function that the compiler should not inline: https://github.com/sustrik/libmill/blob/master/libmill.h#L242 https://github.com/sustrik/libmill/blob/master/libmill.h#L24...
- rumcajz 9y agoYes, it's just a normal function. What go() macro does is that it switches the stack, then lets compiler-generated code put the args of the function at the top of the new stack. Obviously, if the function was inlined, the above would not work.
- luckydude 9y agoI really liked the Linda concept for concurrency but it doesn't seem to have caught on in general. It isn't the same as what is being presented here but it side steps the threads issues, the multiple cpu issues, all that falls out for free. Seemed pretty cool and I suppose you could layer on cancellations (but I suspect it would be a mess like pthread_cancel()). https://en.wikipedia.org/wiki/Linda_(coordination_language) https://en.wikipedia.org/wiki/Linda_(coordination_language) http://linuxtuples.sourceforge.net/ http://linuxtuples.sourceforge.net/ Anyone else like that stuff? Or have any insight why it isn't more widely used? Maybe it's because it was a commercial thing?
- luckydude 9y agoThe fact that this is all on one cpu seems like a problem, does it not? Does someone have a mental model of how to take advantage of multiple cpus without creating a mess? If so, can you explain it like I'm 5 (or more like I'm 55 and tired)?
- Kenji 9y agoYes. Either use threadsafe queues/ringbuffers to transfer data between threads. Or triple-buffer it and swap buffers in a threadsafe manner. Those are the best approaches. Don't start a thread per work item. Start worker threads and distribute work among them. Never ever block a GUI thread. Always know which thread works on which data structures and don't even make more variables accessible in your worker threads. Keep the variables multiple threads can access to an absolute minimum and make sure access is always protected by a mutex. If you repect these rules, threading is safe and fun.
- rumcajz 9y agoI think the environment is a bit different now than it was once. You typically have a cluster of machines, so you have to be able to run multiple instances of your application simultaneously to use all the processing power. And once you can run mutliple instances of the application in parallel you can as well run 16 instances of it on a 16-core box. The upside is that you don't have two levels of granularity (threads, machines) but only a single one (processes) which makes both the coding and the ops as well as stuff like capacity planning much easier.
- aidenn0 9y agoEven absent the cluster of machines I found this to be very useful. Several years ago, I would use zeromq (thank you, btw) for communication and if the socket overhead was found to be an issue, I would merge two processes and switch from tcp:// to inproc://. Many "corrupt the world" type bugs are isolated from each other by processes, and it made it very hard to accidentally share state (the file system of course is a giant hunk of share state, but you very rarely accidentally write to a file).
- 9y ago
- snarfy 9y agoI took a peek at the source briefly. There is a lot more going on than I was expecting and was pleasantly surprised. I was expecting to find a simple wrapper around setjmp/longjmp, but this is much more. Kudos.
- stefantalpalaru 9y agoOr you could use the gccgo runtime from C: https://github.com/stefantalpalaru/golib https://github.com/stefantalpalaru/golib
- ausjke 9y agoso what is this different from pthread? how can I decide to pick one of them? the new C11 has thread built-in but not used widely yet.