10 ms·
Show HN: A pure C89 implementation of Go channels, with blocking selects
- hollimolli 3y agoThanks for this. It would actually be nice to have a plain old Makefile. The .pro file doesn't look like anything I know.
- Rochus 3y agoWelcome; I don't use make; the pro is a qmake file; I likely will add a BUSY file, but until then you can just use e.g. "cc -std=c89 *.c -lpthread".
- sim7c00 3y agonot being a god level programmer i wont go into quality of code, but this looks to me really neat and easy to use. well done!
- eqvinox 3y agoIt's 2023 - even MSVC has supported C11 and C17 for a while now. C89 is no longer a feature to advertise, it's an unhelpful constraint forcing poorer quality code. A good example is that CspChan_closed should use the bool type added in C99. There's nonstandard nomenclature in "_dispose"; if the constructing function is called _create, the common pattern is to pair it with _destroy. typedefs ending with _t are reserved for standards extensions, though these days people ignore this a lot because it feels like general practice to add _t. There's no way to pass in a custom allocator, or logging callback, or set some debug logging flags. Maybe something to tackle later. Both _select functions fall short in that they do not allow select'ing on channels + other file descriptors simultaneously. General library design practice is to have the library return a file descriptor that can be used in whatever event loop the application already uses. (The fd can be a dummy pipe or eventfd.) All of these things are in the header, so they matter the most as they set ABI and API for any user. Unfortunately getting them right as early as possible is important to avoid breaks; yet very hard to do since often API aspects only become clear after a library has non-trivial users. [edited to tone down a bit]
- 1vuio0pswjnm7 3y agoPerhaps I'm readint it wrong bu this comment is crticising someone for not using added language features. Maybe the author does not need them. "YAGNI". IMO, it should be the author's choice whether or not to utilise certain features. As someone else noted in another comment, C89 is more portable than C11 or C17. To me, C89 is like Bourne shell. An author can choose to write scripts in Bash but they will not be as portable as ones written in Dash, nor as fast. The only constraint I have run into with C89, when compiling lex.yy.c, is that it has a limit of 32767 lines.
- coumbaya 3y agoNot dismissing anything you said but bool isn't C89 right ? Back ~6 years ago when I worked in embedded C89, bool was just a #define true (1==1), #define false (0==1) so I guess it makes sense it isn't in the lib ?
- eqvinox 3y agoIndeed, bool (or _Bool) was added in C99, hence my pointing out it not being used… it's not in C89. It matters because the return value sense is different; an "int" return for a status-ish thing in a modern library generally means 0 for success, nonzero for error codes. "bool" is the other way around, 0 for failure-y. In this case it's a status retrieval function so it's pretty clear that it's intended as a boolean but it'd still be better to actually make it bool.
- coumbaya 3y agoAh, I get it now, thx.
- dang 3y agoI realize your intention is to provide good feedback and you obviously know a lot, but if you could shift the pH slightly away from acidic, that would be better. Keep in mind that criticisms like this have a tendency to land 10x harder than you intended. https://news.ycombinator.com/showhn.html https://news.ycombinator.com/showhn.html
- vore 3y agowhile (!CspChan_closed(chan)) { ... } seems like a concurrency footgun – what's stopping the channel from being closed in between the check for it being closed and the operation on the channel?
- withinboredom 3y agoAnd further, what if there are messages still in the closed channel? Do they just go "poof"? It's fine if they do, but that should be documented.
- codys 3y agoNothing stops that. You'll end up getting a message full of 0 bytes (messages are fixed size) if it closes between the `closed()` check and the `receive()` operation. Common designs in this space instead indicate if the receive call has obtained data (return values typically indicate this). This is a kind of time-of-check/time-of-use bug. Go, for its part, has its receive operation, `<-`, return a status indicating if the channel is still open, as in `v, ok := <-ch`
- vore 3y agoI don't think CspChan_closed is a super useful operation if using it is always prone to TOCTTOU issues – maybe as an optimization hint but that's about it. In Go, the receive operation indicating the channel is closed is atomic with receiving a value, so it's not possible to have a TOCTTOU problem there.
- rockwotj 3y agoRelated is libmill, which has been around for awhile and is was previously discussed on HN: https://news.ycombinator.com/item?id=30699829 https://news.ycombinator.com/item?id=30699829 libmill supports a bunch of stuff like sockets, timers, files, etc.
- adastra22 3y agoDomains are expired. Do you have other links?
- pjmlp 3y agoBasically what ended up replacing Alef in Plan 9. https://9p.io/magic/man2html/2/thread https://9p.io/magic/man2html/2/thread
- Rochus 3y agoWell, both use channels, as did Occam, Joyce or Newsqueak even before. But my library uses kernel threads (Pthreads, Win32 threads), no coroutines, nor user mode scheduling. So there is little in common with Alef or Plan 9. But there are indeed C libraries which do that; mine has other goals.
- bufo 3y agoGreat! I was looking into something like this. I assume ending up with epoll will be better?
- eqvinox 3y agoConsidering that epoll is Linux specific anyway, I would highly advise going straight to io_uring. epoll has a whole bunch of footguns in particular with edge triggered modes of operation; io_uring has a higher initial threshold in understanding how it works but is worth that effort. (Unless you need to support older Linux kernels that have epoll but no io_uring yet.)
- bufo 3y agoOh yeah I meant io_uring too. Plus Windows copied it so you can implement things very similarly for Windows.
- another2another 3y agoThanks for that, had no idea. I've used IO Completion ports on Windows before, and found it to be pretty useful API but I guess this is also a useful addition. Basically no good idea goes uncopied by Microsoft :) https://learn.microsoft.com/en-us/windows/win32/api/ioringapi/ https://learn.microsoft.com/en-us/windows/win32/api/ioringap... >Minimum supported client Windows Build 22000 Alas I have no idea from that what the lowest version of Windows that supports this API is though.
- bufo 3y agoBuild 22000 is Windows 10 21H2.
- adastra22 3y agoIs there something similar on macOS/iOS?
- 3y ago
- mmcgaha 3y agoWhen it said pure C89 I figured it was going to have some setjmp and longjmp going on instead of threads.
- samsquire 3y agoWow, thank you for this. Good work! Some thoughts: I've often thought that unbuffered channels would cause scheduling thrashing - higher latency and lower throughput because you're swapping between stopping and starting processes blocked on a channel frequently. If you're sending just a small piece of data like 64 bit integer at a time, or any kind of pattern where you're using threads to break up work into tasks, this is too small breakdown of task to really scale multithreading. Want to communicate something that causes a LARGE AMOUNT of work on the other thread, to keep the processor busy. But there's balance between latency and throughput, if you send a big task you get higher latency to react to the next task but better throughput. Walking through my thinking and help me understand: If you have multiple channels that you could read from in a select call but none are ready, could you block that select instance and process a different process where other selects are potentially waiting? This is similar to blocking a goroutine or a Rust async Future task in Tokio that needs to be waked. I think this would need a scheduler. EDIT: Your scheduler to switch between "select instances" or what is running in a thread is the OS.
- Galanwe 3y ago> If you have multiple channels that you could read from in a select call but none are ready, could you block that select instance and process a different process where other selects are potentially waiting? From my experience, the most versatile approach for the wait on channels is best implemented as a spin lock with exponential backoff, up to a threshold that yields - effectively letting other threads perform their own select. Note that most of the time, if you are targeting latency, you would try not to get more workers than cores, meaning you would pin each worker to a core, thus not really benefiting from yielding on select. > But there's balance between latency and throughput, if you send a big task you get higher latency to react to the next task but better throughput. Agree, but that's mainly a client concern, the queue can propose multiple APIs. Most queues handle that with a "burst" mode, where you pop or push multiple messages at once instead of 1 by 1, to balance throughput vs latency.
- jpc0 3y ago> a spin lock with exponential backoff Also measure before you implement this yourself, to my knowledge this is infact what the linux mutex does anyway.
- galkk 3y agoEven looking at simple code example, with casting from/to void, returning 0 as void etc, I cannot understand how people are calling C easy and productive language
- vngzs 3y agoThat claim isn't made on the linked site, though. I think people who write C nowadays tend to be working in the drivers and OS development spaces.
- trealira 3y ago> I think people who write C nowadays tend to be working in the drivers and OS development spaces. I think it's also students; I think some people are still taught C as their first programming language. If you go on r/C_programming on Reddit, a lot of it is filled by very basic questions from what seem like beginners; I suspect they're students trying to do their homework.
- kristianp 3y agoPerhaps not the first language. In my experience C is often in the second batch of languages after the introductory one (java back in my day).
- 0xbadcafebee 3y agoThere's at least 4 kinds of C programmers today: - People who work on projects which require low-level access, potentially mixing in assembly code (drivers, OS, etc), porting to different architectures, real-time, high-precision, control of timing, lack of MMU, etc - People who want to perform specific actions, like a specific OS syscall, that may not be exposed [easily] by a higher level language in the way the programmer wants - People who want to write very fast/efficient code - Legacy code maintainers Not too long ago I busted out some C to create a stripped binary for a job interview round. The interviewee needed to diagnose what the binary was doing and why and report back. Not all that common for most programmers today, but I expect a good amount of detail from a senior-level systems candidate.
- p_l 3y agofun fact - Go Channels started out as pretty thin syntax sugar over features provided by Plan9's C libthread.
- Zambyte 3y agoThis is really cool! I do want to say though, if you're considering using this for a production system: it's worth also considering ZeroMQ inproc sockets[0]. They allow for very similar semantics to this, with the added benefit of trivially being able to migrate to an inter-process / network channel, by just changing the URL to bind / connect to. [0] http://czmq.zeromq.org/ http://czmq.zeromq.org/
- smabie 3y agoDepending on your use case, zmq inproc I've found has a lot more latency than you would expect. like p50 of >100us and p99 can be over 1ms. Kind of unfortunate as other more handrolled solutions (depending on use case) can easily be <10us. Feels like you really need to be using a lock free ring buffer that only supports 1<>1 communication if you really want to go fast. (p50 of <1us with very compressed tail latencies)
- Matthias247 3y agolibmill (https://github.com/sustrik/libmill https://github.com/sustrik/libmill) and libdill (https://github.com/sustrik/libdill https://github.com/sustrik/libdill) should be similar and probably mentioned. As far as I understand the differences between CspChan and libmill might be that libmill also implements lightweight tasks (coroutines) and everything that goes with it (IO multiplexing, async timers, etc), while CspChan uses OS threads?
- samuell 3y agoWas going to ask if the current implementation could be used together with libmill, but maybe they overlap a lot in functionality?
- Matthias247 3y agoI don't think you can. The library would block the libill worker thread, which would also block all other libmill coroutines from running. The libmill channels need to yield in a way where they don't block the underlying threads.
- segmondy 3y agoif you don't get anything out of this and don't know Hoare's work, go read CSP - http://www.usingcsp.com/cspbook.pdf http://www.usingcsp.com/cspbook.pdf
- KingLancelot 3y ago[dead]
- openasocket 3y agoVery interesting! I program a lot in Go, and while I have my complaints about the language, I find the CSP model to be very easy to work with. I'd recommend adding some additional documentation about the API. You mention trying to keep dynamic allocations to a minimum (which I like to hear) but it would be handy to have documentation stating which functions allocate and how much they allocate. Actually, more documentation in general would be nice, particular relating to edge cases. In Go I know that sending to a closed channel will panic, for example. But what will your library do? Return some sort of error message? Silently drop the message? Segfault? Definitely something you want prominently documented. Especially anything that could potentially segfault should be highlighted. Oh, and some benchmarks would be very interesting!
- xyproto 3y agoWhat are your complaints about the Go language that is better in C89?
- iainmerrick 3y agoSorry for going a bit off-topic, but something I'm curious about: are channels thought to be a good/useful tool for concurrent programming? My feeling is that they're probably too low-level and error-prone, and you really want higher-level structures like worker pools or actors. So as a concurrency building block they'd be on the same abstraction level as pthreads or Java monitors -- nice and flexible for building on top of, but too finicky to be used directly in application code. But I'm not a Go expert, and maybe Go programmers do successfully use channels directly for application code?
- JyB 3y agoThe opposite. Channels are often used to orchestrate higher-level structures/abstractions; not 'low-level' stuff where more common primitives such as mutexes/semaphores are sometime preferred.
- openasocket 3y agoI program in Go a lot professionally. I use channels a decent amount of the time, but often drop down to using mutexes or atomics. But I generally prefer channels to mutexes and atomics; generally I reach for those alternatives only due to performance concerns. One thing that struck me about your comment is referring to actors as a “higher level structure” compared to channels. Comparing Go channels with the actor model is more a matter of apples and oranges than one being more high level than the other. Go is built on the Communicating Sequential Processes (CSP) model: https://en.m.wikipedia.org/wiki/Communicating_sequential_processes https://en.m.wikipedia.org/wiki/Communicating_sequential_pro... . See the comparison to the actor model in that article if you are curious. The essential idea to CSP is that you have independent processes that only communicate with each other via these channels. This is in comparison to the actor mode where each process has its own dedicated “mailbox” that it can receive from (but can send messages to any other actor it wants). I think that wiki does a better job of outlining the differences than I can in a comment, but essentially these are two similar but subtly different ways to model parallelism. Personally, I find the CSP model a bit more intuitive than the actor model (though admittedly I have not worked with the actor model as extensively). For the basic idea of “make a process that receives input data, processes it, and submits output data somewhere else” CSP and the actor model are more or less identical. But for more complex use cases, like a process that can receive multiple kinds of messages from different places, or having multiple processes all handling the same stream of incoming data (aka load balancing) I find the CSP model more intuitive. This might be personal bias though. In particular I’ve never seen an actor model with static types and how that would handle different types of messages coming into the mailbox. Short answer: channels for concurrent programming is definitely a good tool.
- kazinator 3y ago1. There is no <memory.h> in any version of ISO C, nor in POSIX. 2. This is not C89; at best GNU C 89, because a variable is declared after a statement: { if( msgLen == 0 ) msgLen = 1; /* queueLen == 0 is an unbuffered channel, but we still need one slot to transport the message */ CspChan_t* c = (CspChan_t*)malloc(sizeof(CspChan_t) + queueLen*msgLen); To help enforce that you're actually writing C89, you should set your compiler to C89 and turn on whatever additional diagnostics may be required like -pedantic.
- Rochus 3y ago"Pure" doesn't apply to "language purity" (nobody besides a very few seem to really care about the latter, especially not in C, not even compiler authors), but that nothing else is required than a C89 compatible toolchain to use the library (i.e. it doesn't need any assembler or third-party tools/libraries). See also https://www.merriam-webster.com/dictionary/pure?utm_campaign=sd&utm_medium=serp&utm_source=jsonld https://www.merriam-webster.com/dictionary/pure?utm_campaign.... And so far the library compiles with all of my C89 copilers, even the very old ones (and old Posix versions). For a few cases to check, there are notes in the source code.
- deleted 3y ago[deleted]
- kazinator 3y ago> nothing else is required than a C89 compatible toolchain Fair enough. A "pure C89" program should be expected to work in Turbo C on MS-DOS.
- Rochus 3y agoNo C89 compatible toolchain does "pure C89" without extra options (if at all); people would run away. Even if you use GCC or CLANG with the -std=c89 option it doesn't. So the complaints that the presented library is not "pure C89" is nothing but an academic quibble. EDIT: I now have changed the repository description to "A pure C (-std=c89) implementation of Go channels, including blocking and non-blocking selects.", so hopefully this idiotic discussion will come to an end.
- flkenosad 3y agoBrilliant. Thank you so much!
- asylteltine 3y ago[flagged]
- solid_fuel 3y agoI love seeing projects like this, thanks for sharing. I've worked in constrained environments before where anything past C89 was trouble, it's a nice target for portability even if it takes more work.