10 ms·
How I turned Zig into my favorite language to write network programs in
- supportengineer 11mo agoMove Zig, for great justice.
- echelon 11mo agoOne of the very first internet memes. The zig team should adopt it as the slogan. https://en.wikipedia.org/wiki/All_your_base_are_belong_to_us https://en.wikipedia.org/wiki/All_your_base_are_belong_to_us
- dgb23 11mo agoIt has in a way! See: https://github.com/ziglang/zig/blob/master/lib/init/src/main.zig#L6 https://github.com/ziglang/zig/blob/master/lib/init/src/main...
- sionisrecur 11mo agoThere's also https://github.com/allyourcodebase/ https://github.com/allyourcodebase/
- breatheoften 11mo agoWhat makes a NATS client implementation the right prototype from which to extract a generic async framework layer? This looks interesting but I'm not familiar with NATS
- maxbond 11mo agoIf you succeed in creating a generic async primitive, it doesn't really matter what the original task was (as long as it's something that requires async), no? That's an implication of it being generic?
- lukaslalinsky 11mo agoThe layer was not extracted from the NATS client, the NATS client was just a source of frustration that prompted this creation.
- dxxvi 11mo agoDo you know that there's a concurrent Scala library named ZIO (https://zio.dev https://zio.dev)? :-)
- quantummagic 11mo agoIsn't this a bad time to be embracing Zig? It's currently going through an intrusive upheaval of its I/O model. My impression is that it was going to take a few years for things to shake out. Is that wrong?
- geysersam 11mo agoWhat's a few years? They go by in the blink of an eye. Zig is a perfectly usable language. People who want to use it will, those who don't won't.
- tonyhart7 11mo agoonly for hobby project
- nesarkvechnep 11mo agoYou or in general? Because, you know, this is like, your opinion, man.
- tonyhart7 11mo agoMy Opinion??? how about you goes to Zig github and check how progress of the language it literally there and its still beta test and not fit for production let alone have mature ecosystem
- dns_snek 11mo agoYes, your opinion. I run it in production and everything I've built with it has been rock solid (aside from my own bugs). I haven't touched a few of my projects in a few years and they work fine, but if I wanted to update them to the latest version of Zig I'd have a bit of work ahead of me. That's it.
- tonyhart7 11mo ago
- mananaysiempre 11mo ago> Context switching is virtually free, comparable to a function call. If you’re counting that low, then you need to count carefully. A coroutine switch, however well implemented, inevitably breaks the branch predictor’s idea of your return stack, but the effect of mispredicted returns will be smeared over the target coroutine’s execution rather than concentrated at the point of the switch. (Similar issues exist with e.g. measuring the effect of blowing the cache on a CPU migration.) I’m actually not sure if Zig’s async design even uses hardware call/return pairs when a (monomorphized-as-)async function calls another one, or if every return just gets translated to an indirect jump. (This option affords what I think is a cleaner design for coroutines with compact frames, but it is much less friendly to the CPU.) So a foolproof benchmark would require one to compare the total execution time of a (compute-bound) program that constantly switches between (say) two tasks to that of an equivalent program that not only does not switch but (given what little I know about Zig’s “colorless” async) does not run under an async executor(?) at all. Those tasks would also need to yield on a non-trivial call stack each time. Seems quite tricky all in all.
- messe 11mo ago> I’m actually not sure if Zig’s async design even uses hardware call/return pairs Zig no longer has async in the language (and hasn't for quite some time). The OP implemented task switching in user-space.
- loeg 11mo agoEven so. You're talking about storing and loading at least ~16 8-byte registers, including the instruction pointer which is essentially a jump. Even to L1 that takes some time; more than a simple function call (jump + pushed return address).
- lukaslalinsky 11mo agoOnly stack and instruction pointer are explicitly restored. The rest is handled by the compiler, instead of depending on the C calling convention, it can avoid having things in registers during yield. See this for more details on how stackful coroutines can be made much faster: https://photonlibos.github.io/blog/stackful-coroutine-made-fast https://photonlibos.github.io/blog/stackful-coroutine-made-f...
- mrasong 11mo agoThe first time I heard about Zig was actually on Bun’s website, it’s been getting better and better lately.
- tombert 11mo agoI really need to play with Zig. I got really into Rust a few months ago, and I was actually extremely impressed by Tokio, so if this library also gives me Go-style concurrency without having to rely on a garbage collector, then I am likely to enjoy it.
- lukaslalinsky 11mo agoGo has tricks that you can't replicate elsewhere, things like infinitely growable stacks, that's only possible thanks to the garbage collector. But I did enjoy working on this, I'm continually impressed with Zig for how nice high-level looking APIs are possible in such a low-level language.
- aidenn0 11mo agoPre-1.0 Rust used to have infinitely growing stacks, but they abandoned it due to (among other things) performance reasons (IIRC the stacks were not collected with Rust's GC[1], but rather on return; the deepest function calls may happen in tight loops, and if you are allocating and freeing the stack in a tight loop, oops!) 1: Yes, pre-1.0 Rust had a garbage collector.
- RustSupremacist 11mo agoRust still has garbage collection if you use Arc and Rc. Not a garbage collector but this type of garbage collection.
- echelon 11mo agoYou mean Drop, which is entirely predictable and controlled by the user?
- aidenn0 11mo agoI'm going to veer into no-true-scottsman territory for a bit and claim that those don't count since they cannot collect cycles (if I'm wrong and they implement e.g. trial-deletion, let me know). This isn't just academic, since cyclic data-structures are an important place where the borrow-checker can't help you, so a GC would be useful.
- otobrglez 11mo agoThere is an extremely popular library/framework for Scala named ZIO out there,… Naming is hard.
- aidenn0 11mo agoI am still mystified as to why callback-based async seems to have become the standard. What this and e.g. libtask[1] do seems so much cleaner to me. The Rust folks adopted async with callbacks, and they were essentially starting from scratch so had no need to do it that way, and they are smarter than I (both individually and collectively) so I'm sure they have a reason; I just don't know what it is. 1: https://swtch.com/libtask/ https://swtch.com/libtask/
- loeg 11mo agoThe thread stack for something like libtask is ambiguously sized and often really large relative to like, formalized async state.
- vlovich123 11mo agoThe research Microsoft engineers did on stackful vs stackless coroutines for the c++ standard I think swayed this as “the way” to implement it for something targeting a systems level - significantly less memory overhead (you only pay for what you use) and offload the implementation details of the executor (lots of different design choices that can be made).
- zozbot234 11mo agoYup, stackful fibers are an anti-pattern. Here's Gor Nishanov's review for the C++ ISO committee https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13... linked from https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=102989 https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10... . Notice how it sums things up: > DO NOT USE FIBERS!
- sriku 11mo agoThe article says it was created to write audio software but I'm unable to find any first sources for that. Pointers?
- lukaslalinsky 11mo agoSee the first example in Andrew's introduction: https://andrewkelley.me/post/intro-to-zig.html https://andrewkelley.me/post/intro-to-zig.html
- noselasd 11mo agoMostly out of curiosity, a read on a TCP connection could easily block for a month - how does the I/O timeout interface look like ? e.g. if you want to send an application level heartbeat when a read has blocked for 30 seconds.
- lukaslalinsky 11mo agoI don't have a good answer for that yet, mostly because TCP reads are expected to be done through std.Io.Reader which isn't aware of timeouts. What I envision is something like `asyncio.timeout` in Python, where you start a timeout and let the code run as usual. If it's in I/O sleep when the timeout fires, it will get woken up and the operation gets canceled. I see something like this: var timeout: zio.Timeout = .init; defer timeout.cancel(rt); timeout.set(rt, 10); const n = try reader.interface.readVec(&data);
- sgt 11mo agoAre you working using Zig master with the new Io interface passed around, by the way?
- lukaslalinsky 11mo agoNo, I'm targeting Zig 0.15. The new Io interface is not in master yet, it's still evolving. When it's merged to master and stable, I'll start implementing the vtable. But I'm just passing Runtime around, instead of Io. So you can easily migrate code from zio to std when it's released.
- secondcoming 11mo agoThis is very true. Most examples of async io I've seen - regardless of the framework - gloss over timeouts and cancellation. It's really the hardest part. Reading and writing asynchronously from a socket, or whatever, is the straightforward part.
- dgb23 11mo agoYou can set read and write timeouts on TCP sockets: https://linux.die.net/man/3/setsockopt https://linux.die.net/man/3/setsockopt Zig has a posix API layer.
- pjmlp 11mo agoZio already exists, https://zio.dev/ https://zio.dev/
- deleted 11mo ago[deleted]
- cat-whisperer 11mo agoStackful coroutines make sense when you have the RAM for it. I've been using Zig for embedded (ARM Cortex-M4, 256KB RAM) mainly for memory safety with C interop. The explicitness around calling conventions catches ABI mismatches at compile-time instead of runtime crashes. I actually prefer colored async (like Rust) over this approach. The "illusion of synchronous code" feels magical, but magic becomes a gotcha in larger codebases when you can't tell what's blocking and what isn't.
- vrnvu 11mo ago> when you can't tell what's blocking and what isn't. Isn't that exactly why they're making IO explicit in functions? So you can trace it up the call chain.
- audunw 11mo agoThe new Zig IO will essentially be colored, but in a nicer way than Rust. You don't have to color your function based on whether you're supposed to use in in an async or sync manner. But it will essentially be colored based on whether it does I/O or not (the function takes IO interface as argument). Which is actually important information to "color" a function with. Whether you're doing async or sync I/O will be colored at the place where you call an IO function. Which IMO is the correct way to do it. If you call with "async" it's nonblocking, if you call without it, it's blocking. Very explicit, but not in a way that forces you to write a blocking and async version of all IO functions. The Zio readme says it will be an implementation of Zig IO interface when it's released. I guess you can then choose if you want explicit async (use Zig stdlib IO functions) or implicit async (Zio), and I suppose you can mix them. > Stackful coroutines make sense when you have the RAM for it. So I've been thinking a bit about this. Why should stackful coroutines require more RAM? Partly because when you set up the coroutine you don't know how big the stack needs to be, right? So you need to use a safe upper bound. While stackless will only set up the memory you need to yield the coroutine. But Zig has a goal of having a built-in to calculate the required stack size for calling a function. Something it should be able to do (when you don't have recursion and don't call external C code), since Zig compiles everything in one compilation unit. Zig devs are working on stackless coroutines as well. But I wonder if some of the benefits goes away if you can allocate exactly the amount of stack a stackful coroutine needs to run and nothing more.
- d3ckard 11mo agoHonestly, have been excited about Zig for quite a while, dabbled a bit a while back and was waiting for it getting closer to 1.0 to actually do a deep dive... but that moment doesn't seem to come. I don't mind, it's up to the maintainers on how they want to proceed. However, I would greatly appreciate if Zig news was a bit clearer on what's happening, timelines etc. I think it takes relatively little time to do so, but optics would be so much better.
- RustSupremacist 11mo ago> In the previous C++ version, I used Qt, which might seem very strange for a server software, but I wanted a nice way of doing asynchronous I/O and Qt allowed me to do that. It was callback-based, but Qt has a lot of support for making callbacks usable. In the newer prototypes, I used Go, specifically for the ease of networking and concurrency. With Zig, I was stuck. There are new Qt bindings for these. Go has https://github.com/mappu/miqt https://github.com/mappu/miqt and Zig has https://github.com/rcalixte/libqt6zig https://github.com/rcalixte/libqt6zig. I wonder if the author knew about them. I don't know enough about either language to speak on the async parts. For me, I want these for Rust, especially what Zig has because I use KDE. I know about https://github.com/KDAB/cxx-qt https://github.com/KDAB/cxx-qt and it is the only maintained effort for Rust that is left standing after all these years. But I don't want QML. I definitely don't want C++ or CMake. I just want Rust and Cargo.
- 5- 11mo agoperhaps it's a trivial observation that people tend to conflate the programming language in the strict sense (syntax, semantics, compiler implementation etc.) with its standard and/or community libraries and tooling. of course these are very important, but perhaps i'm just a language nerd/pedant who gets confused when an article about a programming language tends to be about async i/o libraries.