9 ms·
How I turned Zig into my favorite language to write network programs in
- supportengineer 1y agoMove Zig, for great justice.
- echelon 1y agoOne of the very first internet memes. The zig team should adopt it as the slogan. https://en.wikipedia.org/wiki/All_your_base_are_belong_to_us https://en.wikipedia.org/wiki/All_your_base_are_belong_to_us
- dgb23 1y agoIt has in a way! See: https://github.com/ziglang/zig/blob/master/lib/init/src/main.zig#L6 https://github.com/ziglang/zig/blob/master/lib/init/src/main...
- sionisrecur 1y agoThere's also https://github.com/allyourcodebase/ https://github.com/allyourcodebase/
- breatheoften 1y agoWhat makes a NATS client implementation the right prototype from which to extract a generic async framework layer? This looks interesting but I'm not familiar with NATS
- maxbond 1y agoIf you succeed in creating a generic async primitive, it doesn't really matter what the original task was (as long as it's something that requires async), no? That's an implication of it being generic?
- lukaslalinsky 1y agoThe layer was not extracted from the NATS client, the NATS client was just a source of frustration that prompted this creation.
- dxxvi 1y agoDo you know that there's a concurrent Scala library named ZIO (https://zio.dev https://zio.dev)? :-)
- quantummagic 1y agoIsn't this a bad time to be embracing Zig? It's currently going through an intrusive upheaval of its I/O model. My impression is that it was going to take a few years for things to shake out. Is that wrong?
- geysersam 1y agoWhat's a few years? They go by in the blink of an eye. Zig is a perfectly usable language. People who want to use it will, those who don't won't.
- tonyhart7 1y agoonly for hobby project
- nesarkvechnep 1y agoYou or in general? Because, you know, this is like, your opinion, man.
- tonyhart7 1y agoMy Opinion??? how about you goes to Zig github and check how progress of the language it literally there and its still beta test and not fit for production let alone have mature ecosystem
- dns_snek 1y agoYes, your opinion. I run it in production and everything I've built with it has been rock solid (aside from my own bugs). I haven't touched a few of my projects in a few years and they work fine, but if I wanted to update them to the latest version of Zig I'd have a bit of work ahead of me. That's it.
- tonyhart7 1y ago
- mananaysiempre 1y ago> Context switching is virtually free, comparable to a function call. If you’re counting that low, then you need to count carefully. A coroutine switch, however well implemented, inevitably breaks the branch predictor’s idea of your return stack, but the effect of mispredicted returns will be smeared over the target coroutine’s execution rather than concentrated at the point of the switch. (Similar issues exist with e.g. measuring the effect of blowing the cache on a CPU migration.) I’m actually not sure if Zig’s async design even uses hardware call/return pairs when a (monomorphized-as-)async function calls another one, or if every return just gets translated to an indirect jump. (This option affords what I think is a cleaner design for coroutines with compact frames, but it is much less friendly to the CPU.) So a foolproof benchmark would require one to compare the total execution time of a (compute-bound) program that constantly switches between (say) two tasks to that of an equivalent program that not only does not switch but (given what little I know about Zig’s “colorless” async) does not run under an async executor(?) at all. Those tasks would also need to yield on a non-trivial call stack each time. Seems quite tricky all in all.
- messe 1y ago> I’m actually not sure if Zig’s async design even uses hardware call/return pairs Zig no longer has async in the language (and hasn't for quite some time). The OP implemented task switching in user-space.
- loeg 1y agoEven so. You're talking about storing and loading at least ~16 8-byte registers, including the instruction pointer which is essentially a jump. Even to L1 that takes some time; more than a simple function call (jump + pushed return address).
- lukaslalinsky 1y agoOnly stack and instruction pointer are explicitly restored. The rest is handled by the compiler, instead of depending on the C calling convention, it can avoid having things in registers during yield. See this for more details on how stackful coroutines can be made much faster: https://photonlibos.github.io/blog/stackful-coroutine-made-fast https://photonlibos.github.io/blog/stackful-coroutine-made-f...
- mrasong 1y agoThe first time I heard about Zig was actually on Bun’s website, it’s been getting better and better lately.
- tombert 1y agoI really need to play with Zig. I got really into Rust a few months ago, and I was actually extremely impressed by Tokio, so if this library also gives me Go-style concurrency without having to rely on a garbage collector, then I am likely to enjoy it.
- lukaslalinsky 1y agoGo has tricks that you can't replicate elsewhere, things like infinitely growable stacks, that's only possible thanks to the garbage collector. But I did enjoy working on this, I'm continually impressed with Zig for how nice high-level looking APIs are possible in such a low-level language.
- aidenn0 1y agoPre-1.0 Rust used to have infinitely growing stacks, but they abandoned it due to (among other things) performance reasons (IIRC the stacks were not collected with Rust's GC[1], but rather on return; the deepest function calls may happen in tight loops, and if you are allocating and freeing the stack in a tight loop, oops!) 1: Yes, pre-1.0 Rust had a garbage collector.
- RustSupremacist 1y agoRust still has garbage collection if you use Arc and Rc. Not a garbage collector but this type of garbage collection.
- echelon 1y agoYou mean Drop, which is entirely predictable and controlled by the user?
- aidenn0 1y agoI'm going to veer into no-true-scottsman territory for a bit and claim that those don't count since they cannot collect cycles (if I'm wrong and they implement e.g. trial-deletion, let me know). This isn't just academic, since cyclic data-structures are an important place where the borrow-checker can't help you, so a GC would be useful.
- otobrglez 1y agoThere is an extremely popular library/framework for Scala named ZIO out there,… Naming is hard.
- aidenn0 1y agoI am still mystified as to why callback-based async seems to have become the standard. What this and e.g. libtask[1] do seems so much cleaner to me. The Rust folks adopted async with callbacks, and they were essentially starting from scratch so had no need to do it that way, and they are smarter than I (both individually and collectively) so I'm sure they have a reason; I just don't know what it is. 1: https://swtch.com/libtask/ https://swtch.com/libtask/
- loeg 1y agoThe thread stack for something like libtask is ambiguously sized and often really large relative to like, formalized async state.
- vlovich123 1y agoThe research Microsoft engineers did on stackful vs stackless coroutines for the c++ standard I think swayed this as “the way” to implement it for something targeting a systems level - significantly less memory overhead (you only pay for what you use) and offload the implementation details of the executor (lots of different design choices that can be made).
- zozbot234 1y agoYup, stackful fibers are an anti-pattern. Here's Gor Nishanov's review for the C++ ISO committee https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13... linked from https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=102989 https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10... . Notice how it sums things up: > DO NOT USE FIBERS!
- sriku 1y agoThe article says it was created to write audio software but I'm unable to find any first sources for that. Pointers?
- lukaslalinsky 1y agoSee the first example in Andrew's introduction: https://andrewkelley.me/post/intro-to-zig.html https://andrewkelley.me/post/intro-to-zig.html
- noselasd 1y agoMostly out of curiosity, a read on a TCP connection could easily block for a month - how does the I/O timeout interface look like ? e.g. if you want to send an application level heartbeat when a read has blocked for 30 seconds.
- lukaslalinsky 1y agoI don't have a good answer for that yet, mostly because TCP reads are expected to be done through std.Io.Reader which isn't aware of timeouts. What I envision is something like `asyncio.timeout` in Python, where you start a timeout and let the code run as usual. If it's in I/O sleep when the timeout fires, it will get woken up and the operation gets canceled. I see something like this: var timeout: zio.Timeout = .init; defer timeout.cancel(rt); timeout.set(rt, 10); const n = try reader.interface.readVec(&data);
- sgt 1y agoAre you working using Zig master with the new Io interface passed around, by the way?
- lukaslalinsky 1y agoNo, I'm targeting Zig 0.15. The new Io interface is not in master yet, it's still evolving. When it's merged to master and stable, I'll start implementing the vtable. But I'm just passing Runtime around, instead of Io. So you can easily migrate code from zio to std when it's released.
- secondcoming 1y agoThis is very true. Most examples of async io I've seen - regardless of the framework - gloss over timeouts and cancellation. It's really the hardest part. Reading and writing asynchronously from a socket, or whatever, is the straightforward part.
- dgb23 1y agoYou can set read and write timeouts on TCP sockets: https://linux.die.net/man/3/setsockopt https://linux.die.net/man/3/setsockopt Zig has a posix API layer.
- pjmlp 1y agoZio already exists, https://zio.dev/ https://zio.dev/
- deleted 1y ago[deleted]
- cat-whisperer 1y agoStackful coroutines make sense when you have the RAM for it. I've been using Zig for embedded (ARM Cortex-M4, 256KB RAM) mainly for memory safety with C interop. The explicitness around calling conventions catches ABI mismatches at compile-time instead of runtime crashes. I actually prefer colored async (like Rust) over this approach. The "illusion of synchronous code" feels magical, but magic becomes a gotcha in larger codebases when you can't tell what's blocking and what isn't.
- vrnvu 1y ago> when you can't tell what's blocking and what isn't. Isn't that exactly why they're making IO explicit in functions? So you can trace it up the call chain.
- audunw 1y agoThe new Zig IO will essentially be colored, but in a nicer way than Rust. You don't have to color your function based on whether you're supposed to use in in an async or sync manner. But it will essentially be colored based on whether it does I/O or not (the function takes IO interface as argument). Which is actually important information to "color" a function with. Whether you're doing async or sync I/O will be colored at the place where you call an IO function. Which IMO is the correct way to do it. If you call with "async" it's nonblocking, if you call without it, it's blocking. Very explicit, but not in a way that forces you to write a blocking and async version of all IO functions. The Zio readme says it will be an implementation of Zig IO interface when it's released. I guess you can then choose if you want explicit async (use Zig stdlib IO functions) or implicit async (Zio), and I suppose you can mix them. > Stackful coroutines make sense when you have the RAM for it. So I've been thinking a bit about this. Why should stackful coroutines require more RAM? Partly because when you set up the coroutine you don't know how big the stack needs to be, right? So you need to use a safe upper bound. While stackless will only set up the memory you need to yield the coroutine. But Zig has a goal of having a built-in to calculate the required stack size for calling a function. Something it should be able to do (when you don't have recursion and don't call external C code), since Zig compiles everything in one compilation unit. Zig devs are working on stackless coroutines as well. But I wonder if some of the benefits goes away if you can allocate exactly the amount of stack a stackful coroutine needs to run and nothing more.
- d3ckard 1y agoHonestly, have been excited about Zig for quite a while, dabbled a bit a while back and was waiting for it getting closer to 1.0 to actually do a deep dive... but that moment doesn't seem to come. I don't mind, it's up to the maintainers on how they want to proceed. However, I would greatly appreciate if Zig news was a bit clearer on what's happening, timelines etc. I think it takes relatively little time to do so, but optics would be so much better.
- RustSupremacist 1y ago> In the previous C++ version, I used Qt, which might seem very strange for a server software, but I wanted a nice way of doing asynchronous I/O and Qt allowed me to do that. It was callback-based, but Qt has a lot of support for making callbacks usable. In the newer prototypes, I used Go, specifically for the ease of networking and concurrency. With Zig, I was stuck. There are new Qt bindings for these. Go has https://github.com/mappu/miqt https://github.com/mappu/miqt and Zig has https://github.com/rcalixte/libqt6zig https://github.com/rcalixte/libqt6zig. I wonder if the author knew about them. I don't know enough about either language to speak on the async parts. For me, I want these for Rust, especially what Zig has because I use KDE. I know about https://github.com/KDAB/cxx-qt https://github.com/KDAB/cxx-qt and it is the only maintained effort for Rust that is left standing after all these years. But I don't want QML. I definitely don't want C++ or CMake. I just want Rust and Cargo.
- 5- 1y agoperhaps it's a trivial observation that people tend to conflate the programming language in the strict sense (syntax, semantics, compiler implementation etc.) with its standard and/or community libraries and tooling. of course these are very important, but perhaps i'm just a language nerd/pedant who gets confused when an article about a programming language tends to be about async i/o libraries.