4 ms·

Having used zig a bit as a hobby. Why is it more ergonomic? Using await vs passing a token have similar ergonomics to me. The one thing you could say is that us

Having used zig a bit as a hobby. Why is it more ergonomic? Using await vs passing a token have similar ergonomics to me. The one thing you could say is that using some kind of token makes it dead simple to have different tokens. But that's really not something I run into often at all when using async.

messe 11mo ago
> The one thing you could say is that using some kind of token makes it dead simple to have different tokens. But that's really not something I run into often at all when using async. It's valuable to library authors who can now write code that's agnostic of the users' choice of runtime, while still being able to express that asynchronicity is possible for certain code paths.
- rowanG077 11mo ago
  But that can already be done using async await. If you write an async function in Rust for example you are free to call it with any async runtime you want.
  - messe 11mo ago
    But you can't call it from synchronous rust. Zig is moving toward all sync code also using the Io interface.
    - tcfhgj 11mo ago
      yes, you can: runtime.block_on(async { }) https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=060fc0424005ecf2d892e55e39988a03 https://play.rust-lang.org/?version=stable&mode=debug&editio...
      - whytevuhuni 11mo ago
        Here's a problem with that: Cannot start a runtime from within a runtime. This happens because a function (like `block_on`) attempted to block the current thread while the thread is being used to drive asynchronous tasks. https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=8abf9e00ef6039d9cd5f8fd7adc6bab0 https://play.rust-lang.org/?version=stable&mode=debug&editio...
        tcfhgj 11mo ago
        just pass around handles like you do in zig, alright? also: spawn_blocking for blocking code
        whytevuhuni 11mo ago
        But that's the thing, idiomatic Rust sync code almost never passes around handles, even when they need to do I/O. You might be different, and you might start doing that in your code, but almost none of either std or 3rd party libraries will cooperate with you. The difference with Zig is not in its capabilities, but rather in how the ecosystem around its stdlib is built. The equivalent in Rust would be if almost all I/O functions in std would be async; granted that would be far too expensive and disruptive given how async works.
        tcfhgj 11mo ago
        > But that's the thing, idiomatic Rust sync code almost never passes around handles, even when they need to do I/O. Because they don't use async inside. Zig code is passing around handles in code without io?
        whytevuhuni 11mo ago
        > Because they don't use async inside. But they use I/O inside, and we arrive at this issue: I'm writing async, and I need to call std::fs::read. I can't, because it blocks the thread; I could use spawn_blocking but that defeats the purpose of async. So instead I have to go look for a similar function but of the other color, probably from tokio. In Zig, if you're writing sync, you call the standard library function for reading files. If you're writing async, you call the same library function for reading files. Then, the creator of the `io` object decides whether the whole thing will be sync or async.
      - messe 11mo ago
        Let me rephrase, you can't call it like any other function. In Zig, a function that does IO can be called the same way whether or not it performs async operations or not. And if those async operations don't need concurrency (which Zig expresses separately to asynchronicity), then they'll run equally well on a sync Io runtime.
        tcfhgj 11mo ago
        > In Zig, a function that does IO can be called the same way whether or not it performs async operations or not. no, you can't, you need to pass a IO parameter
        messe 11mo ago
        You will need to pass that for synchronous IO as well. All IO in the standard library is moving to the Io interface. Sync and async. If I want to call a function that does asynchronous IO, I'll use: foo(io, ...); If I want to call one that does synchronous IO, I'll write: foo(io, ...); If I want to express that either one of the above can be run asynchronously if possible, I'll write: io.async(foo, .{ io, ... }); If I want to express that it must be run concurrently, then I'll write: try io.concurrent(foo, .{ io, ... }); Nowhere in the above do I distinguish whether or not foo does synchronous or asynchronous IO. I only mark that it does IO, by passing in a parameter of type std.Io.
        tcfhgj 11mo ago
        what about non-io code?
        messe 11mo ago
        What about it? It gets called without an Io parameter. Same way that a function that doesn't allocate doesn't get an allocator. I feel like you're trying to set me up for a gotcha "see, zig does color functions because it distinguishes functions that do io and those that don't!". And yes, that's true. Zig, at least Zig code using std, will mark functions that do Io with an Io parameter. But surely you can see how that will lead to less of a split in the ecosystem compared to sync and async rust?
hansvm 11mo ago
Making it dead simple to have different tokens is exactly the goal. A smattering of examples recently on my mind: As a background, you might ask why you need different runtimes ever. Why not just make everything async and be done with it, especially if the language is able to hide that complexity? 1. In the context of a systems language that's not an option. You might be writing an OS, embedded code, a game with atypical performance demands requiring more care with the IO, some kernel-bypass shenanigan, etc. Even just selecting between a few builtin choices (like single-threaded async vs multi-threaded async vs single-threaded sync) doesn't provide enough flexibility for the range of programs you're trying to allow a user to write. 2. Similarly, even initializing a truly arbitrary IO effect once at compile-time doesn't always suffice. Maybe you normally want a multi-threaded solution but need more care with respect to concurrency in some critical section and need to swap in a different IO. Maybe you normally get to interact with the normal internet but have a mode/section/interface/etc where you need to send messages through stranger networking conditions (20s ping, 99% packet loss, 0.1kbps upload on the far side, custom hardware, etc). Maybe some part of your application needs bounded latency and is fine dropping packets but some other part needs high throughput and no dropped packets at any latency cost. Maybe your disk hardware is such that it makes sense for networking to be async and disk to be sync. And so on. You can potentially work around that in a world with a single IO implementation if you can hack around it with different compilation units or something, but it gets complicated. Part of the answer then is that you need (or really want) something equivalent to different IO runtimes, hot-swappable for each function call. I gave some high-level ideas as to why that might be the case, but high-level observations often don't resonate, so let's look at a concrete case where `await` is less ergonomic: 1. Take something like TLS as an example (stdlib or 3rd-party, doesn't really matter). The handshake code is complicated, so a normal implementation calls into an IO abstraction layer and physically does reads and writes (as opposed to, e.g., a pure state-machine implementation which returns some metadata about which action to perform next -- I hacked together a terrible version of that at one point [0] if you want to see what I mean). What if you want to run it on an embedded device? If it were written with async it would likely have enough other baggage that it wouldn't fit or otherwise wouldn't work. What if you want to hide your transmission in other data to sneak it past prying eyes (steganography, nowadays that's relatively easy to do via LLMs interestingly enough, and you can embed arbitrary data in messages which are human-readable and purport to discuss completely other things without exposing hi/lo-bit patterns or other such things that normally break steganography)? Then the kernel socket abstraction doesn't work at all, and "just using await" doesn't fix the problem. Basically, any place you want to use that library (and, arguably, that's the sort of code where you should absolutely use a library rather than rolling it yourself), if the implementer had a "just use await" mentality then you're SOL if you need to use it in literally any other context. I was going to write more concrete cases, but this comment is getting to be too long. The general observation is that "just use await" hinders code re-use. If you're writing code for your own consumption and also never need those other uses then it's a non-issue, but with a clever choice of abstraction it _might_ be possible (old Zig had a solution that didn't quite hit the mark IMO, and time will tell if this one is good enough, but I'm optimistic) to enable the IO code people naturally write to be appropriately generic by default and thus empower future developers via a more composable set of primitives. They really nailed that with the allocator interface, and if this works then my only real concern is a generic "what next" -- it's pushing toward an effect system, but integrating those with a systems language is mostly an unsolved problem, and adding a 3rd, 4th, etc explicit parameter to nearly every function is going to get unwieldy in a hurry (back-of-the-envelope idea I've had stewing if I ever write a whole "major" language is to basically do what Zig currently does and pack all those "effects" into a single effect parameter that you pass into each function, still allowing you to customize each function call, still allowing you to inspect which functions require allocators or whatever, but making the experience more pleasant if you have a little syntactic sugar around sub-effects and if the parent type class is comptime-known). [0] https://github.com/hmusgrave/rayloop/blob/d5e797967c42b9c891f1fb0603cc2498a896d75d/src/main.zig#L526 https://github.com/hmusgrave/rayloop/blob/d5e797967c42b9c891...

messe 11mo ago

> The one thing you could say is that using some kind of token makes it dead simple to have different tokens. But that's really not something I run into often at all when using async. It's valuable to library authors who can now write code that's agnostic of the users' choice of runtime, while still being able to express that asynchronicity is possible for certain code paths.

rowanG077 11mo ago
But that can already be done using async await. If you write an async function in Rust for example you are free to call it with any async runtime you want.
- messe 11mo ago
  But you can't call it from synchronous rust. Zig is moving toward all sync code also using the Io interface.
  - tcfhgj 11mo ago
    yes, you can: runtime.block_on(async { }) https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=060fc0424005ecf2d892e55e39988a03 https://play.rust-lang.org/?version=stable&mode=debug&editio...
    - whytevuhuni 11mo ago
      Here's a problem with that: Cannot start a runtime from within a runtime. This happens because a function (like `block_on`) attempted to block the current thread while the thread is being used to drive asynchronous tasks. https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=8abf9e00ef6039d9cd5f8fd7adc6bab0 https://play.rust-lang.org/?version=stable&mode=debug&editio...
      - tcfhgj 11mo ago
        just pass around handles like you do in zig, alright? also: spawn_blocking for blocking code
        whytevuhuni 11mo ago
        But that's the thing, idiomatic Rust sync code almost never passes around handles, even when they need to do I/O. You might be different, and you might start doing that in your code, but almost none of either std or 3rd party libraries will cooperate with you. The difference with Zig is not in its capabilities, but rather in how the ecosystem around its stdlib is built. The equivalent in Rust would be if almost all I/O functions in std would be async; granted that would be far too expensive and disruptive given how async works.
        tcfhgj 11mo ago
        > But that's the thing, idiomatic Rust sync code almost never passes around handles, even when they need to do I/O. Because they don't use async inside. Zig code is passing around handles in code without io?
        whytevuhuni 11mo ago
        > Because they don't use async inside. But they use I/O inside, and we arrive at this issue: I'm writing async, and I need to call std::fs::read. I can't, because it blocks the thread; I could use spawn_blocking but that defeats the purpose of async. So instead I have to go look for a similar function but of the other color, probably from tokio. In Zig, if you're writing sync, you call the standard library function for reading files. If you're writing async, you call the same library function for reading files. Then, the creator of the `io` object decides whether the whole thing will be sync or async.
    - messe 11mo ago
      Let me rephrase, you can't call it like any other function. In Zig, a function that does IO can be called the same way whether or not it performs async operations or not. And if those async operations don't need concurrency (which Zig expresses separately to asynchronicity), then they'll run equally well on a sync Io runtime.
      - tcfhgj 11mo ago
        > In Zig, a function that does IO can be called the same way whether or not it performs async operations or not. no, you can't, you need to pass a IO parameter
        messe 11mo ago
        You will need to pass that for synchronous IO as well. All IO in the standard library is moving to the Io interface. Sync and async. If I want to call a function that does asynchronous IO, I'll use: foo(io, ...); If I want to call one that does synchronous IO, I'll write: foo(io, ...); If I want to express that either one of the above can be run asynchronously if possible, I'll write: io.async(foo, .{ io, ... }); If I want to express that it must be run concurrently, then I'll write: try io.concurrent(foo, .{ io, ... }); Nowhere in the above do I distinguish whether or not foo does synchronous or asynchronous IO. I only mark that it does IO, by passing in a parameter of type std.Io.
        tcfhgj 11mo ago
        what about non-io code?
        messe 11mo ago
        What about it? It gets called without an Io parameter. Same way that a function that doesn't allocate doesn't get an allocator. I feel like you're trying to set me up for a gotcha "see, zig does color functions because it distinguishes functions that do io and those that don't!". And yes, that's true. Zig, at least Zig code using std, will mark functions that do Io with an Io parameter. But surely you can see how that will lead to less of a split in the ecosystem compared to sync and async rust?

hansvm 11mo ago

Making it dead simple to have different tokens is exactly the goal. A smattering of examples recently on my mind: As a background, you might ask why you need different runtimes ever. Why not just make everything async and be done with it, especially if the language is able to hide that complexity? 1. In the context of a systems language that's not an option. You might be writing an OS, embedded code, a game with atypical performance demands requiring more care with the IO, some kernel-bypass shenanigan, etc. Even just selecting between a few builtin choices (like single-threaded async vs multi-threaded async vs single-threaded sync) doesn't provide enough flexibility for the range of programs you're trying to allow a user to write. 2. Similarly, even initializing a truly arbitrary IO effect once at compile-time doesn't always suffice. Maybe you normally want a multi-threaded solution but need more care with respect to concurrency in some critical section and need to swap in a different IO. Maybe you normally get to interact with the normal internet but have a mode/section/interface/etc where you need to send messages through stranger networking conditions (20s ping, 99% packet loss, 0.1kbps upload on the far side, custom hardware, etc). Maybe some part of your application needs bounded latency and is fine dropping packets but some other part needs high throughput and no dropped packets at any latency cost. Maybe your disk hardware is such that it makes sense for networking to be async and disk to be sync. And so on. You can potentially work around that in a world with a single IO implementation if you can hack around it with different compilation units or something, but it gets complicated. Part of the answer then is that you need (or really want) something equivalent to different IO runtimes, hot-swappable for each function call. I gave some high-level ideas as to why that might be the case, but high-level observations often don't resonate, so let's look at a concrete case where `await` is less ergonomic: 1. Take something like TLS as an example (stdlib or 3rd-party, doesn't really matter). The handshake code is complicated, so a normal implementation calls into an IO abstraction layer and physically does reads and writes (as opposed to, e.g., a pure state-machine implementation which returns some metadata about which action to perform next -- I hacked together a terrible version of that at one point [0] if you want to see what I mean). What if you want to run it on an embedded device? If it were written with async it would likely have enough other baggage that it wouldn't fit or otherwise wouldn't work. What if you want to hide your transmission in other data to sneak it past prying eyes (steganography, nowadays that's relatively easy to do via LLMs interestingly enough, and you can embed arbitrary data in messages which are human-readable and purport to discuss completely other things without exposing hi/lo-bit patterns or other such things that normally break steganography)? Then the kernel socket abstraction doesn't work at all, and "just using await" doesn't fix the problem. Basically, any place you want to use that library (and, arguably, that's the sort of code where you should absolutely use a library rather than rolling it yourself), if the implementer had a "just use await" mentality then you're SOL if you need to use it in literally any other context. I was going to write more concrete cases, but this comment is getting to be too long. The general observation is that "just use await" hinders code re-use. If you're writing code for your own consumption and also never need those other uses then it's a non-issue, but with a clever choice of abstraction it _might_ be possible (old Zig had a solution that didn't quite hit the mark IMO, and time will tell if this one is good enough, but I'm optimistic) to enable the IO code people naturally write to be appropriately generic by default and thus empower future developers via a more composable set of primitives. They really nailed that with the allocator interface, and if this works then my only real concern is a generic "what next" -- it's pushing toward an effect system, but integrating those with a systems language is mostly an unsolved problem, and adding a 3rd, 4th, etc explicit parameter to nearly every function is going to get unwieldy in a hurry (back-of-the-envelope idea I've had stewing if I ever write a whole "major" language is to basically do what Zig currently does and pack all those "effects" into a single effect parameter that you pass into each function, still allowing you to customize each function call, still allowing you to inspect which functions require allocators or whatever, but making the experience more pleasant if you have a little syntactic sugar around sub-effects and if the parent type class is comptime-known). [0] https://github.com/hmusgrave/rayloop/blob/d5e797967c42b9c891f1fb0603cc2498a896d75d/src/main.zig#L526 https://github.com/hmusgrave/rayloop/blob/d5e797967c42b9c891...