14 ms·

I feel that I have to point this out once again, because the article goes so far as to state that: > With this last improvement Zig has completely defeated fun

• andyferris 1y ago
  I think of it this way. Given an `io` you can, technically, build another one from it with the same interface. For example given an async IO runime, you could create an `io` object that is blocking (awaits every command eagerly). That's not too special - you can call sync functions from async functions. (But in JavaScript you'd have trouble calling a sync function that relies on `await`s inside, so that's still something). Another thing that is interesting is given a blocking posix I/O that also allows for creating processes or threads, you could build in userspace a truly asynchronous `io` object from that blocking one. It wouldn't be as efficient as one based directly on iouring, and it would be old school, but it would basically work. Going either way (changing `io` to sync or async) the caller doesn't actually care. Yes the caller needs a context, but most modern apps rely on some form of dependency injection. Most well-factored apps would probably benefit from a more refined and domain-specific "environment" (or set of platform effects, perhaps to use the Roc terminology), not Zig's posix-flavoured standard library `io` thing. Yes rust achieves this to some extent; you can swap an async runtime for another and your app might still compile and run fine. Overall I like this alot - I am wondering if Richard Feldmann managed to convince Andrew Kelley that "platforms" are cool and some ideas were borrowed from Roc?
  • dminik 1y ago
    > but most modern apps rely on some form of dependency injection Does Zig actually do anything here? If anything, this seems to be anti-Zig, where everything must be explicit.
    • philwelch 1y ago
      Passing in your dependencies as function arguments is a form of dependency injection. It is the simplest and thus arguably best form of dependency injection.
      • almostgotcaught 1y ago
        This is like saying arithmetic is a form of calculus, the simplest form. Ie it reduces the concept (DI) to a meaningless tautology.
        • philwelch 1y ago
          Not at all. Dependency injection is the injection of dependencies as logical parameters. The simplest and arguably best way to inject logical parameters to a segment of code is to use function parameters. You can have a complicated DI framework that involves Java class annotations and megabytes of XML, but that’s not the central idea.
          • dminik 1y ago
            I'm sorry but if I read > most modern apps rely on some form of dependency injection then plain parameter passing is not what I'm thinking of. Especially not manually doing that to hundreds or thousands of calls.
            • philwelch 1y ago
              Most forms of dependency injection are abstractions over parameter passing. If you find yourself passing the same parameters into many different function calls there are ways of abstracting that, even in Zig. It’s not going to look like Spring and if that’s a dealbreaker for you, just use Spring, it’s a free country.
• mlugg 1y ago
  The key difference to typical async function coloring is that `Io` isn't something you need specifically for asynchronicity; it's something which (unless you make a point to reach into very low-level primitives) you will need in order to perform any IO, including reading a file, sleeping, getting the time, etc. It's also just a value which you can keep wherever you want, rather than a special attribute/property of a function. In practice, these properties solve the coloring problem: * It's quite rare for a function to unexpectedly gain a dependency on "doing IO" in general. In practice, most of your codebase will have access to an `Io`, and only leaf functions doing pure computation will not need them. * If a function does start needing to do IO, it almost certainly doesn't need to actually take it as a parameter. As in many languages, it's typical in Zig code to have one type which manages a bunch of core state, and which the whole codebase has easy access to (e.g. in the Zig compiler itself, this is the `Compilation` type). Because of this, despite the perception, Zig code doesn't usually pass (for instance) allocators explicitly all the way down the function call graph! Instead, your "general purpose allocator" is available on that "application state" type, so you can fetch it from essentially wherever. IO will work just the same in practice. So, if you discover that a code path you previously thought was pure actually does need to perform IO, then you don't need to apply some nasty viral change; you just grab `my_thing.io`. I do agree that in principle, there's still a form of function coloring going on. Arguably, our solution to the problem is just to color every function async-colored (by giving most or all of them access to an `Io`). But it's much like the "coloring" of having to pass `Allocator`s around: it's not a problem in practice, because you'll basically always have easy access to one even if you didn't previously think you'd need it. I think seasoned Zig developers will pretty invariably agree with the statement that explicitly passing `Allocator`s around really does not introduce function coloring annoyances in practice, and I see no reason that `Io` would be particularly different.
  • dminik 1y ago
    > It's quite rare for a function to unexpectedly gain a dependency on ... If this was true in general, the function coloring problem wouldn't be talked about. However, the second point is more interesting. I think there's a bit of a Stockholm syndrome thing here with Zig programmers and Allocator. It's likely that Zig programmers won't mind passing around an extra param. If anything, it would make sense to me to have IO contain an allocator too. Allocation is a kind of IO too. But I guess it's going to be 2 params from now on.
• n42 1y ago
  Aside from the ridiculous argument that function parameters color them, the assertion that you can’t call a function that takes IO from inside a function that does not is false, since you can initialize one to pass it in
  • dminik 1y ago
    To me, there's no difference between the IO param and async/await. Adding either one causes it to not be callable from certain places. As for the second thing: You can do that, but... You can also do this in Rust. Yet nobody would say Rust has solved function coloring. Also, check this part of the article: > In the less common case when a program instantiates more than one Io implementation, virtual calls done through the Io interface will not be de-virtualized, ... Doing that is an instant performance hit. Not to mention annoying to do.
    • n42 1y ago
      You’re allowed to not like it, but that doesn’t change that your argument that this is a form of coloring is objectively false. I’m not sure what Rust has to do with it.
      • rowanG077 1y ago
        Sure it is a function coloring. Just in a different form. `async` in other languages is something like an implicit parameter. In zig they made this implicit parameter explicit. Is that more better/more ergonomic? I don't know yet. The sugar is different, but the end result the same. Unless you can show me concrete example of things that the approach zig has taken can do that is not possible in say, rust. Than I don't buy that it's not just another form of function coloring.
        • throwawaymaths 1y ago
          > Unless you can show me concrete example add io to a struct and let the struct keep track of its own io.
• ayuhito 1y ago
  Go also suffers from this form of “subtle coloring”. If you’re working with goroutines, you would always pass in a context parameter to handle cancellation. Many library functions also require context, which poisons the rest of your functions. Technically, you don’t have to use context for a goroutine and could stub every dependency with context.Background, but that’s very discouraged.
  • tidwall 1y ago
    Context is not required in Go and I personally encourage you to avoid it. There is no shame in blazing a different path.
    • nu11ptr 1y ago
      What would you use in its place? I've never had an issue with it. I use it for 1) early termination 2) carrying custom request metadata. I don't really think it is fully the coloring problem because you can easily call non-context functions from context functions (but not other way around, so one way coloring issue), but you need to be aware the cancellation chain of course stops then.
    • schrodinger 1y ago
      Why do you encourage avoiding it? Afaik it's the only way to early-abort an operation since Goroutines operate in a cooperative, not preemptive, paradigm. To be very clear, I'm asking this completely in good faith looking to learn something new!
      • pjmlp 1y ago
        You are expecting them to actually check the value, there is nothing preemptive. Another approach is special messages over a side channel.
        • deepsun 1y ago
          So instead of a context you need to pass a a channel. Same problem.
• kristoff_it 1y ago
  Here's a trick to make every function red (or blue? I'm colorblind, you decide): var io: std.Io = undefined; pub fn main() !void { var impl = ...; io = impl.io(); } Just put io in a global variable and you won't have to worry about coloring in your application. Are your functions blue, red or green now? Jokes aside, I agree that there's obviously a non-zero amount of friction to using the `Io` intreface, but it's something qualitatively very different from what causes actual real-world friction around the use of async await. > but the general problem behind function coloring is that of context I would disagree, to me the problem seems, from a practical perspective that: 1. Code can't be reused because the async keyword statically colors a function as red (e.g. python's blocking redis client and asyncio-redis). In Zig any function that wants to do Io, be it blue (non-async) or red (async) still has to take in that parameter so from that perspective the Io argument is irrelevant. 2. Using async and await opts you automatically into stackless coroutines with no way of preventing that. With this new I/O system even if you decide to use a library that interally uses async, you can still do blocking I/O, if you want. To me these seems the real problems of function coloring.
  • ismailmaj 1y ago
    The global io trick would totally be valid if you’re writing an application (i.e. not a library) and don’t have use of two different implementations of io
    • throwawaymaths 1y ago
      you could still have a library-global io, let the user set it as desired. > use of two different implementations of io functionally rare situation.
    • laserbeam 1y ago
      There are plenty of libraries out there which require users to do an init() call of some sorts at startup. It is perfectly possible to design a library that only works with 1 io instance and gets it at init(). Whether people like or want that… I have no clue.
  • 1y ago
• cryptonector 1y ago
  > Well, you don't have async/sync/red/blue anymore, but you now have IO and non-IO functions. > However, the coloring problem hasn't really been defeated. Well, yes, but if the only way to do I/O were to have an Io instance to do it with then Io would infect all but pure(ish, non-Io) functions, so calling Io functions would be possible in all but those contexts where calling Io functions is explicitly something you don't want to be possible. So in a way the color problem is lessened. And on top of that you get something like Haskell's IO monad (ok, no monad, but an IO interface). Not too shabby, though you're right of course. Next Zig will want monadic interfaces so that functions only have to have one special argument that can then be hidden.
  • throwawaymaths 1y ago
    Zig's not really about hiding things but you could put it in an options struct that has defaults unless overridden at compile time.
• nmilo 1y ago
  The original “function colouring” blogpost has done irreparable damage to PL discussions because it’s such a stupid concept to begin with. Of course I want async functions to be “coloured” differently, they do different things! How else is a “normal function” supposed to call a function that gives you a result later——obviously you want to be forced to say what to do with the result; await it, ignore it, .then() in JS terms, etc. these are important decisions that you can’t just ignore because it’s “painful”
  • yxhuvud 1y ago
    There is nothing obvious around that - it is driven by what abstractions the language provides related to concurrency, and with different choices you will end needing different ways to interact with it. So yes, given how the language designers of C# and JavaScript choose to implement concurrency and the APIs around that, then coloring is necessary. But it is very much implementation driven and implementation of other concurrency models then other ways to do it that don't involve keywords can make sense. So when people complain about function coloring, they are complaining about the choice of concurrency model that a language uses.
• throwawaymaths 1y ago
  > Technically you could pass in a new executor, but is that really what you want? why does it have to be new? just use one executor, set it as const in some file, and use that one at every entrypoint that needs io! now your io doesn't propagate downwards.
• jaredklewis 1y ago
  So this is a tangent from the main article, but this comment made me curious and I read the original "What color is Your Function" post. It was an interesting read, but I guess I came away confused about why "coloring" functions is a problem. Isn't "coloring" just another form of static typing? By giving the compiler (or interpreter) more meta data about your code, it can help you avoid mistakes. But instead of the usual "first argument is an integer" type meta data, "coloring" provides useful information like: "this function behaves in this special way" or "this function can be called in these kinds of contexts." Seems reasonable? Like the author seems very perturbed that there can be different "colors" of functions, but a function that merely calculates (without any IO or side-effects) is different than one that does perform IO. A function with only synchronous code behaves very differently than one that runs code inside another thread or in a different tick of the event loop. Why is it bad to have functions annotated with this meta data? The functions behave in a fundamentally different way whether you give them special annotations/syntax or not. Shouldn't different things look different? He mentions 2015 era Java as being ok, but as someone that’s written a lot of multithreaded Java code, it’s easy to mess up and people spam the “synchronized” keyword/“color” everywhere as a result. I don’t feel the lack of colors in Java makes it particularly intuitive or conceptually simpler.
  • deleted 1y ago
    [deleted]
  • com2kid 1y ago
    > Isn't "coloring" just another form of static typing? Yes, and so is declaring what exceptions a function can throw (checked exceptions in Java). > Why is it bad to have functions annotated with this meta data? The functions behave in a fundamentally different way whether you give them special annotations/syntax or not. Shouldn't different things look different? It really isn't a problem. The article makes people think they've discovered some clever gotcha when they first read it, but IMHO people who sit down for a bit and think through the issue come to the same conclusion you have - Function coloring isn't a problem in practice.
    • kristoff_it 1y ago
• ozgrakkurt 1y ago
  You are skipping the massive point here. If you are using a library in rust, it has to be async await, tokio, send+sync and all the other crap. Or if it is sync api then it is useless for async application. This approach of passing IO removes this problem and this is THE main problem. This way you don’t have to use procedural macros or other bs to implement multi versioning for the functions in your library, which doesn’t work well anyway in the end. https://nullderef.com/blog/rust-async-sync/ https://nullderef.com/blog/rust-async-sync/ You can find 50 other ones like this by searching. To be honest I don’t hope they will solve cooperative scheduling, high performance, optionally thread-per-core async soon and the API won’t be that good anyway. But hope it solves all that in the future.
  • dwattttt 1y ago
    > Or if it is sync api then it is useless for async application. The rest is true, but this part isn't really an issue. If you're in an async function you can call sync functions still. And if you're worried it'll block and you can afford that, I know tokio offers spawn_blocking for this purpose.
  • tcfhgj 1y ago
    > If you are using a library in rust, it has to be async await, tokio, send+sync and all the other crap Send and sync is only required if you want to access something from multiple threads, which isn't required by async await (parallelism vs concurrency) 1) You can use async await without parallelism and 2) send and sync aren't a product of async/await in Rust, but generally memory safety, i.e. you need Send generally when something can/is allowed to move between threads.
    • m11a 1y ago
      Yes, but async Rust is basically built on tokio's runtime, which is what most the big async libraries depend on, like hyper/axum/tokio etc. And tokio is a thread-per-core work-stealing architecture, which requires Send + Sync bounds everywhere. You can avoid them if you depend on tokio-proper, but it's more icky when building on something like axum, where your application handlers also require these bounds. A good article on this: https://emschwartz.me/async-rust-can-be-a-pleasure-to-work-with-without-send-sync-static/ https://emschwartz.me/async-rust-can-be-a-pleasure-to-work-w...
• flohofwoe 1y ago
  > In order to call such a function you also need to provide the context. Zig hasn't really solved this. It is much more flexible though since you don't need to pass the IO implementation into each function that needs to do IO. You could pass it once into an init function and then use that IO impl throughout the object or module. Whether that's good style is debatable - the Zig stdlib currently has containers that take an allocator in the init function, but those are on the way out in favour of explicitly taking the allocator in each function that needs to allocate - but the user is still free to write a minimal wrapper to restore the 'pass allocator into init' behaviour. Odin has an interesting solution in that it passes an implicit context pointer into each function, but I don't know if the compiler is clever enough to remove the overhead for called functions that don't access the context (since it also needs to peek into all called functions - theoretically Zig with it's single-compilation-unit approach could probably solve that problem better).
  • tcfhgj 1y ago
    You can write a wrapper in other langs, too, e.g. in Rust: block_on(async_fn)
• nurettin 1y ago
  I think the point is 3 doesn't fully apply anymore. And that was the main pain point. You couldn't call a blue in red even if it didn't use IO without some kind of execution wrapper or waiter. Now you clearly can.
• tcfhgj 1y ago
  > If anything, it does a great job at abstracting the usage from the implementation. This is something Rust fails at spectacularly. Could you expand on this? I don't get what you mean
  • koito17 1y ago
    I am not very experienced in async Rust, but it seems there are some pieces of async Rust that rely too much on tokio internals, so using an alternative runtime (like pollster) results in broken code. Searching for comments mentioning "pollster" and "tokio" on HN brings a few results, but not one I recall seeing a while ago where someone demonstrated an example of a library (using async Rust) that crashes when not using tokio as the executor. Related documentation: https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html#determining-ecosystem-compatibility https://rust-lang.github.io/async-book/08_ecosystem/00_chapt...
    • tcfhgj 1y ago
      thanks, got it!
    • pimeys 1y ago
      Yep. The old async wars in the Rust ecosystem. It's the AsyncRead and AsyncWrite traits. Tokio has its own, there was a standard brewing at the same time in the futures crate. Tokio did their own thing, people burnt out, these traits were never standardized to std. So you cannot use most of the async crates easily outside Tokio.
    • conradludgate 1y ago
      There's two details that are important to highlight. tokio is actually 2 components, it's the async scheduler, and it's the IO runtime. Pollster is only a scheduler, and does not offer any IO functionality. You can actually use tokio libraries with pollster, but you need to register the IO runtime (and spawn a thread to manage it) - this is done with Runtime::enter() and it configures the thread local interface so any uses of tokio IO know what runtime to use. There are ideas to abstract the IO runtime interface into the async machinery (in Rust that's the Context object that schedulers pass into the Future) but so far that hasn't gotten anywhere.
• benreesman 1y ago
  I'll let a real category theorist get into the details that I'll likely flub, but the IO monad is where you end up if you start on this path. That context can be implicit, but it's there, and if you want any help from the compiler (to, for example, guide Claude Code towards useful outcomes) you've got to reify it as a real thing in the formality of the system. Async and coroutines are the graveyard of dreams for systems programming languages, and Andrew by independently rediscovering the IO monad and getting it right? Hope of a generation. Functions in the real world have colors: you can have predictable rules for moving between colors, or you can wing it and get C++ co_await and tokio and please kill me. This is The Way.
  • throwawaymaths 1y ago
    it's not a monad, since you can do unholy (for fp) things with it, like stash it in a struct and pass the struct around (even to functions which have no clue theres an io call) or just grab a globalized io object and use that at will arbitrarily at many entrypoints in your function. most importantly, besides the obvious situations (creating the io object, binding it to another object), it's not generally going to be returned from a function as part of its value.
    • benreesman 1y ago
      i'm relatively confident that andrew will happily break the language again (god love him for that) when it becomes clear that you really want that algebra. though i will say, for a systems language, it's probably better to invert the lift/unlift relationship, default to do-notation and explicitly unlift into pure functions. that's almost what const meant in C++ to begin with but it lost it's way.
      • throwawaymaths 1y ago
        you're just spreading FUD about zig breaking before 1.0. there is zero basis for you to be "relatively confident" about this matter, because 1. the io situation is basically restructuring analgously to the allocator situation, which is at this point battle tested. there are currently no "monads wrapping statefulness" anywhere in zig. 2. It's not in zig's nature to build something because it it satisifies an fp idiom. the abstractions and resulting "thing that the hardware does" (at least in release builds) are generally more or less obvious. the levels of compiler reinterpretation to achieve functional purity are not really the sort of thing that zig does. for example, zig does not have a privileged "iterator" method that the compiler reinterprets in a way that unrolls blocks or lambdas into loops without crossing a frame boundary.
• osigurdson 1y ago
  Agree that with something like go, there is truly no function coloring at all. However, since most real world async things require cancellation, a context parameter is always present so there is some "coloring" do to that. Still, it is much less viral than C# style async await as if you don't have a context in your call stack you can still create one when needed and call the function. I don't think it is reasonable to abstract cancellation in a way that nothing has to be passed in so perhaps the approach presented here is realistically as good as it gets.
• panzi 1y ago
  Does Zig have closures? If yes, than at least in that case the IO pointer can be a bound parameter. For languages with the async keyword function coloring also applied to closures.
• gpderetta 1y ago
  I have a much longer rant elsethread, but the tl;dr; is: In some languages red can call blue, but blue cannot call red (JS). In some other languages blue can call red, but the resulting combined function is blue (traditional async with optional blocking). Finally some languages allow blue to call red and having the resulting combined function to be red (lua, scheme, go, and I believe Zig). As color is no longer a n unabstractable restriction in these languages, it no different than other kind of typing.