6 ms·
Zig's new I/O: function coloring is inevitable?
- audunw 1y agoI really don’t agree with the idea that this is functional colouring. Then we have to start talking about function colouring in a whole bunch of new contexts like with Zigs explicit passing of allocator. Or any other parameter that needs to be explicitly passed to use some kind of interface. I think we should stick to talking about colouring when there is special calling conventions or syntax, which has the consequence of having to write separate libraries/modules for async code and non-async code. That is the significant problem we have been seeing with many async implementation, and the one which Zig apparently fully solves.
- dwattttt 1y ago> Then we have to start talking about function colouring in a whole bunch of new contexts like with Zigs explicit passing of allocator. That's pretty much where we are though. If you have a function that isn't passed an allocator, and now it needs to call a function that does take an allocator, we're in the same place. Rust's 'async' keyword changes the type of the return value, but you can just write the different return value yourself; it's 'coloured' purely by what it means to be returning a 'Future'.
- koolala 1y agoSo its impossible to get an allocator through a different means from inside a function? It must be passed in?
- dwattttt 1y agoNo, it's entirely possible. And that's why we don't think about "has an allocator" as a colouring problem. Likewise if JavaScript had an easy way to get a handle to its Runtime, and a function "block on promise" in its early days, we'd have never had all these "colouring" arguments.
- Dylan16807 1y agoIf javascript worked differently we wouldn't have arguments about that significant feature of javascript. How would you add "block on promise"? Javascript started off without the ability to block on anything, not even I/O because it didn't have I/O. Later when XMLHttpRequeest was added it was purely callback-based. There are ways to make that API work but introducing the concept of blocking would be a big language change. It's not fixing a flaw, it's trading away some complications for a different pile of complications. There's still a conflict to resolve in language design when you want some functions that are synchronous and don't block and other functions that are asynchronous. Edit: I guess javascript blocks on alert() but that doesn't seem like what you'd want "block on promise" to do...
- dwattttt 1y agoJavaScript is (notionally) "single threaded", you can block with just a busy loop; that's not a language change. It's bad design to block, which I expect is what you mean, but that's not a language problem. If you did so in a Worker, you'd have your "don't block the UI thread" in there too.
- Dylan16807 1y agoIf you busy loop waiting for something in javascript, your program dies. They're not just a bad idea, they're a non-option. Blocking in the sense of blocking I/O or blocking on a system call is not possible in javascript. So again I ask what you have in mind for a "block on promise" call. The ideas that come to my mind all require significant redesigns of the execution model, or copping out by making every function async and treating it like an await. In particular with workers in mind, I'll put it this way: If you don't allow "block on promise" in the main thread, you generally avoid these issues, but you also don't solve the problem, you still have async versus not async causing many headaches. If you do allow it on the main thread, how do you keep your page functioning? I can imagine resolutions but they all have severe compromises.
- dwattttt 1y ago
- woodruffw 1y agoThis is effectively a special calling convention: Zig expects you to pass in a "token" object that communicates a kind of effect (I/O in this case). No token, no effect (modulo a soundness hole). This is not a new pattern, and I think it's a pretty good one (and is arguably more ergonomic and general than syntax-level effects). But it's quintessential function coloring.
- throwawaymaths 1y ago> Zig expects you to pass It does no such thing. you could pass a function a vtable and the vtable could have one implementation that calls an io stashed in the parent of the vtable, and a different vtable that doesnt and the function calling the vtable would be none the wiser. what is the color of the function that took the vtable? this is not just academic; it would be for example the basis for mocked integration tests on a database or over the net api call.
- woodruffw 1y agoThat's a calling convention, with indirection. You need some kind of capability token for this kind of asynchronicity scheme; it doesn't matter how you get it, but it needs to be there. To be clear, there's nothing wrong with this; it's just another way to encode capabilities/effects. > what is the color of the function that took the vtable? It has the I/O effect.
- throwawaymaths 1y ago> It has the I/O effect it does not, because it can take a vtable that does not call i/o concretely: const VTable = struct { f: &fn (*VTable) void, }; const A = struct { io: IO, v: VTable = .{ .f = &A.uses_io }, fn uses_io(this: *VTable) void { const self: *A = @fieldParentPtr(.v, this); self.io.some_io_fn(...); } }; const B = struct{v: VTable = .{.f = &void_fn}}; fn void_fn(_: *VTable) void {} \\ WHAT IS THE COLOR OF THIS FUNCTION? pub fn calls_vtable(v: VTable) { v.f() }
- hamandcheese 1y ago> which has the consequence of having to write separate libraries/modules for async code and non-async code. That is not a consequence of function coloring or syntax, it is a consequence of having multiple ways of performing IO.
- deleted 1y ago[deleted]
- camgunz 1y agoLook, either you move the program counter to a different place in memory (function call) or you push a task into an event loop. Even if you somehow elide all these differences, they're so different under the hood you'll always have to know in some circumstances. It's honestly wild we conflate them at all.
- kristoff_it 1y agoIf you want to go down that route, any function that has, or doesn't have, any given resource is colored then. fn foo(db: *Db) !void { ... } fn bar() !void { ... } Would you consider `foo` a blue function and `bar` a red function? That doesn't seem particularly helpful to me. The virality of async await is that once you mark a function async, then you can only call it from another async function, which forces you to mark more functions async, which in turn means that if you want to use blocking I/O APIs then you just can't because it's incompatible with your execution model because by daring to express asynchronicity of operations, you were forcefully opted into stackless coroutines. That's what Zig solves, and that's what is real function coloring. People have written reimplementations of the same libraries multiple times because of it. https://github.com/redis/redis-py https://github.com/redis/redis-py https://github.com/jonathanslenders/asyncio-redis https://github.com/jonathanslenders/asyncio-redis Just as an example. Note also how, coincidentally, this duplication of effort resulted in asyncio-redis being semi-abandoned and looking for maintainers. And you have to have both libraries because the asyncio one can't do blocking, and vice versa the other one can't do async. Would you write two instances of essentially the same library just because one is missing an argument that gives it access to an `Io` interface? No, because you would just pass that extra argument around and nothing else would have to change.
- n42 1y agoI've been trying to beat this point in and failing. If a parameter type creates "colors", you can extrapolate that to an infinite set of colors in every single language and every single standard library, and the discussion on colors becomes meaningless. Some people are so focused on categorical thinking that they are missing the forest for the trees. The colors are a means of describing an observed outcome -- in Node's case, callback hell, in Rust's, 4 different standard libraries. Whatever it may be, the point is not that there are colors, it's the impact on there being colors. > But there is a catch: with this new I/O approach it is impossible to write to a file without std.Io! This sentence just makes me laugh, like it's some kind of "gotcha". It is the ENTIRE BASIS of the design!
- deleted 1y ago[deleted]
- orphea 1y agoRelated discussion (Zig's New Async I/O): https://news.ycombinator.com/item?id=44545949 https://news.ycombinator.com/item?id=44545949
- AYBABTME 1y agoOne way or another, I like how this is implemented. Explicitely passing dependencies like this, with a tight syntax, makes things easy to understand and write. I haven't written any Zig but these demonstrations give me strong vibes of how I felt when I picked up Go more than 10y ago.
- dwattttt 1y agoFunction colouring created a lot of angst when it first came about, particularly because of the difficulties of calling a function of one colour from another. Whether that was possible, what it actually meant, wasn't really well defined. As other comments have said, there's nothing special about "colouring"; sync/async functions are a case where those above problems are tough, but simpler versions of the problem are everywhere and we don't freak out about them e.g. call a fallible function from an infallible function. It really all turns on how easy it is to ultimately make the call to the other "function" colour. In Zig's case, if its easy to get an Io in a function that didn't take an Io, it's a non-issue. Likewise for the "fallible function call from infallible function": if it fails, do something that doesn't result in the infallible function failing (do something else? Terminate? Anything will do).
- andyferris 1y agoI am actually really excited about this. The issues I've had with function colouring had to do with trying to compose code using (or expecting) blocking effects with those using async effects in NodeJS - if one library has a higher-order function that expects a non-async function and you have functionality which is provided to you as async, it can be very difficult to plumb them together! And if it's the other way around, it can be quite the performance killer (think how much faster better-sqlite3 is than alternatives). Zig's approach eliminates this problem, AFAICT. If I had to choose between having to pass through an effect handler like `io` or write `async` everywere, the former seems like a better use of my time. It's explicit, but that can be good. It also fits Zig well with the allocator. Code can expect an allocator or perhaps an allocator and `io`, or perhaps neither. It's analogous to Rust code that is core vs alloc/nostd vs std. I am slightly amused that a "C-but-better" language is going to have an `io` passed through non-pure functions much like Haskell. It's that idea combined with Rust's pluggable async runtimes (and stackless concurrency) combined with Roc's "platforms" - but for systems programmers. Quite amazing.
- ivanjermakov 1y ago> much like Haskell I have not mentioned this in the post, but it's surprising how monadic I/O operations are, including async/await approach And it is certainly in FP land to pass I/O as parameter, because everything is a function
- et1337 1y agoI think this is a good kind of function coloring. It would avoid some scars I have from: - seemingly harmless functions that unexpectedly end up writing four different files to disk. - Packages that do I/O or start threads when you simply import them.
- hiccuphippo 1y agoI don't care about the color of the function. What matters is if you'll have to write two versions of the function, and this seems to solve that.
- koiueo 1y agoIf we apply your logic, then every mainstream language has the coloring problem with not just two, but a multitude of colors. // OMG we can't call this without passing the service // This function is people-colored public Person findPersonByName(PeopleService service, String name) { // OMG we can't find without the service service.find(name) } EDIT: formatting
- vips7L 1y agoThis is off topic. But I really hate “services”. They always are just some random collection of functions and aren’t a real object at all.
- koiueo 1y agoI think I know what you are talking about. In a typical three-tier enterprise application there's no clear separation of concerns between "services" and DAO/Repo... And this is due to how most persistence libraries are designed: they eagerly execute side-effects, this breaks composability due to poor transaction control, so you end up leaking a lot of your business logic into your supposedly persistence layer. Take a look at Scala's doobie[1]. Any doobie operation returns a `ConnectionIO`, which is only a description of an operation (free monad). With a proper doobie usage the DAO layer is an algebra of possible persistence-related operations, and the service layer implements business logic by combining primitive ConnectionIOs and interpreting them with full control of transaction boundaries. [1] https://typelevel.org/doobie/index.html https://typelevel.org/doobie/index.html
- jmull 1y agoWith Zig I/O you can call i/o functions from non-i/o functions and non-i/o functions from i/o functions. In the analogy of “What color is your function”, you can call blue functions from red functions and red functions from blue functions. The pernicious viral nature of function coloring doesn’t apply.
- ivanjermakov 1y ago> With Zig I/O you can call i/o functions from non-i/o functions and non-i/o functions from i/o functions Yes, but you would either create a new std.Io just for that, or use a global/semi-global std.Io from somwhere else
- jmull 1y agoYes… like every other argument one function provides to another that it doesn’t pass directly through from one of its own arguments. The function color analogy captures a pernicious viral dynamic with async where it allows only one-way composition. That just doesn’t exist with zig i/o.
- Surac 1y agoi never got warm with this async/await style of calling things. i much more prefer using threads/fibers/protothreads instead. Sure you have to be more expicit on creating and controlling these and also passing around data may get more complex, but at least they do not gloss over the underlying problems async and await tries to cover up
- SkiFire13 1y agoI believe Zig's new I/O is neither colored nor not colored, instead the design is orthogonal to that. You won't be able to make code magically async by just swapping the implementation of Io, instead you'll need some kind of runtime that allows functions to wait without blocking OS threads. Depending on how this is implemented (if it's possible at all) you'll have colored functions or not.