3 ms·

If calling the same function with a different argument would be considered 'function coloring', every function in a program is 'colored' and the word loses its

If calling the same function with a different argument would be considered 'function coloring', every function in a program is 'colored' and the word loses its meaning ;) Zig actually also had solved the coloring problem in the old and abandondend async-await solution because the compiler simply stamped out a sync- or async-version of the same function based on the calling context (this works because everything is a single compilation unit).

deleted 11mo ago
[deleted]
woodruffw 11mo ago
> If calling the same function with a different argument would be considered 'function coloring', than every function in a program is 'colored' and the word loses its meaning ;) Well, yes, but in this case the colors (= effects) are actually important. The implications of passing an effect through a system are nontrivial, which is why some languages choose to promote that effect to syntax (Rust) and others choose to make it a latent invariant (Java, with runtime exceptions). Zig chooses another path not unlike Haskell's IO.
rowanG077 11mo ago
If your functions suddenly requires (currently)unconstructable instance "Magic" which you now have to pass in from somewhere top level, that indeed suffers from the same issue as async/await. Aka function coloring. But most functions don't. They require some POD or float, string or whatever that can be easily and cheaply constructed in place.
- mk12 11mo ago
  Colors for 2 ways of doing IO vs colors for doing IO or not are so different that it’s confusing to call both of them “function coloring problem”. Only the former leads to having to duplicate everything (sync version and async version). If only the latter was a thing, no one would have coined the term and written the blog post.
  - rowanG077 11mo ago
    IMO the problem was never about it actually doing IO or an async actions or whatever. It's about not being able to a call a async function from a sync function. Because in my experience you almost never wholesale move from sync to async everywhere. In fact I would consider that an extremely dangerous practice.
jcranmer 11mo ago
> If calling the same function with a different argument would be considered 'function coloring', than every function in a program is 'colored' and the word loses its meaning ;) I mean, the concept of "function coloring" in the first place is itself an artificial distinction invented to complain about the incongruent methods of dealing with "do I/O immediately" versus "tell me when the I/O is done"--two methods of I/O that are so very different that it really requires very different designs of your application on top of those I/O methods: in a sync I/O case, I'm going to design my parser to output a DOM because there's little benefit to not doing so; in an async I/O case, I'm instead going to have a streaming API. I'm still somewhat surprised that "function coloring" has become the default lens to understand the semantics of async, because it's a rather big misdirection from the fundamental tradeoffs of different implementation designs.
- omnicognate 11mo ago
  100% agree, but fortunately I don't think it is the "default lens". If it were nobody would be adding new async mechanisms to languages, because "what color is your function" was a self-described rant against async, in favour of lightweight threads. It does seem to have established itself as an unusually persistent meme, though.
- zelphirkalt 11mo ago
  Function coloring is the issue, that arises in practice, which is why people discuss, whether some approach solves it or does not. Why do you think it automatically follows, that with an async I/O you are going to have a streaming API? An async I/O can just like the sync I/O return a whole complete result, only that you are not waiting for that to happen, but the called async procedure will call you back once the result is calculated. I think a streaming API requires additional implementation effort, not merely async.
- conradev 11mo ago
  My understanding of this design is that you can write the logic separately from the decision to "do I/O immediately" versus "tell me when the I/O is done" You can write a parser thats outputs a DOM and run it on a stream, or write a parser with a streaming API and run it synchronously on a buffer. You should pick the optimal tool for the situation, but there is no path dependence anymore.
adamwk 11mo ago
The subject of the function coloring article was callback APIs in Node, so an argument you need to pass to your IO functions is very much in the spirit of colored functions and has the same limitations.
- jakelazaroff 11mo ago
  In Zig's case you pass the argument whether or not it's asynchronous, though. The caller controls the behavior, not the function being called.
  - layer8 11mo ago
    The coloring is not the concrete argument (Io implementation) that is passed, but whether the function has an Io parameter in the first place. Whether the implementation of a function performs IO is in principle an implementation detail that can change in the future. A function that doesn't take an Io argument but wants to call another function that requires an Io argument can't. So you end up adding Io parameters just in case, and in turn require all callers to do the same. This is very much like function coloring. In a language with objects or closures (which Zig doesn't have first-class support for), one flexibility benefit of the Io object approach is that you can move it to object/closure creation and keep the function/method signature free from it. Still, you have to pass it somewhere.
    - quantummagic 11mo ago
      Is that a problem in practice though? Zig already has this same situation with its memory allocators; you can't allocate memory unless you take a parameter. Now you'll just have to take a memory allocator AND an additional io object. Doesn't sound very ergonomic to me, but if all Zig code conforms to this scheme, in practice there will only-one-way-to-do-it. So one of the colors will never be needed, or used.
    - derriz 11mo ago
      > A function that doesn't take an Io argument but wants to call another function that requires an Io argument can't. Why? Can’t you just create an instance of an Io of whatever flavor you prefer and use that? Or keep one around for use repeatedly? The whole “hide a global event loop behind language syntax” is an example of a leaky abstraction which is also restrictive. The approach here is explicit and doesn’t bind functions to hidden global state.
SkiFire13 11mo ago
> Zig actually also had solved the coloring problem in the old and abandondend async-await solution because the compiler simply stamped out a sync- or async-version of the same function based on the calling context (this works because everything is a single compilation unit). AFAIK this still leaked through function pointers, which were still sync or async (and this was not visible in their type)
- throwawaymaths 11mo ago
  Pretty sure the Zig team is aware of this and has plans to fix it before they re-release async.
  - geon 11mo ago
    I’m pretty sure that was an issue specifically with the old implementation, and not something still left to fix.
zarzavat 11mo ago
In that case JS is not colored either because an async function is simply a normal function that returns a Promise. As far as I understand, coloring refers to async and sync functions having the same calling syntax and interface, I.e. b = readFileAsync(p) b = readFileSync(p) share the same calling syntax. Whereas b = await readFileAsync(p) readFileAsync(p).then(b => ...) b = readFileSync(b) are different. If you have to call async functions with a different syntax or interface, then it's colored.
- flohofwoe 11mo ago
  > In that case JS is not colored either because an async function is simply a normal function that returns a Promise. Exactly, IMHO at least, JS doesn't suffer from the coloring problem because you can call async functions from sync functions (because the JS Promise machinery allows to fall back to completion callbacks instead of using await). It's the 'virality' of await which causes the coloring problem, but in JS you can freely mix await and completion callbacks for async operations).
  - zarzavat 11mo ago
    await isn't viral per se, it's a purely local transformation. The virality is from CPS/callbacks and Promise.
  - wavemode 11mo ago
    No, async and callbacks in JS are extremely viral. If a function returns a Promise or takes a callback, there is no possible way to execute it synchronously. Hence, coloring. The reason this coloring isn't a problem for the JS ecosystem, is that it's a single-threaded language by design. So, async/callbacks are the only reasonable way to do anything external to the JS runtime (i.e. reading files, connecting to APIs, etc.) (notwithstanding that node.js introduced some synchronous external operations in its stdlib - those are mostly unused in practice.) To put it a different way - yes, JS has function coloring, but it's not a big deal because almost the entire JS ecosystem is colored red anyway.
unscaled 11mo ago
Let's revisit the original article[1]. It was not about arguments, but about the pain of writing callbacks and even async/await compared to writing the same code in Go. It had 5 well-defined claims about languages with colored functions: 1. Every function has a color. This is true for the new zig approach: functions that deal with IO are red, functions that do not need to deal with IO are blue. 2. The way you call a function depends on its color. This is also true for Zig: Red functions require an Io argument. Blue functions do not. Calling a red function means you need to have an Io argument. 3. You can only call a red function from within another red function. You cannot call a function that requires an Io object in Zig without having an Io in context. Yes, in theory you can use a global variable or initialize a new Io instance, but this is the same as the workarounds you can do for calling an async function from a non-async function For instance, in C# you can write 'Task.Run(() -> MyAsyncMethod()).Wait()'. 4. Red functions are more painful to call. This is true in Zig again, since you have to pass down an Io instance. You might say this is not a big nuisance and almost all functions require some argument or another... But by this measure, async/await is even less troublesome. Compare calling an async function in Javascript to an Io-colored function in Zig: function foo() { blueFunction(); // We don't add anything } async function bar() { await redFunction(); // We just add "await" } And in Zig: fn foo() void { blueFunction() } fn bar(io: Io) void { redFunction(io); // We just add "io". } Zig is more troublesome since you don't just add a fixed keyword: you need a add a variable that is passed along through somewhere. 5. Some core library functions are red. This is also true in Zig: Some core library functions require an Io instance. I'm not saying Zig has made the wrong choice here, but this is clearly not colorless I/O. And it's ok, since colorless I/O was always just hype. --- [1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
- ghosty141 11mo ago
  In my opinion you must have function coloring, it's impossible to do async (in the common sense) without it. If you break it down one function has a dependency on the async execution engine, the other one doesn't, and that alone colors them. Most languages just change the way that dependency is expressed and that can have impacts on the ergonomics.

deleted 11mo ago

[deleted]

woodruffw 11mo ago

> If calling the same function with a different argument would be considered 'function coloring', than every function in a program is 'colored' and the word loses its meaning ;) Well, yes, but in this case the colors (= effects) are actually important. The implications of passing an effect through a system are nontrivial, which is why some languages choose to promote that effect to syntax (Rust) and others choose to make it a latent invariant (Java, with runtime exceptions). Zig chooses another path not unlike Haskell's IO.

rowanG077 11mo ago

If your functions suddenly requires (currently)unconstructable instance "Magic" which you now have to pass in from somewhere top level, that indeed suffers from the same issue as async/await. Aka function coloring. But most functions don't. They require some POD or float, string or whatever that can be easily and cheaply constructed in place.

mk12 11mo ago
Colors for 2 ways of doing IO vs colors for doing IO or not are so different that it’s confusing to call both of them “function coloring problem”. Only the former leads to having to duplicate everything (sync version and async version). If only the latter was a thing, no one would have coined the term and written the blog post.
- rowanG077 11mo ago
  IMO the problem was never about it actually doing IO or an async actions or whatever. It's about not being able to a call a async function from a sync function. Because in my experience you almost never wholesale move from sync to async everywhere. In fact I would consider that an extremely dangerous practice.

jcranmer 11mo ago

> If calling the same function with a different argument would be considered 'function coloring', than every function in a program is 'colored' and the word loses its meaning ;) I mean, the concept of "function coloring" in the first place is itself an artificial distinction invented to complain about the incongruent methods of dealing with "do I/O immediately" versus "tell me when the I/O is done"--two methods of I/O that are so very different that it really requires very different designs of your application on top of those I/O methods: in a sync I/O case, I'm going to design my parser to output a DOM because there's little benefit to not doing so; in an async I/O case, I'm instead going to have a streaming API. I'm still somewhat surprised that "function coloring" has become the default lens to understand the semantics of async, because it's a rather big misdirection from the fundamental tradeoffs of different implementation designs.

omnicognate 11mo ago
100% agree, but fortunately I don't think it is the "default lens". If it were nobody would be adding new async mechanisms to languages, because "what color is your function" was a self-described rant against async, in favour of lightweight threads. It does seem to have established itself as an unusually persistent meme, though.
zelphirkalt 11mo ago
Function coloring is the issue, that arises in practice, which is why people discuss, whether some approach solves it or does not. Why do you think it automatically follows, that with an async I/O you are going to have a streaming API? An async I/O can just like the sync I/O return a whole complete result, only that you are not waiting for that to happen, but the called async procedure will call you back once the result is calculated. I think a streaming API requires additional implementation effort, not merely async.
conradev 11mo ago
My understanding of this design is that you can write the logic separately from the decision to "do I/O immediately" versus "tell me when the I/O is done" You can write a parser thats outputs a DOM and run it on a stream, or write a parser with a streaming API and run it synchronously on a buffer. You should pick the optimal tool for the situation, but there is no path dependence anymore.

adamwk 11mo ago

The subject of the function coloring article was callback APIs in Node, so an argument you need to pass to your IO functions is very much in the spirit of colored functions and has the same limitations.

jakelazaroff 11mo ago
In Zig's case you pass the argument whether or not it's asynchronous, though. The caller controls the behavior, not the function being called.
- layer8 11mo ago
  The coloring is not the concrete argument (Io implementation) that is passed, but whether the function has an Io parameter in the first place. Whether the implementation of a function performs IO is in principle an implementation detail that can change in the future. A function that doesn't take an Io argument but wants to call another function that requires an Io argument can't. So you end up adding Io parameters just in case, and in turn require all callers to do the same. This is very much like function coloring. In a language with objects or closures (which Zig doesn't have first-class support for), one flexibility benefit of the Io object approach is that you can move it to object/closure creation and keep the function/method signature free from it. Still, you have to pass it somewhere.
  - quantummagic 11mo ago
    Is that a problem in practice though? Zig already has this same situation with its memory allocators; you can't allocate memory unless you take a parameter. Now you'll just have to take a memory allocator AND an additional io object. Doesn't sound very ergonomic to me, but if all Zig code conforms to this scheme, in practice there will only-one-way-to-do-it. So one of the colors will never be needed, or used.
  - derriz 11mo ago
    > A function that doesn't take an Io argument but wants to call another function that requires an Io argument can't. Why? Can’t you just create an instance of an Io of whatever flavor you prefer and use that? Or keep one around for use repeatedly? The whole “hide a global event loop behind language syntax” is an example of a leaky abstraction which is also restrictive. The approach here is explicit and doesn’t bind functions to hidden global state.

SkiFire13 11mo ago

> Zig actually also had solved the coloring problem in the old and abandondend async-await solution because the compiler simply stamped out a sync- or async-version of the same function based on the calling context (this works because everything is a single compilation unit). AFAIK this still leaked through function pointers, which were still sync or async (and this was not visible in their type)

throwawaymaths 11mo ago
Pretty sure the Zig team is aware of this and has plans to fix it before they re-release async.
- geon 11mo ago
  I’m pretty sure that was an issue specifically with the old implementation, and not something still left to fix.

zarzavat 11mo ago

In that case JS is not colored either because an async function is simply a normal function that returns a Promise. As far as I understand, coloring refers to async and sync functions having the same calling syntax and interface, I.e. b = readFileAsync(p) b = readFileSync(p) share the same calling syntax. Whereas b = await readFileAsync(p) readFileAsync(p).then(b => ...) b = readFileSync(b) are different. If you have to call async functions with a different syntax or interface, then it's colored.

flohofwoe 11mo ago
> In that case JS is not colored either because an async function is simply a normal function that returns a Promise. Exactly, IMHO at least, JS doesn't suffer from the coloring problem because you can call async functions from sync functions (because the JS Promise machinery allows to fall back to completion callbacks instead of using await). It's the 'virality' of await which causes the coloring problem, but in JS you can freely mix await and completion callbacks for async operations).
- zarzavat 11mo ago
  await isn't viral per se, it's a purely local transformation. The virality is from CPS/callbacks and Promise.
- wavemode 11mo ago
  No, async and callbacks in JS are extremely viral. If a function returns a Promise or takes a callback, there is no possible way to execute it synchronously. Hence, coloring. The reason this coloring isn't a problem for the JS ecosystem, is that it's a single-threaded language by design. So, async/callbacks are the only reasonable way to do anything external to the JS runtime (i.e. reading files, connecting to APIs, etc.) (notwithstanding that node.js introduced some synchronous external operations in its stdlib - those are mostly unused in practice.) To put it a different way - yes, JS has function coloring, but it's not a big deal because almost the entire JS ecosystem is colored red anyway.

unscaled 11mo ago

Let's revisit the original article[1]. It was not about arguments, but about the pain of writing callbacks and even async/await compared to writing the same code in Go. It had 5 well-defined claims about languages with colored functions: 1. Every function has a color. This is true for the new zig approach: functions that deal with IO are red, functions that do not need to deal with IO are blue. 2. The way you call a function depends on its color. This is also true for Zig: Red functions require an Io argument. Blue functions do not. Calling a red function means you need to have an Io argument. 3. You can only call a red function from within another red function. You cannot call a function that requires an Io object in Zig without having an Io in context. Yes, in theory you can use a global variable or initialize a new Io instance, but this is the same as the workarounds you can do for calling an async function from a non-async function For instance, in C# you can write 'Task.Run(() -> MyAsyncMethod()).Wait()'. 4. Red functions are more painful to call. This is true in Zig again, since you have to pass down an Io instance. You might say this is not a big nuisance and almost all functions require some argument or another... But by this measure, async/await is even less troublesome. Compare calling an async function in Javascript to an Io-colored function in Zig: function foo() { blueFunction(); // We don't add anything } async function bar() { await redFunction(); // We just add "await" } And in Zig: fn foo() void { blueFunction() } fn bar(io: Io) void { redFunction(io); // We just add "io". } Zig is more troublesome since you don't just add a fixed keyword: you need a add a variable that is passed along through somewhere. 5. Some core library functions are red. This is also true in Zig: Some core library functions require an Io instance. I'm not saying Zig has made the wrong choice here, but this is clearly not colorless I/O. And it's ok, since colorless I/O was always just hype. --- [1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

ghosty141 11mo ago
In my opinion you must have function coloring, it's impossible to do async (in the common sense) without it. If you break it down one function has a dependency on the async execution engine, the other one doesn't, and that alone colors them. Most languages just change the way that dependency is expressed and that can have impacts on the ergonomics.