7 ms·

What Color is Your Function? (2015)

53 points by adunk 2y ago

omgbear 2y ago
I've thought about this a lot in relation to typescript over the years and had various opinions -- For some time I thought it'd be better if there was an implicit `await` on every line and require `void` or some other keyword to break execution like `go` in Golang. But, eventually I realized the difference in pre-emption between languages -- Go can (now) preempt your code in many places, so locks and thread-safety are very important. The javascript runtime only preempts at certain places, `await` being one. This means I can know no other code can be running without explicit locks around all critical sections. Finally understanding the trade-offs, I no longer am as frustrated when recoloring a bunch of functions. Instead, I can appreciate the areas where I'm not required to lock certain operations that I would in other languages.
- sakex 2y ago
  There is no preemption in Javascript. It is based on cooperative multitasking[1] (your await statements and non-blocking callback) which is the opposite of preemption. [1]https://en.wikipedia.org/wiki/Cooperative_multitasking#:~:text=Cooperative%20multitasking%2C%20also%20known%20as,running%20process%20to%20another%20process https://en.wikipedia.org/wiki/Cooperative_multitasking#:~:te....
  - ndndjdueej 2y ago
    As anyone who has done the old setTimeout(fn, 0) trick in the browser probably knows.
    - Feathercrown 2y ago
      They even created setImmediate(fn) for this purpose
  - orlp 2y ago
    If every line had an implicit await then it is indistinguishable from pre-emption, which I think is the point the person you're replying to is trying to make.
Rapzid 2y ago
I still haven't seen a good "solution" to this "problem" for imperative style programming. Why the quotes? Because the functions have different signatures, thus different functions(colors). And IT MATTERS. That's why you have structured concurrency hot on the heels of Loom. And once you are using structured concurrency constructs... Ehh, it's not the magic "solution" we were sold anymore is it?
- immibis 2y ago
  The article is not about functions having different type signatures but being somehow fundamentally different. It occurs every time we try to qualify and enforce restrictions on functions, whether it's async, pure, nonblocking, or whatever. Although they aren't all harder to call, they always cause problems with higher order functions. An idea of "effect generics" tries to improve upon this - you write something like template<effect T> T_function List filter(List list, T_function pred) (element type committed for brevity) and then filter is async if the predicate is async, etc - it's pure if the predicate is, nonblocking if the predicate is. If you continue in this direction, your language gets more complex than C++. The other options are to just accept that you need a separate filter function for async and non async predicates, or to do away with function flags (doesn't work for CPS style async of course).
  - Rapzid 2y ago
    This article is about exactly what I'm addressing. I stopped reading after your false premise.
- HiJon89 2y ago
  Not sure I follow. The function coloring problem is still solved in that scenario. You can apply structured concurrency to any function (there’s no colors), and any function can contain structured concurrency within it (ie, using structured concurrency doesn’t color your function)
  - Rapzid 2y ago
    You don't follow. Nothing against you in particular, but this is why we've had 10+ years of "color problem" articles and no real solutions. The problem isn't the colors, it's the ergonomics. The naive solution doesn't work because it turns out the difference between functions that return a value and functions that return a promise matters.. The other approaches destroy ergonomics.
immibis 2y ago
There is also coloured data: stack vs heap, threadsafe vs not, disk vs memory vs SQL, mutable vs immutable, etc ... Another comment: Python's async/await is precisely syntactic sugar for generators. It just changes the keywords and checks that you only use await in a function labeled async. Edit: my rate limit appears to have been increased.
wesselbindt 2y ago
I've always found the criticism leveled by the colored functions blog post a bit contrived. Yes, when you replace the words async/await with meaningless concepts I do not care about such as color, it's very annoying to have to arbitrarily mark a function as blue or red. But when you're honest with yourself and interpret the word "async" as "expensive", or as "does network calls", it becomes clear that "async/await" makes important features of your function explicit, rather than implicit. Seeing "await" gives me more information about the function, without having to read the body of the function (and the ones it calls, and the ones they call, etc). That's a good thing. There are serious drawbacks to async/await, and the red/blue blog post manages to list none of them. I've wrote a blog in response to OP containing a more detailed version of the above: https://wpbindt.github.io/async/opinions/programming/2024/01/21/red-code-blue-code-honesty.html https://wpbindt.github.io/async/opinions/programming/2024/01...
- klabb3 2y ago
  > But when you're honest with yourself and interpret the word "async" as "expensive", or as "does network calls", it becomes clear that "async/await" makes important features of your function explicit, rather than implicit. The main issue here is that whether or not a function is expensive is known upfront. In practice, this bites you when (a) you refactor (the impl changes), and (b) when you’re writing interfaces (the impl is unknown). This causes cumbersome cascading refactors and even complete loss of interop (in case where a 3p interface is sync and your impl is not). In order to argue against that point, you have to argue that async is upfront-predictable and obviously part of the interface. And that it should be considered a breaking change - or semantically different if you will.
  - shadowgovt 2y ago
    Key to note here is that asynchronicity impacts interface because it's an entirely different kind of function in a way that "regular" functions are not. Regular functions always return (or crash the program). Regular functions run to completion before returning control to the caller. These are two key properties async functions lack, so the caller has to care. It's analogous to the notion that checked exceptions are challenging because they also become part of the interface and so the caller is impacted by implementation details. (This does raise the question of whether there's a form of async hiding just out of sight of current language designs that's analogous to "unchecked exceptions": can one construct a model of function calling that makes one not have to care about async vs. sync functions at the cost of something else? I suspect, without exploring too deeply, that Orc (https://orc.csres.utexas.edu/ https://orc.csres.utexas.edu/) has gone that road).
vitiral 2y ago
This is one of the things I love most about Lua. You can coroutine.yield inside of any function, it's up the the caller to handle it correctly. This means I can write my tech stack to run in either mode, then swap out async/sync functions in the application. This is exactly what I do in https://Lua.civboot.org#Package_lap https://Lua.civboot.org#Package_lap
littlestymaar 2y ago
Ah my pet peeve reaching the front page again… The “coloring problem” has nothing to do with async at all, it's just about making effects explicit or implicit in the code. In Go or Rust, returning an error is also a “color” that spreads out to the top of the call stack. Same for checked exception in Java. Unchecked exceptions are like blocking functions, it's invisible so you don't have to manually annotate your functions but as a result any function can now fail without the programmer knowing it. JavaScript having async/await but unchecked exception may sound bit paradoxical in that regard, but every language has a combinations of both explicit and implicit effects: Rust has both async/await and explicit errors, but it also has panics (implicit errors) and implicit allocation for instance. I personally live explicit effects, but at the same time when you have too many of them (for example I'd like to have effects like “pure”, “allocating” and “panics”) they start to cause combinatorial explosion of cases in libraries accepting closures as parameter. Algebraic effects supposedly solve this particular problem but I'm not sure the additional conceptual learning effort is sustainable for a mainstream language, so in the meantime I think every language must keep its amount of explicit effects under a certain threshold.
- randomdata 2y ago
  > In Go or Rust, returning an error is also a “color” that spreads out to the top of the call stack. Not unless you're doing it wrong. Errors need to be handled. That may, in some cases, mean handling it with another error, but that isn't adding any "color". The new error is independent of any previous error state.
  - littlestymaar 2y ago
    You're misunderstanding: the fact that the new error is different from the original one doesn't change the fact that the function returns an error. Exactly like how async functions awaiting on a promise to create new promises doesn't change the fact that the function is an async one. Sure sometimes you can avoid propagating the error upward, but that's not the most common case and it also happens with async/await (if you don't need the result of the promise in the caller function you don't need to await and to make it a async function). In Rust the similarity is very flagrant because there are one postfix operators for both (? and .await).
fleabitdev 2y ago
I've spent some time thinking about language design recently. Coloured functions have a surprising number of downsides: - They're sometimes much too explicit. When writing a complicated generator, I don't necessarily want to annotate every call to a sub-generator with `yield*`, especially if I need to drill that annotation through wrapper functions which aren't really generators themselves. - Colours show up everywhere. If you have several functions with a `context: GodObject` parameter, then that's a function colour. Most third-party code will be unable to forward a `context` argument to a callback, so you'll have to manually smuggle it in using closures instead. - Different "colour channels" don't compose nicely with one another. Even though JavaScript ES2017 provided both async functions and generators, ES2018 had to add multiple new pieces of syntax to permit async generators. - It's normally impossible to write code which is generic over a function's colours. For example, if you have `function callTwice(f)` in JavaScript, you'd need a separate `async function callTwiceAsync(f)`, and a `function* iterateTwice(iterable)`, and an `async function* iterateTwiceAsync(iterable)`, despite the fact that all of those functions are doing the same thing. Several small languages are experimenting with algebraic effect systems [1], which would make all functions colourless. If JavaScript had this feature, the syntax for defining and calling functions would be the same, no matter whether you're dealing with a normal function, a generator, or an async function. No more `async`, `await`, `function*`, `yield`, `yield*`, `for-of`, `for await`... This can make everything too implicit and vague. The state of the art is for algebraic effect systems to be implemented in a statically-typed language, which then uses type inference to quietly tag all function types with their colours. This means that the compiler can enforce requirements like "you can only call generators within a scope which is prepared to collect their results", but that rule wouldn't prevent you from wrapping a generator in `callTwice`. [1]: https://github.com/ocaml-multicore/ocaml-effects-tutorial https://github.com/ocaml-multicore/ocaml-effects-tutorial
- magicalhippo 2y ago
  > Most third-party code will be unable to forward a `context` argument to a callback Having a context argument which is passed to the callback has been par for the course in all cases I've seen, but I've mostly been in C land and related areas. Though, if you have closures, should you still prefer a context param? > If JavaScript had this feature, the syntax for defining and calling functions would be the same, no matter whether you're dealing with a normal function, a generator, or an async function. How would one reason about performance and possibly concurrency in such languages?
pornel 2y ago
This article gets referenced a lot, but it does a poor job of defining what color actually is. The biggest issue the article describes — inability for sync code to wait for an async result — is limited mostly to JavaScript, and doesn't exist in most other languages that have async and a blocking wait (they can call red functions from blue functions). If this architectural hurdle is meant to be the color, then lots of languages with async functions don't have it. This reduces the coloring problem down to a minor issue of whether async needs to be used with a special syntax or not, and that's not such a big deal, and some may prefer async to be explicit anyway.
- enragedcacti 2y ago
  Python also has it. You can call async code with `asyncio.run` but it is extremely limited by the fact that nested event loops are prohibited. Any function that uses it becomes extremely brittle because it can't be called from async functions unlike all other synchronous code. This is for the most part an intentional design decision to "make sure that when the user is already in async code they don't call the sync form out of laziness." [1] [1] https://github.com/python/cpython/issues/66435#issuecomment-1531879319 https://github.com/python/cpython/issues/66435#issuecomment-...
from-nibly 2y ago
Wouldn't adding a way to block on an async function make javascript colorless? No one would want that though right? Like in what context would you even have a consequenceless block? A CLI maybe?
- shadowgovt 2y ago
  You got the meat of the situation in these four sentences. Yes, mitigating the color issue in JavaScript is as "trivial" as allowing the `await` keyword inside non-`async` functions. And what that would mean more-or-less makes sense: the runtime sleeps the active execution thread and resumes it if and when something else in the runtime satisfies the await. But then you can't know, as the caller, whether any function has "sometimes this function just never returns" semantics without inspecting the whole call chain of every function you call (an arguably impossible task), and that's worse. Especially because the dominant use-case for JavaScript is browsers, where just blocking the UI thread forever is extremely bad. The fact `await` can only show up in a function declared `async` is a feature, not a bug---it saves the developer from a type of hell where they can't know, in the general case, if a function never returns.
  - dragonwriter 2y ago
    > But then you can't know, as the caller, whether any function has "sometimes this function just never returns" semantics without inspecting the whole call chain of every function you call (an arguably impossible task), and that's worse. You have exactly as much guarantee of this with async functions as with sync ones, though, but if there is a reason to worry about an async function where it is being called from a sync function and you need to provide a better guarantee, Promise.race() it with a timeout and...problem solved.
withinboredom 2y ago
This is how we got "fibers" in PHP and they are absolutely worthless. With async/await, I can choose or choose not to wait for the result. Or I can trigger the work asynchronously and wait on it later, in an entirely different function. With Fibers, you cannot choose. You must wait, and you must wait now.
mattxxx 2y ago
I've spend a lot of time writing things using async in rust, python, and typescript, and I still find it un-intuitive / conceptually incorrect. Once you're in an async function the rules then fundamentally change for how everything operates; it's almost like programming within a dialect of the same language. In particular, I'm referring to everything from function calling, managing concurrency, waiting on results, sleeping threads. Comparatively, when you're in a go-block in go, you're still writing within the same dialect as outside of it.
- mattgreenrocks 2y ago
  It ultimately amounts to a dialect. It seems confusing because it is a well-supported dialect in each language, which makes you think it is first-class. The JVM's virtual threads approach is the right way. The runtime should be able to do everything needed to deal with async, and I, the developer, can write simple blocking-style code. Currently there are still issues around pinning when using the older concurrency APIs, but I'm hopeful they can push through them.
recursivedoubts 2y ago
My web scripting language, https://hyperscript.org https://hyperscript.org, tries to hide the difference between sync and async by resolving promises in the runtime. My theory is that this is something that web script writers should not be concerned with (this theory makes more sense when you consider hyperscript is a companion to htmx, and favors a particular approach to scripting[1].) on click fetch /whatever as json put the result's data into #some-div wait 2s put '' into #some-div You don't have to mark the script as sync or async, and the runtime will resolve everything for you, making everything feel like synchronous scripting. This obviously has limitations and foot guns, but it works reasonably well for light scripting. More info here: https://hyperscript.org/docs/#async https://hyperscript.org/docs/#async And the implementation of it in the runtime here: https://github.com/bigskysoftware/_hyperscript/blob/c81b07cec82af15f7088205467ec6b35f52faa74/src/_hyperscript.js#L1640 https://github.com/bigskysoftware/_hyperscript/blob/c81b07ce... https://github.com/bigskysoftware/_hyperscript/blob/c81b07cec82af15f7088205467ec6b35f52faa74/src/_hyperscript.js#L1705 https://github.com/bigskysoftware/_hyperscript/blob/c81b07ce... -- [1] - https://htmx.org/essays/hypermedia-friendly-scripting/ https://htmx.org/essays/hypermedia-friendly-scripting/
- paldepind2 2y ago
  I'm curious how you would you do something like the following? Is that just not supported? const promise1 = job1(); const promise2 = job2(); const [result1, result2] = Promise.all(promise1, promise2);
  - recursivedoubts 2y ago
    If you want job1 and job2 to be executed in parallel, you could do this: set result to [job1(), job2()] set result1 to the first result set result2 to the last result The array expression will wait until all promise values resolve. This isn't really what hyperscript is designed for, however: its async behavior is designed instead to take the async/sync distinction (and callback hell) out of "normal" DOM scripting. If you have sophisticated async needs then JavaScript is probably a better option. The good news is that hyperscript can call JavaScript functionality in the normal way, so that is an easy option if hyperscripts behavior isn't sufficient for your needs.
throwaway313373 2y ago
A slightly separate but still related question that bothers me every time I, a Python programmer, read about function coloring and different approaches to concurrency: Does anyone understand why asyncio-style approach to async won over gevent-style in Python? Why was the former approach accepted into to stdlib, got special syntax and is preferred by the community while the latter is a fringe niche thing?
- zahlman 2y ago
  The async keyword takes over from the asyncio standard library module; the relevant PEP for the former is https://peps.python.org/pep-0492/ https://peps.python.org/pep-0492/ and for the latter is https://peps.python.org/pep-3156/ https://peps.python.org/pep-3156/. That might give some hints.
  - throwaway313373 2y ago
    Unfortunately, 3156 doesn't really have an explanation why did Guido chose this particular implementation. It just says "yeah, that's what we gonna do". And 492 was written when tulip/asyncio was already chosen and included into stdlib.
- 0cf8612b2e1e 2y ago
  I hope there that was the result of some back room deals made in cigar filled rooms. gevent feels more pythonic in the amount of things that effortlessly worked.
dmvdoug 2y ago
Maybe Chomsky was on to something… Colorless green functions async/await furiously.
ks2048 2y ago
Do any relatively-popular languages avoid “two colors” by essentially making everything async?
- rererereferred 2y ago
  Zig was[0] going to make functions "colorblind" by not needing the function to be declared differently. Just a configuration in your root file[1] [0] I say was because I don't know the current status of async in zig or if it will come back in the future. [1] https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/
bazoom42 2y ago
It is similar to IO in Haskell which also force you to explicitly declare it in the signature (and in turn force callers to be IO), where most other languages (even functional) allow you to do IO anywhere.
jpc0 2y ago
I think the main issue with async/await specifically is that it is abstracted away from most developers. I may have a sync program running a main loop and I want to have a subset of that program using something like the reactor pattern. In any multithreading aware programming language I can do that by just sticking that in another thread and syncing however I want, even using coroutines I can do that, it doesn't need to follow the reactor pattern. However libraries (tokio) and languages (js) don't make that the obvious default. In something like JS it isn't even possible, the entire language is built on top of coroutines. For tokio the default is "put tokio main here and voila async" and you need to actually understand what it's doing under the hood to know that that isn't the behaviour you want in all cases. This is more bottom-up vs top-down learning. Most people learn and teach top-down because it gets you to productive really quickly. Bottom-up is much harder and has the chance of getting lost in the weeds but makes what is happening very obvious. Does tokio use reactor or coroutines? It depends...
dang 2y ago
Related. Others? Ruby methods are colorless - https://news.ycombinator.com/item?id=41001951 https://news.ycombinator.com/item?id=41001951 - July 2024 (235 comments) On 'function coloring' (2018) - https://news.ycombinator.com/item?id=36199499 https://news.ycombinator.com/item?id=36199499 - June 2023 (21 comments) I Believe Zig Has Function Colors - https://news.ycombinator.com/item?id=30965805 https://news.ycombinator.com/item?id=30965805 - April 2022 (158 comments) In Defense of Async: Function Colors Are Rusty - https://news.ycombinator.com/item?id=29793428 https://news.ycombinator.com/item?id=29793428 - Jan 2022 (101 comments) The Function Colour Myth, Or: async/await is not what you think it is - https://news.ycombinator.com/item?id=28904863 https://news.ycombinator.com/item?id=28904863 - Oct 2021 (7 comments) What color is your function? (2015) - https://news.ycombinator.com/item?id=28657358 https://news.ycombinator.com/item?id=28657358 - Sept 2021 (58 comments) What Color Is Your Function? (2015) - https://news.ycombinator.com/item?id=23218782 https://news.ycombinator.com/item?id=23218782 - May 2020 (85 comments) What Color is Your Function? (2015) - https://news.ycombinator.com/item?id=16732948 https://news.ycombinator.com/item?id=16732948 - April 2018 (45 comments) The Function Colour Myth (or async/await is not what you think it is) - https://news.ycombinator.com/item?id=12300441 https://news.ycombinator.com/item?id=12300441 - Aug 2016 (4 comments) What Color Is Your Function? - https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648 - Feb 2015 (146 comments)

53 points by adunk 2y ago

I've thought about this a lot in relation to typescript over the years and had various opinions -- For some time I thought it'd be better if there was an implicit `await` on every line and require `void` or some other keyword to break execution like `go` in Golang. But, eventually I realized the difference in pre-emption between languages -- Go can (now) preempt your code in many places, so locks and thread-safety are very important. The javascript runtime only preempts at certain places, `await` being one. This means I can know no other code can be running without explicit locks around all critical sections. Finally understanding the trade-offs, I no longer am as frustrated when recoloring a bunch of functions. Instead, I can appreciate the areas where I'm not required to lock certain operations that I would in other languages.

sakex 2y ago
There is no preemption in Javascript. It is based on cooperative multitasking[1] (your await statements and non-blocking callback) which is the opposite of preemption. [1]https://en.wikipedia.org/wiki/Cooperative_multitasking#:~:text=Cooperative%20multitasking%2C%20also%20known%20as,running%20process%20to%20another%20process https://en.wikipedia.org/wiki/Cooperative_multitasking#:~:te....
- ndndjdueej 2y ago
  As anyone who has done the old setTimeout(fn, 0) trick in the browser probably knows.
  - Feathercrown 2y ago
    They even created setImmediate(fn) for this purpose
- orlp 2y ago
  If every line had an implicit await then it is indistinguishable from pre-emption, which I think is the point the person you're replying to is trying to make.

Rapzid 2y ago

I still haven't seen a good "solution" to this "problem" for imperative style programming. Why the quotes? Because the functions have different signatures, thus different functions(colors). And IT MATTERS. That's why you have structured concurrency hot on the heels of Loom. And once you are using structured concurrency constructs... Ehh, it's not the magic "solution" we were sold anymore is it?

immibis 2y ago
The article is not about functions having different type signatures but being somehow fundamentally different. It occurs every time we try to qualify and enforce restrictions on functions, whether it's async, pure, nonblocking, or whatever. Although they aren't all harder to call, they always cause problems with higher order functions. An idea of "effect generics" tries to improve upon this - you write something like template<effect T> T_function List filter(List list, T_function pred) (element type committed for brevity) and then filter is async if the predicate is async, etc - it's pure if the predicate is, nonblocking if the predicate is. If you continue in this direction, your language gets more complex than C++. The other options are to just accept that you need a separate filter function for async and non async predicates, or to do away with function flags (doesn't work for CPS style async of course).
- Rapzid 2y ago
  This article is about exactly what I'm addressing. I stopped reading after your false premise.
HiJon89 2y ago
Not sure I follow. The function coloring problem is still solved in that scenario. You can apply structured concurrency to any function (there’s no colors), and any function can contain structured concurrency within it (ie, using structured concurrency doesn’t color your function)
- Rapzid 2y ago
  You don't follow. Nothing against you in particular, but this is why we've had 10+ years of "color problem" articles and no real solutions. The problem isn't the colors, it's the ergonomics. The naive solution doesn't work because it turns out the difference between functions that return a value and functions that return a promise matters.. The other approaches destroy ergonomics.

immibis 2y ago

There is also coloured data: stack vs heap, threadsafe vs not, disk vs memory vs SQL, mutable vs immutable, etc ... Another comment: Python's async/await is precisely syntactic sugar for generators. It just changes the keywords and checks that you only use await in a function labeled async. Edit: my rate limit appears to have been increased.

wesselbindt 2y ago

I've always found the criticism leveled by the colored functions blog post a bit contrived. Yes, when you replace the words async/await with meaningless concepts I do not care about such as color, it's very annoying to have to arbitrarily mark a function as blue or red. But when you're honest with yourself and interpret the word "async" as "expensive", or as "does network calls", it becomes clear that "async/await" makes important features of your function explicit, rather than implicit. Seeing "await" gives me more information about the function, without having to read the body of the function (and the ones it calls, and the ones they call, etc). That's a good thing. There are serious drawbacks to async/await, and the red/blue blog post manages to list none of them. I've wrote a blog in response to OP containing a more detailed version of the above: https://wpbindt.github.io/async/opinions/programming/2024/01/21/red-code-blue-code-honesty.html https://wpbindt.github.io/async/opinions/programming/2024/01...

klabb3 2y ago
> But when you're honest with yourself and interpret the word "async" as "expensive", or as "does network calls", it becomes clear that "async/await" makes important features of your function explicit, rather than implicit. The main issue here is that whether or not a function is expensive is known upfront. In practice, this bites you when (a) you refactor (the impl changes), and (b) when you’re writing interfaces (the impl is unknown). This causes cumbersome cascading refactors and even complete loss of interop (in case where a 3p interface is sync and your impl is not). In order to argue against that point, you have to argue that async is upfront-predictable and obviously part of the interface. And that it should be considered a breaking change - or semantically different if you will.
- shadowgovt 2y ago
  Key to note here is that asynchronicity impacts interface because it's an entirely different kind of function in a way that "regular" functions are not. Regular functions always return (or crash the program). Regular functions run to completion before returning control to the caller. These are two key properties async functions lack, so the caller has to care. It's analogous to the notion that checked exceptions are challenging because they also become part of the interface and so the caller is impacted by implementation details. (This does raise the question of whether there's a form of async hiding just out of sight of current language designs that's analogous to "unchecked exceptions": can one construct a model of function calling that makes one not have to care about async vs. sync functions at the cost of something else? I suspect, without exploring too deeply, that Orc (https://orc.csres.utexas.edu/ https://orc.csres.utexas.edu/) has gone that road).

vitiral 2y ago

This is one of the things I love most about Lua. You can coroutine.yield inside of any function, it's up the the caller to handle it correctly. This means I can write my tech stack to run in either mode, then swap out async/sync functions in the application. This is exactly what I do in https://Lua.civboot.org#Package_lap https://Lua.civboot.org#Package_lap

littlestymaar 2y ago

Ah my pet peeve reaching the front page again… The “coloring problem” has nothing to do with async at all, it's just about making effects explicit or implicit in the code. In Go or Rust, returning an error is also a “color” that spreads out to the top of the call stack. Same for checked exception in Java. Unchecked exceptions are like blocking functions, it's invisible so you don't have to manually annotate your functions but as a result any function can now fail without the programmer knowing it. JavaScript having async/await but unchecked exception may sound bit paradoxical in that regard, but every language has a combinations of both explicit and implicit effects: Rust has both async/await and explicit errors, but it also has panics (implicit errors) and implicit allocation for instance. I personally live explicit effects, but at the same time when you have too many of them (for example I'd like to have effects like “pure”, “allocating” and “panics”) they start to cause combinatorial explosion of cases in libraries accepting closures as parameter. Algebraic effects supposedly solve this particular problem but I'm not sure the additional conceptual learning effort is sustainable for a mainstream language, so in the meantime I think every language must keep its amount of explicit effects under a certain threshold.

randomdata 2y ago
> In Go or Rust, returning an error is also a “color” that spreads out to the top of the call stack. Not unless you're doing it wrong. Errors need to be handled. That may, in some cases, mean handling it with another error, but that isn't adding any "color". The new error is independent of any previous error state.
- littlestymaar 2y ago
  You're misunderstanding: the fact that the new error is different from the original one doesn't change the fact that the function returns an error. Exactly like how async functions awaiting on a promise to create new promises doesn't change the fact that the function is an async one. Sure sometimes you can avoid propagating the error upward, but that's not the most common case and it also happens with async/await (if you don't need the result of the promise in the caller function you don't need to await and to make it a async function). In Rust the similarity is very flagrant because there are one postfix operators for both (? and .await).

fleabitdev 2y ago

I've spent some time thinking about language design recently. Coloured functions have a surprising number of downsides: - They're sometimes much too explicit. When writing a complicated generator, I don't necessarily want to annotate every call to a sub-generator with `yield*`, especially if I need to drill that annotation through wrapper functions which aren't really generators themselves. - Colours show up everywhere. If you have several functions with a `context: GodObject` parameter, then that's a function colour. Most third-party code will be unable to forward a `context` argument to a callback, so you'll have to manually smuggle it in using closures instead. - Different "colour channels" don't compose nicely with one another. Even though JavaScript ES2017 provided both async functions and generators, ES2018 had to add multiple new pieces of syntax to permit async generators. - It's normally impossible to write code which is generic over a function's colours. For example, if you have `function callTwice(f)` in JavaScript, you'd need a separate `async function callTwiceAsync(f)`, and a `function* iterateTwice(iterable)`, and an `async function* iterateTwiceAsync(iterable)`, despite the fact that all of those functions are doing the same thing. Several small languages are experimenting with algebraic effect systems [1], which would make all functions colourless. If JavaScript had this feature, the syntax for defining and calling functions would be the same, no matter whether you're dealing with a normal function, a generator, or an async function. No more `async`, `await`, `function*`, `yield`, `yield*`, `for-of`, `for await`... This can make everything too implicit and vague. The state of the art is for algebraic effect systems to be implemented in a statically-typed language, which then uses type inference to quietly tag all function types with their colours. This means that the compiler can enforce requirements like "you can only call generators within a scope which is prepared to collect their results", but that rule wouldn't prevent you from wrapping a generator in `callTwice`. [1]: https://github.com/ocaml-multicore/ocaml-effects-tutorial https://github.com/ocaml-multicore/ocaml-effects-tutorial

magicalhippo 2y ago
> Most third-party code will be unable to forward a `context` argument to a callback Having a context argument which is passed to the callback has been par for the course in all cases I've seen, but I've mostly been in C land and related areas. Though, if you have closures, should you still prefer a context param? > If JavaScript had this feature, the syntax for defining and calling functions would be the same, no matter whether you're dealing with a normal function, a generator, or an async function. How would one reason about performance and possibly concurrency in such languages?

pornel 2y ago

This article gets referenced a lot, but it does a poor job of defining what color actually is. The biggest issue the article describes — inability for sync code to wait for an async result — is limited mostly to JavaScript, and doesn't exist in most other languages that have async and a blocking wait (they can call red functions from blue functions). If this architectural hurdle is meant to be the color, then lots of languages with async functions don't have it. This reduces the coloring problem down to a minor issue of whether async needs to be used with a special syntax or not, and that's not such a big deal, and some may prefer async to be explicit anyway.

enragedcacti 2y ago
Python also has it. You can call async code with `asyncio.run` but it is extremely limited by the fact that nested event loops are prohibited. Any function that uses it becomes extremely brittle because it can't be called from async functions unlike all other synchronous code. This is for the most part an intentional design decision to "make sure that when the user is already in async code they don't call the sync form out of laziness." [1] [1] https://github.com/python/cpython/issues/66435#issuecomment-1531879319 https://github.com/python/cpython/issues/66435#issuecomment-...

from-nibly 2y ago

Wouldn't adding a way to block on an async function make javascript colorless? No one would want that though right? Like in what context would you even have a consequenceless block? A CLI maybe?

shadowgovt 2y ago
You got the meat of the situation in these four sentences. Yes, mitigating the color issue in JavaScript is as "trivial" as allowing the `await` keyword inside non-`async` functions. And what that would mean more-or-less makes sense: the runtime sleeps the active execution thread and resumes it if and when something else in the runtime satisfies the await. But then you can't know, as the caller, whether any function has "sometimes this function just never returns" semantics without inspecting the whole call chain of every function you call (an arguably impossible task), and that's worse. Especially because the dominant use-case for JavaScript is browsers, where just blocking the UI thread forever is extremely bad. The fact `await` can only show up in a function declared `async` is a feature, not a bug---it saves the developer from a type of hell where they can't know, in the general case, if a function never returns.
- dragonwriter 2y ago
  > But then you can't know, as the caller, whether any function has "sometimes this function just never returns" semantics without inspecting the whole call chain of every function you call (an arguably impossible task), and that's worse. You have exactly as much guarantee of this with async functions as with sync ones, though, but if there is a reason to worry about an async function where it is being called from a sync function and you need to provide a better guarantee, Promise.race() it with a timeout and...problem solved.

withinboredom 2y ago

This is how we got "fibers" in PHP and they are absolutely worthless. With async/await, I can choose or choose not to wait for the result. Or I can trigger the work asynchronously and wait on it later, in an entirely different function. With Fibers, you cannot choose. You must wait, and you must wait now.

mattxxx 2y ago

I've spend a lot of time writing things using async in rust, python, and typescript, and I still find it un-intuitive / conceptually incorrect. Once you're in an async function the rules then fundamentally change for how everything operates; it's almost like programming within a dialect of the same language. In particular, I'm referring to everything from function calling, managing concurrency, waiting on results, sleeping threads. Comparatively, when you're in a go-block in go, you're still writing within the same dialect as outside of it.

mattgreenrocks 2y ago
It ultimately amounts to a dialect. It seems confusing because it is a well-supported dialect in each language, which makes you think it is first-class. The JVM's virtual threads approach is the right way. The runtime should be able to do everything needed to deal with async, and I, the developer, can write simple blocking-style code. Currently there are still issues around pinning when using the older concurrency APIs, but I'm hopeful they can push through them.

recursivedoubts 2y ago

My web scripting language, https://hyperscript.org https://hyperscript.org, tries to hide the difference between sync and async by resolving promises in the runtime. My theory is that this is something that web script writers should not be concerned with (this theory makes more sense when you consider hyperscript is a companion to htmx, and favors a particular approach to scripting[1].) on click fetch /whatever as json put the result's data into #some-div wait 2s put '' into #some-div You don't have to mark the script as sync or async, and the runtime will resolve everything for you, making everything feel like synchronous scripting. This obviously has limitations and foot guns, but it works reasonably well for light scripting. More info here: https://hyperscript.org/docs/#async https://hyperscript.org/docs/#async And the implementation of it in the runtime here: https://github.com/bigskysoftware/_hyperscript/blob/c81b07cec82af15f7088205467ec6b35f52faa74/src/_hyperscript.js#L1640 https://github.com/bigskysoftware/_hyperscript/blob/c81b07ce... https://github.com/bigskysoftware/_hyperscript/blob/c81b07cec82af15f7088205467ec6b35f52faa74/src/_hyperscript.js#L1705 https://github.com/bigskysoftware/_hyperscript/blob/c81b07ce... -- [1] - https://htmx.org/essays/hypermedia-friendly-scripting/ https://htmx.org/essays/hypermedia-friendly-scripting/

paldepind2 2y ago
I'm curious how you would you do something like the following? Is that just not supported? const promise1 = job1(); const promise2 = job2(); const [result1, result2] = Promise.all(promise1, promise2);
- recursivedoubts 2y ago
  If you want job1 and job2 to be executed in parallel, you could do this: set result to [job1(), job2()] set result1 to the first result set result2 to the last result The array expression will wait until all promise values resolve. This isn't really what hyperscript is designed for, however: its async behavior is designed instead to take the async/sync distinction (and callback hell) out of "normal" DOM scripting. If you have sophisticated async needs then JavaScript is probably a better option. The good news is that hyperscript can call JavaScript functionality in the normal way, so that is an easy option if hyperscripts behavior isn't sufficient for your needs.

throwaway313373 2y ago

A slightly separate but still related question that bothers me every time I, a Python programmer, read about function coloring and different approaches to concurrency: Does anyone understand why asyncio-style approach to async won over gevent-style in Python? Why was the former approach accepted into to stdlib, got special syntax and is preferred by the community while the latter is a fringe niche thing?

zahlman 2y ago
The async keyword takes over from the asyncio standard library module; the relevant PEP for the former is https://peps.python.org/pep-0492/ https://peps.python.org/pep-0492/ and for the latter is https://peps.python.org/pep-3156/ https://peps.python.org/pep-3156/. That might give some hints.
- throwaway313373 2y ago
  Unfortunately, 3156 doesn't really have an explanation why did Guido chose this particular implementation. It just says "yeah, that's what we gonna do". And 492 was written when tulip/asyncio was already chosen and included into stdlib.
0cf8612b2e1e 2y ago
I hope there that was the result of some back room deals made in cigar filled rooms. gevent feels more pythonic in the amount of things that effortlessly worked.

dmvdoug 2y ago

Maybe Chomsky was on to something… Colorless green functions async/await furiously.

ks2048 2y ago

Do any relatively-popular languages avoid “two colors” by essentially making everything async?

rererereferred 2y ago
Zig was[0] going to make functions "colorblind" by not needing the function to be declared differently. Just a configuration in your root file[1] [0] I say was because I don't know the current status of async in zig or if it will come back in the future. [1] https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/

bazoom42 2y ago

It is similar to IO in Haskell which also force you to explicitly declare it in the signature (and in turn force callers to be IO), where most other languages (even functional) allow you to do IO anywhere.

jpc0 2y ago

I think the main issue with async/await specifically is that it is abstracted away from most developers. I may have a sync program running a main loop and I want to have a subset of that program using something like the reactor pattern. In any multithreading aware programming language I can do that by just sticking that in another thread and syncing however I want, even using coroutines I can do that, it doesn't need to follow the reactor pattern. However libraries (tokio) and languages (js) don't make that the obvious default. In something like JS it isn't even possible, the entire language is built on top of coroutines. For tokio the default is "put tokio main here and voila async" and you need to actually understand what it's doing under the hood to know that that isn't the behaviour you want in all cases. This is more bottom-up vs top-down learning. Most people learn and teach top-down because it gets you to productive really quickly. Bottom-up is much harder and has the chance of getting lost in the weeds but makes what is happening very obvious. Does tokio use reactor or coroutines? It depends...

dang 2y ago

Related. Others? Ruby methods are colorless - https://news.ycombinator.com/item?id=41001951 https://news.ycombinator.com/item?id=41001951 - July 2024 (235 comments) On 'function coloring' (2018) - https://news.ycombinator.com/item?id=36199499 https://news.ycombinator.com/item?id=36199499 - June 2023 (21 comments) I Believe Zig Has Function Colors - https://news.ycombinator.com/item?id=30965805 https://news.ycombinator.com/item?id=30965805 - April 2022 (158 comments) In Defense of Async: Function Colors Are Rusty - https://news.ycombinator.com/item?id=29793428 https://news.ycombinator.com/item?id=29793428 - Jan 2022 (101 comments) The Function Colour Myth, Or: async/await is not what you think it is - https://news.ycombinator.com/item?id=28904863 https://news.ycombinator.com/item?id=28904863 - Oct 2021 (7 comments) What color is your function? (2015) - https://news.ycombinator.com/item?id=28657358 https://news.ycombinator.com/item?id=28657358 - Sept 2021 (58 comments) What Color Is Your Function? (2015) - https://news.ycombinator.com/item?id=23218782 https://news.ycombinator.com/item?id=23218782 - May 2020 (85 comments) What Color is Your Function? (2015) - https://news.ycombinator.com/item?id=16732948 https://news.ycombinator.com/item?id=16732948 - April 2018 (45 comments) The Function Colour Myth (or async/await is not what you think it is) - https://news.ycombinator.com/item?id=12300441 https://news.ycombinator.com/item?id=12300441 - Aug 2016 (4 comments) What Color Is Your Function? - https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648 - Feb 2015 (146 comments)