5 ms·

Function coloring article conflated two things: one was legitimate limitation of JS unable to wait for async result from sync call, and the other was just autho

by pornel 3y ago

Function coloring article conflated two things: one was legitimate limitation of JS unable to wait for async result from sync call, and the other was just author's opinion how the syntax should look like. Rust's async doesn't have the limitation author described – you can spawn and block both (Tokio has some limits there, but that's Tokyo's choice, not language limitation), making "color" largely irrelevant. The second point was that both should have identical syntax, which Rust deliberately chose not to, because from Rusts perspective that would be too much implicit magic.

sph 3y ago
I disagree. You can call an async function in Rust from a regular one, that's true, but the returned value still needs to be passed through an executor to be useful. Async without function coloring means being able to call `async fn add(a, b: usize) -> usize` from anywhere and having an usize back, whether you're calling from a regular or async function. If you need slightly different logic because of async, you got function coloring. In fact, a hypothetical async without coloring functionality would not even need the `async` keyword at all, as all functions would be effectively the same, so you could choose to call one asynchronously or not with no particular ceremony. Is this even possible to do without compiler magic?
- creatonez 3y ago
  Thing is, it's a function coloring that mostly doesn't impede you by making certain operations impossible without massive refactoring. Regardless of an extra method call needed, it gets rid of the "you can only call a red function from within another red function" barrier by allowing you to do it, as long as you specify how (execute inside this thread, or delegate to a worker thread). This makes most of the issues associated with Javascript function coloring evaporate.
  - sph 3y ago
    Fair enough. It's not a deal-breaker kind of coloring, my issue is mostly conceptual. It's simply not elegant as a construct, but perhaps it doesn't have to be.
    - johnnyanmac 3y ago
      My experience with asynchronous programming is brief, but it taught me that you will never have elegant parallelism. It's a completely different problemset, you need a completely different solution to handle it. But the ideal API wants to treat the problems as one in the same. So the two camps I've seen on solving it is either a) blackbox it and set some rules for implementers, inevitably leaving to some hidden and very nasty bugs on the user end or bottlnecks for advanced users, or b) give implementers the full tools to setup, launch, and synchronize themselves, which may end up being slower than a single threaded solution if the user isn't adept already with parallel programming. You inevitably always have bits of a) creep in, even for the most explicit solutions
- ohgodplsno 3y ago
  Not knowing about function coloring is a stupid dream and has terrible implications on the performance of your code, which is infinitely more important than you losing 2 minutes having to figure out that you really want to `runBlocking { callThatBlocksForADamnLongTime() }`. The insight that function coloring gives you is terribly important. Roman Elizarov (of Jetbrains, author of Kotlin and lead on Kotlin Coroutines) agrees too: https://elizarov.medium.com/how-do-you-color-your-functions-a6bb423d936d https://elizarov.medium.com/how-do-you-color-your-functions-... async functions without coloring means that the only warning you'll ever get that `calculate2Plus2()` actually ends up running a distributed BigQuery and writing the end result to disk, printing it to stdout() and parsing that result to give it back to you is... hopefully, documentation is up to date and you read it? Async function coloring is not a problem. Async function coloring is a solution to "software developers are awful and will do awful things without any warning". If `calculate2Plus2` did not exhibit the write to disk behaviour in v1.0.0, but bumping to v1.0.1 does, I'd really want a warning that it does at least, and ideally a compiler error. The proper solution to function coloring is to have a pleasant API to interop between both worlds so that, at worst it's just a dozen characters more to say "yep, I really want to block here".
  - mike_hearn 3y ago
    Async doesn't solve that problem. What you're asking for is some sort of expression in the type system of expected performance, but there's no tech that can do this. Consider that modern NVMe SSDs can do disk reads faster than many calculations over the content of what was just read. A function that changes its algorithm to have worse time complexity won't be marked as async but could break your app, adding a single disk read would require marking as async but probably won't break your app (especially if it's in a non-hot path), adding 10,000 more reads where previously you had one doesn't require changing your async annotation but might break your app, etc. So the whole sync/async distinction doesn't really make much sense outside the context of single threaded UI toolkits, where indeed you have to read the docs anyway because doing some very CPU intensive work on the UI thread will block it even if no "blocking" APIs are ever invoked (and what is or is not blocking is somewhat arbitrary anyway in many cases).
    - 3y ago
- kaba0 3y ago
  Virtual threads (like java’s loom, erlang’s and goroutines) are arguably that. In case of java there really is no compiler magic even, “just” runtime magic.
  - pas 3y ago
    how big is the overhead for this? and how much easier is working with virtual threads than Rust's async?
    - winrid 3y ago
      1. Low enough that you don't need to worry for most web backend code and 2. Anything is easier than that. *XYZ is not Send* amiright? With Rust's async you have to worry about a different worker thread picking up the work after a context switch, which makes things complicated. Not so with Java.
- josephg 3y ago
  Its certainly possible with compiler magic. Zig does exactly that - where functions can be colorless and they work slightly differently in async and syncronous contexts, doing the obvious thing in both cases. I'm not sure how well it works in practice (I haven't spent enough time with zig to know). I'd worry that bugs would creep in to whichever variant of the function you aren't exercising. Its a lovely design though. https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/
  - svieira 3y ago
    _Unyielding_ describes the issues that arise when the author of the code doesn't have control of the transactionality / atomicity of their operations at the level they need to: https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html To my mind it's very much a balancing act between "low power to the developer, high power to the language" and "high power to the developer, low power to the language" all up and down the stack from software / hardware to "consumer / framework".
    - svieira 3y ago
      See also: https://www.tedinski.com/2018/11/06/concurrency-models.html https://www.tedinski.com/2018/11/06/concurrency-models.html which is the first article I've ever seen that says that `m:n`-style-green-threads are bad and good at the same time. TL;DR - he says that "doParallel" and "doConcurrently" are separate operations with distinct semantics that designers of programs must care about and that conflating the two (especially the common "doConcurrently-and-often-but-not-always-in-parallel") is one of the most common causes of bugs in programs that need to make progress in multiple threads of execution.
      - astrange 3y ago
        A common thing about writing like this article is that it confidently declares futures/promises are good for parallelism, but this is wrong - they are flawed because they introduce priority inversions. To schedule tasks properly, you need to know who is waiting on them as early as possible. With a promise, you only know when you get to the "wait()" call, which is too late. The correct solution is called structured concurrency.
- xmcqdpt2 3y ago
  You can do it using a runtime with scheduling and fibers. The issue is that you want to be able to save the entire thread stack cheaply so that you can switch to another task at an async yield point, and then go back to your previous stack frame. You want to do this without spawning system threads. Say you have an f -> g -> h call stack that blocks on IO. If all the functions are in fact async state machines (or some other kind of callbacks) your thread stack will look like this: Executor.loop() -> Executor.run(h) -> h.await() That is, your functions are really objects that are queued up and taken as needed by the executor. If h has a yield point, you can put it back on the queue, go back to the loop and pick up some other work. Later you can do the above again and await h to again up to the next yield point. Now consider the case where the middle function g is NOT async. In that case the call from f to g will be a normal function call that gets put on your call stack. In a language where this is allowed, you'll still have to wait or execute h and you'll get a call stack that looks like, Executor.loop() -> Executor.run(f) -> f.await() -> g() -> h.awaitUntilCompletion() (where our awaitUntilCompletion is minimally just calling h.await in a loop until h is finished). At this point, you are stuck: you can't yield from g() because it is a normal function with a normal stack frame, so your thread has to wait till g() finishes before it can be used for anything else. You are basically blocking a thread on h. At this point you might spawn a new thread to keep your thread pool count up (which is expensive and memory hungry) or just accept the forced synchronous blocking (which reduces throughput). If this happens over and over again, you either end up running out of memory or end up running code in a synchronous fashion but with worse performance (because of all the state machines). This is why functions are usually coloured, so that you don't get yourself in this accursed state by accident. AFAIK the alternative is to allocate all of your call stacks in the heap so that you can switch them in and out of kernel threads, which is what fibers are. This requires a complicated runtime with a scheduler such as in Loom, https://cr.openjdk.org/~rpressler/loom/Loom-Proposal.html https://cr.openjdk.org/~rpressler/loom/Loom-Proposal.html
- belmont_sup 3y ago
  Algebraic effects like how OCaml’s now implementing it may be possible! The ecosystem around effects in OCaml is still really young, but here’s an example of an http request being made that is asynchronous, non-blocking but looks synchronous with no special syntax. https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c99615cd1f58b99b02e3d0499/cohttp-eio/examples/client1.ml#L5 https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c... Performing these effects is similar to throwing exceptions up the callstack where whichever ancestor handles the IO work, then resumes the child with IO work done in hand.
  - mjmahone17 3y ago
    As someone who likes OCaml, but hasn’t touched OCaml in ~5 years, that’s a very hard example to read. I can tell it’s making a network request, but I have no clue: - what the type of res is (is it a string? A buffer? Async string? Something else?) - how this code does not block: what is happening in parallel? At which point will it block? I would like to be excited about OCaml’s algebraic effects, but right now I don’t really get it.
    - yawaramin 3y ago
      - It's some type from which you can read a response body. In OCaml we often work in terms of abstract types i.e. the operations which can be done on types. If you want to see the exact name of the type you can always look up the signature of `Client.get`: https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c99615cd1f58b99b02e3d0499/cohttp-eio/src/cohttp_eio.mli#L193 https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c... (nowadays LSP support in any good editor will show the type on hover) - It's implementing non-blocking I/O using effect handlers. The complexity is not exposed to the library user, which is actually the whole point. If you want to dive into the concurrency library (Eio) and study how it works, that's very doable, just like any concurrency library in any other language. There's really not that much to get–OCaml will do async I/O without having to colour functions, just like Go, Erlang/Elixir, and soon Java. It's not something sexy like monads that excite people with mysterious FP concepts, it's just a lot of hard work that went into simplifying how developers code at scale.
    - 3y ago
moth-fuzz 3y ago
You can't evaluate a future in normal rust as there's no default executor, you need to pull in some library to even make blocking calls to async functions. IMO, this is even worse than function coloring.
- zanellato19 3y ago
  You can also make your own executor. That's exactly how it was intended.
- weinzierl 3y ago
  It's a bit strange indeed. On the other hand you can't allocate either if you don't have an allocator. The difference is just that the allocator is opt-out while the reactor and executor are opt-in with no formal default.
- jarjoura 3y ago
  It's the same in C++. The compiler and language design came first and then the library (aka executor) part comes next. Rust has basically 2 executor libraries, tokio and async-std. It seems to me like tokio is solidifying as the executor of choice and it's only a matter of time before that design is baked into the std library.
  - menaerus 3y ago
    In C++ there is a default executor that is part of the standard library and has been so since C++11, e.g. no external library is needed - std::async. std::async is mostly "good enough" for ordinary cases where you only want to fire-and-forget and have no control over the underlying dispatching and scheduling algorithms. If you want something tweaked towards your use-case, and hence actual high-performance executor library, std::async is not gonna cut it. And that's where the "C++ executors" proposal come in.
    - BatmanAoD 3y ago
      That's...not precisely true. The C++ standard doesn't specify how std::async works, and for a while GCC just ran the operation sequentially, and later both GCC and Clang launched new OS threads by default. https://stackoverflow.com/q/10059239/1858225 https://stackoverflow.com/q/10059239/1858225
moron4hire 3y ago
I've been using C# and (Java|Type)Script pretty much since they were invented. Both use a function coloring async system. I don't know what it's like in Rust, but at least in the two examples with which I have experience with this apparently much-maligned system, I really don't get the complaints. Having to "color" functions really isn't that big of a deal.
z3t4 3y ago
JS has first class functions that you can pass around. The different colors debate was just a confusion - there where only one type of function, just that some functions took a function as an argument, althgough JS has no async functions. All async functions in JS comes from the runtime, like addEventListener for web DOM events. The "solution" to the non existing problem the committie came up with was to introduce 3 new types of function. So JS went from one type of function, to four types of functions.

by pornel 3y ago

sph 3y ago

I disagree. You can call an async function in Rust from a regular one, that's true, but the returned value still needs to be passed through an executor to be useful. Async without function coloring means being able to call `async fn add(a, b: usize) -> usize` from anywhere and having an usize back, whether you're calling from a regular or async function. If you need slightly different logic because of async, you got function coloring. In fact, a hypothetical async without coloring functionality would not even need the `async` keyword at all, as all functions would be effectively the same, so you could choose to call one asynchronously or not with no particular ceremony. Is this even possible to do without compiler magic?

creatonez 3y ago
Thing is, it's a function coloring that mostly doesn't impede you by making certain operations impossible without massive refactoring. Regardless of an extra method call needed, it gets rid of the "you can only call a red function from within another red function" barrier by allowing you to do it, as long as you specify how (execute inside this thread, or delegate to a worker thread). This makes most of the issues associated with Javascript function coloring evaporate.
- sph 3y ago
  Fair enough. It's not a deal-breaker kind of coloring, my issue is mostly conceptual. It's simply not elegant as a construct, but perhaps it doesn't have to be.
  - johnnyanmac 3y ago
    My experience with asynchronous programming is brief, but it taught me that you will never have elegant parallelism. It's a completely different problemset, you need a completely different solution to handle it. But the ideal API wants to treat the problems as one in the same. So the two camps I've seen on solving it is either a) blackbox it and set some rules for implementers, inevitably leaving to some hidden and very nasty bugs on the user end or bottlnecks for advanced users, or b) give implementers the full tools to setup, launch, and synchronize themselves, which may end up being slower than a single threaded solution if the user isn't adept already with parallel programming. You inevitably always have bits of a) creep in, even for the most explicit solutions
ohgodplsno 3y ago
Not knowing about function coloring is a stupid dream and has terrible implications on the performance of your code, which is infinitely more important than you losing 2 minutes having to figure out that you really want to `runBlocking { callThatBlocksForADamnLongTime() }`. The insight that function coloring gives you is terribly important. Roman Elizarov (of Jetbrains, author of Kotlin and lead on Kotlin Coroutines) agrees too: https://elizarov.medium.com/how-do-you-color-your-functions-a6bb423d936d https://elizarov.medium.com/how-do-you-color-your-functions-... async functions without coloring means that the only warning you'll ever get that `calculate2Plus2()` actually ends up running a distributed BigQuery and writing the end result to disk, printing it to stdout() and parsing that result to give it back to you is... hopefully, documentation is up to date and you read it? Async function coloring is not a problem. Async function coloring is a solution to "software developers are awful and will do awful things without any warning". If `calculate2Plus2` did not exhibit the write to disk behaviour in v1.0.0, but bumping to v1.0.1 does, I'd really want a warning that it does at least, and ideally a compiler error. The proper solution to function coloring is to have a pleasant API to interop between both worlds so that, at worst it's just a dozen characters more to say "yep, I really want to block here".
- mike_hearn 3y ago
  Async doesn't solve that problem. What you're asking for is some sort of expression in the type system of expected performance, but there's no tech that can do this. Consider that modern NVMe SSDs can do disk reads faster than many calculations over the content of what was just read. A function that changes its algorithm to have worse time complexity won't be marked as async but could break your app, adding a single disk read would require marking as async but probably won't break your app (especially if it's in a non-hot path), adding 10,000 more reads where previously you had one doesn't require changing your async annotation but might break your app, etc. So the whole sync/async distinction doesn't really make much sense outside the context of single threaded UI toolkits, where indeed you have to read the docs anyway because doing some very CPU intensive work on the UI thread will block it even if no "blocking" APIs are ever invoked (and what is or is not blocking is somewhat arbitrary anyway in many cases).
  - 3y ago
kaba0 3y ago
Virtual threads (like java’s loom, erlang’s and goroutines) are arguably that. In case of java there really is no compiler magic even, “just” runtime magic.
- pas 3y ago
  how big is the overhead for this? and how much easier is working with virtual threads than Rust's async?
  - winrid 3y ago
    1. Low enough that you don't need to worry for most web backend code and 2. Anything is easier than that. *XYZ is not Send* amiright? With Rust's async you have to worry about a different worker thread picking up the work after a context switch, which makes things complicated. Not so with Java.
josephg 3y ago
Its certainly possible with compiler magic. Zig does exactly that - where functions can be colorless and they work slightly differently in async and syncronous contexts, doing the obvious thing in both cases. I'm not sure how well it works in practice (I haven't spent enough time with zig to know). I'd worry that bugs would creep in to whichever variant of the function you aren't exercising. Its a lovely design though. https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/
- svieira 3y ago
  _Unyielding_ describes the issues that arise when the author of the code doesn't have control of the transactionality / atomicity of their operations at the level they need to: https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html To my mind it's very much a balancing act between "low power to the developer, high power to the language" and "high power to the developer, low power to the language" all up and down the stack from software / hardware to "consumer / framework".
  - svieira 3y ago
    See also: https://www.tedinski.com/2018/11/06/concurrency-models.html https://www.tedinski.com/2018/11/06/concurrency-models.html which is the first article I've ever seen that says that `m:n`-style-green-threads are bad and good at the same time. TL;DR - he says that "doParallel" and "doConcurrently" are separate operations with distinct semantics that designers of programs must care about and that conflating the two (especially the common "doConcurrently-and-often-but-not-always-in-parallel") is one of the most common causes of bugs in programs that need to make progress in multiple threads of execution.
    - astrange 3y ago
      A common thing about writing like this article is that it confidently declares futures/promises are good for parallelism, but this is wrong - they are flawed because they introduce priority inversions. To schedule tasks properly, you need to know who is waiting on them as early as possible. With a promise, you only know when you get to the "wait()" call, which is too late. The correct solution is called structured concurrency.
xmcqdpt2 3y ago
You can do it using a runtime with scheduling and fibers. The issue is that you want to be able to save the entire thread stack cheaply so that you can switch to another task at an async yield point, and then go back to your previous stack frame. You want to do this without spawning system threads. Say you have an f -> g -> h call stack that blocks on IO. If all the functions are in fact async state machines (or some other kind of callbacks) your thread stack will look like this: Executor.loop() -> Executor.run(h) -> h.await() That is, your functions are really objects that are queued up and taken as needed by the executor. If h has a yield point, you can put it back on the queue, go back to the loop and pick up some other work. Later you can do the above again and await h to again up to the next yield point. Now consider the case where the middle function g is NOT async. In that case the call from f to g will be a normal function call that gets put on your call stack. In a language where this is allowed, you'll still have to wait or execute h and you'll get a call stack that looks like, Executor.loop() -> Executor.run(f) -> f.await() -> g() -> h.awaitUntilCompletion() (where our awaitUntilCompletion is minimally just calling h.await in a loop until h is finished). At this point, you are stuck: you can't yield from g() because it is a normal function with a normal stack frame, so your thread has to wait till g() finishes before it can be used for anything else. You are basically blocking a thread on h. At this point you might spawn a new thread to keep your thread pool count up (which is expensive and memory hungry) or just accept the forced synchronous blocking (which reduces throughput). If this happens over and over again, you either end up running out of memory or end up running code in a synchronous fashion but with worse performance (because of all the state machines). This is why functions are usually coloured, so that you don't get yourself in this accursed state by accident. AFAIK the alternative is to allocate all of your call stacks in the heap so that you can switch them in and out of kernel threads, which is what fibers are. This requires a complicated runtime with a scheduler such as in Loom, https://cr.openjdk.org/~rpressler/loom/Loom-Proposal.html https://cr.openjdk.org/~rpressler/loom/Loom-Proposal.html
belmont_sup 3y ago
Algebraic effects like how OCaml’s now implementing it may be possible! The ecosystem around effects in OCaml is still really young, but here’s an example of an http request being made that is asynchronous, non-blocking but looks synchronous with no special syntax. https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c99615cd1f58b99b02e3d0499/cohttp-eio/examples/client1.ml#L5 https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c... Performing these effects is similar to throwing exceptions up the callstack where whichever ancestor handles the IO work, then resumes the child with IO work done in hand.
- mjmahone17 3y ago
  As someone who likes OCaml, but hasn’t touched OCaml in ~5 years, that’s a very hard example to read. I can tell it’s making a network request, but I have no clue: - what the type of res is (is it a string? A buffer? Async string? Something else?) - how this code does not block: what is happening in parallel? At which point will it block? I would like to be excited about OCaml’s algebraic effects, but right now I don’t really get it.
  - yawaramin 3y ago
    - It's some type from which you can read a response body. In OCaml we often work in terms of abstract types i.e. the operations which can be done on types. If you want to see the exact name of the type you can always look up the signature of `Client.get`: https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c99615cd1f58b99b02e3d0499/cohttp-eio/src/cohttp_eio.mli#L193 https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c... (nowadays LSP support in any good editor will show the type on hover) - It's implementing non-blocking I/O using effect handlers. The complexity is not exposed to the library user, which is actually the whole point. If you want to dive into the concurrency library (Eio) and study how it works, that's very doable, just like any concurrency library in any other language. There's really not that much to get–OCaml will do async I/O without having to colour functions, just like Go, Erlang/Elixir, and soon Java. It's not something sexy like monads that excite people with mysterious FP concepts, it's just a lot of hard work that went into simplifying how developers code at scale.
  - 3y ago

moth-fuzz 3y ago

You can't evaluate a future in normal rust as there's no default executor, you need to pull in some library to even make blocking calls to async functions. IMO, this is even worse than function coloring.

zanellato19 3y ago
You can also make your own executor. That's exactly how it was intended.
weinzierl 3y ago
It's a bit strange indeed. On the other hand you can't allocate either if you don't have an allocator. The difference is just that the allocator is opt-out while the reactor and executor are opt-in with no formal default.
jarjoura 3y ago
It's the same in C++. The compiler and language design came first and then the library (aka executor) part comes next. Rust has basically 2 executor libraries, tokio and async-std. It seems to me like tokio is solidifying as the executor of choice and it's only a matter of time before that design is baked into the std library.
- menaerus 3y ago
  In C++ there is a default executor that is part of the standard library and has been so since C++11, e.g. no external library is needed - std::async. std::async is mostly "good enough" for ordinary cases where you only want to fire-and-forget and have no control over the underlying dispatching and scheduling algorithms. If you want something tweaked towards your use-case, and hence actual high-performance executor library, std::async is not gonna cut it. And that's where the "C++ executors" proposal come in.
  - BatmanAoD 3y ago
    That's...not precisely true. The C++ standard doesn't specify how std::async works, and for a while GCC just ran the operation sequentially, and later both GCC and Clang launched new OS threads by default. https://stackoverflow.com/q/10059239/1858225 https://stackoverflow.com/q/10059239/1858225

moron4hire 3y ago

I've been using C# and (Java|Type)Script pretty much since they were invented. Both use a function coloring async system. I don't know what it's like in Rust, but at least in the two examples with which I have experience with this apparently much-maligned system, I really don't get the complaints. Having to "color" functions really isn't that big of a deal.

z3t4 3y ago

JS has first class functions that you can pass around. The different colors debate was just a confusion - there where only one type of function, just that some functions took a function as an argument, althgough JS has no async functions. All async functions in JS comes from the runtime, like addEventListener for web DOM events. The "solution" to the non existing problem the committie came up with was to introduce 3 new types of function. So JS went from one type of function, to four types of functions.