8 ms·

What color is your function? (2015)

129 points by xiaodai 5y ago

deleted 5y ago
[deleted]
tonetheman 5y ago
All my functions ... every one I have ever written in my entire life: the color ass or doo doo brown.
- dang 5y ago
  Can you please not post like this to HN? We're trying for higher-quality discussion here, to the extent possible. It's especially important not to post unsubstantive stuff to a fresh thread, because threads are super sensitive to initial conditions. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
dang 5y ago
Previous related threads: What Color Is Your Function? (2015) - https://news.ycombinator.com/item?id=23218782 https://news.ycombinator.com/item?id=23218782 - May 2020 (85 comments) What Color is Your Function? (2015) - https://news.ycombinator.com/item?id=16732948 https://news.ycombinator.com/item?id=16732948 - April 2018 (45 comments) What Color Is Your Function? - https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648 - Feb 2015 (143 comments) What Color Is Your Function? - https://news.ycombinator.com/item?id=8982494 https://news.ycombinator.com/item?id=8982494 - Feb 2015 (3 comments)
nvader 5y ago
Oh, I didn't realize it was a retro night at HN
jedisct1 5y ago
Zig’s functions are colorblind: https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/
- BiteCode_dev 5y ago
  That's an interesting design, it seems like the best of both worlds. Zip has really neat ideas. I wish it didn't go the "case as you want road" though.
  - messe 5y ago
    > I wish it didn't go the "case as you want road" though. That's almost inevitable given its desire for close interop with C (as in, the Zig compiler includes a C compiler and can directly #include C headers). The standard library has a relatively consistent case style though, which the Zig guide recommends.
- daurnimator 5y ago
  Indeed, partially thanks to this article! It was the key piece of material that helped me convince Andrew to make zig's async colour-less.
  - meowface 5y ago
    The internet is pretty cool.
- XorNot 5y ago
  This doesn't seem as useful as it implies though with the proviso statement that you can wind up with deadlocks in sync mode that you wouldn't have in async mode. Sounds like more of a footgun you can hand yourself.
- jolux 5y ago
  This is interesting, it seems like a global mode that kind of turns normal Zig code into Go or Erlang-like non-blocking code. It does mean you have to hold non-local context in your head while reading functions, but maybe that's a worthy tradeoff?
  - messe 5y ago
    > This is interesting, it seems like a global mode that kind of turns normal Zig code into Go or Erlang-like non-blocking code Nope! There's no language magic going on, and the global switch is just a global variable in the root module. It only affects a few functions in the standard library that check that variable. Zig relies heavily on compile time code executing for metaprogramming, and your code can read symbols from the root source file via @import("root"). All that pub const io_mode = .evented; does is set a variable that the standard library then checks. When io_mode is set to .evented, the library sets up an event loop and makes async calls internally. When it isn't, it makes blocking calls as you would expect. Unless you explicitly use 'async' to call functions, then they're going to act as if they block as they will be immediately await-ed upon, so there's no change.
quickthrower2 5y ago
You can call an async from a non async function of course. You just can’t return it’s result synchronously. Nitpick aside, I have got to the point where I care little! Every language has its boilerplatish things, be it Elm’s JSON parsing, GO’s “if (err != nil)” or Haskells myriad language extension declarations. I don’t think asyncs on a chain of functions, or converting a callback function into a promise based one is that bad.
- skybrian 5y ago
  The sticky part is when a callback returns a value and you can’t compute it without doing I/O.
- ddek 5y ago
  The issue is if you need to add an async call to an existing synchronous code base. If you introduce async in this situation, you’ll probably need to rewrite a huge chunk of your code to allow it. To prevent inconsistencies, this leads to a practice where programmers return promises or tasks by default, even in synchronous functions; which can cause a myriad of problems beyond simply looking really ugly.
  - dragonwriter 5y ago
    > The issue is if you need to add an async call to an existing synchronous code base. What it seems is needed is a blue function that wraps a red function with (in the simple case, though the actual inplementation needs to be more complicated to identify when thr sinple case applies vs when you have to do something more complicated) a dedicated single-item event loop and returns its result [0] (in async-await, I’d want to have a keyword “sync” used much like “await” but in an otherwise synchronous context to wrap a red function in this blue function.) JS—or at least browsers specifically—might avoid having this, or the tools to do it, to avoid encouraging use of async constructs in a sync context which could kill UI liveness, but for other red/blue async/sync environments it would be useful to have, whether in function or keyword form. [0] “async_to_sync" in this article is an example (for Python, but the general idea would be the same un any environment): https://www.aeracode.org/2018/02/19/python-async-simplified/ https://www.aeracode.org/2018/02/19/python-async-simplified/
isaacimagine 5y ago
In previous threads I haven't seen an in-depth discussion of algebraic effects as a solution to this problem, i.e. composing arbitrarily-many 'red' functions. Thoughts?
- skybrian 5y ago
  The danger is that declaring too much at the API boundary makes the problem worse. Declared effects result in more ways that API’s can be incompatible and fewer ways that an implementation can change without breaking compatibility.
- Ezku 5y ago
  I’m thinking your reference to composing red functions doesn’t seem congruent with your desire to discuss algebraic effects. Composing arbitrarily many red functions, or blue functions, is a solved problem. (I won’t mention the lingo knowing many in the audience to be allergic.) Composing functions with heterogenous, or completely arbitrary, colors is much less so. That’s what the article is about, too, is it not - combining blue and red? But I might well be missing something deeper here, perhaps you’ll enlighten the reader :) I would be really keen on seeing a mainstream-ish language that does algebraic effects or that can even implement them in a legitimate sense (discounting React’s implementation here, for instance). Purescript tried, but are doing something different these days. https://purescript-resources.readthedocs.io/en/latest/eff-to-effect.html https://purescript-resources.readthedocs.io/en/latest/eff-to...
  - isaacimagine 5y ago
    By 'red' I was referring to anything effectful, whether that be concurrency, IO, state, or otherwise. Composing any one effect is a solved problem (e.g. monad); composing arbitrarily many effects is a lot harder. Recent languages like Koka describe effects over the free monad; because they are described over a single monad, and this single monad composes, effects compose. I am by no means an expert in this area, but I hope this clarifies the intent of my original comment. What transformations and/or constraints exist for composable heterogeneous effects? Koka: http://koka-lang.org/ http://koka-lang.org/
JayRoni 5y ago
7 Simple Techniques To Improve Website Ranking https://seospecialistssrilanka.com/2021/09/24/website-ranking/ https://seospecialistssrilanka.com/2021/09/24/website-rankin...
superjan 5y ago
There can be very specific exceptions. In C# Winforms code I wrote, I have data loading code which I made async, called from UI code that was not. You need a running messageloop which was provided by a modal progress bar. Users could cancel the async code from the dialog. In the real world it is a quite a typical example where you want to call async code from non-async.
habibur 5y ago
Two years after this article was written, async-await was introduced in Javascript as a solution to this problem. As far as I understand.
- dragonwriter 5y ago
  Nope, async-await isn’t a solution to the problem, its just a slightly less annoying syntax sugar for defining and calling “red” functions, to which everything in the article applies.
- brazzy 5y ago
  The article specifically mentions the async/await feature of C# and explains how it does not solve the problem, just make it less ugly to deal with.
- hsn915 5y ago
  async/await in js exhibits the exact problem mentioned in the article.
  - BiteCode_dev 5y ago
    Not really, you can call any async function in JS in sync code. It is automatically scheduled, and it will return a result immediately. This result is a Promise, that you can pass sync callbacks to, and keep the whole code away from async/await. You can also decide that async/await is nice, and just use that instead. I'm a Python dev, but that part is better in JS.
vermilingua 5y ago
I might be showing my inexperience here, but my understanding of async/await and promises in JS is that they create threads under the hood to solve the problem the author is talking about. I'm afraid I just don't quite understand why async/await don't solve the problem and instead sweep it under the rug, could someone explain it?
- dkdbejwi383 5y ago
  If you want to use ‘await’, the calling scope needs to be an async function, which means any other function which calls the now async caller also needs to be async (well, if you need the resolved value. You can just orphan promises but then you’ll probably have unintended consequences). You’re forced to propagate it up the chain, which one may argue is a leaky abstraction.
- yuchi 5y ago
  Absolutely not. JS is single thread and async/await is more or less a syntax sugar around promise APIs.
- brazzy 5y ago
  > my understanding of async/await and promises in JS is that they create threads under the hood to solve the problem the author is talking about. No, that is wrong in multiple ways. Javascript is fundamentally designed around an event loop processed by a single thread. The only multitreading facility it has are web workers, which don't share memory with the main thread. > I'm afraid I just don't quite understand why async/await don't solve the problem and instead sweep it under the rug, could someone explain it? Both are really only syntactic sugar over callbacks. Under the hood, async/await splits the await-ing function and passes the second part as a callback to the await-ed function. And it requires the await-ing function itself to be async. Promises are just a nicer way to do callbacks. It's impossible to get a result from a promise (or an ansync function) synchronously. If there were a way, it would block the main thread and kill performance completely.
BiteCode_dev 5y ago
In Python at least, you can easily call an async function from a sync function: result = asyncio.run_until_complete(func) Also, having explicit async/await are a wait to tell you: here are the point in your code that have something doing unreliable side effects down there. You can then use this information to: - move that code and group it, so you get the rest of the code sansIO style - put that in a try/except, because this code WILL fail at some point - think about the perfs of this part of your code, which is what you are doing async io the first place I do wish that python coro would schedule themself automatically, like JS does, and return a future that you can await explicitly. But that ship as sailed.
- Joker_vD 5y ago
  You can do that in pretty much every language, it's just that in most of implementations, blocking on an async call can easily dead-lock, .NET is particularly notorious in this regard: you call GetResult(), the scheduler tries to schedule the task to the thread that have just called the GetResult() on it -- and this thread is now deadlocked, because it waits for itself. But since a threadpool has many threads, such degradation takes some time until the whole of the system deadlocks, and it's not even guaranteed to happen: the task scheduling in .NET is very convoluted.
  - corey_moncure 5y ago
    I've been burned by this so many times that I just simply will never use async in any code that is beyond trivial complexity and/or has performance requirements attached. I'll recommend breaking your solution into small pieces that execute separately and pass data to one another via messaging, to isolate the "colors" from one another at the process level.
    - omnicognate 5y ago
      I'm not a huge fan of async/await (though I have to write a fair amount of it) but that's throwing the baby out with the bathwater. The way to avoid "sync-over-async" deadlocks is to not do sync-over-async, i.e. don't call blocking methods/properties like Wait() and Result on tasks from threadpool threads - it's not what they're for (though they have valid, necessary uses and therefore can't simply be removed). I've never written code that's deadlocked in this way. I've fixed a ton of such deadlocks, though, and every one boils down to somebody not following this simple rule, and more generally not thinking clearly about what their multithreaded code will actually do when it runs. (Encouraging people to be sloppy in their thinking about these issues is one of the reasons I mildly dislike C#'s concurrency facilities - but nonetheless it's perfectly possible to write code with them that isn't riddled with deadlocks.)
rlass 5y ago
Async implementations like Python's have roughly the same speed as Bernstein's tcpserver, which forks a process for every connection and as such has perfect isolation between connections. Progress, I guess.
- jayd16 5y ago
  How does the memory usage compare?
jillesvangurp 5y ago
Kotlin's co-routines are pretty nice. Technically it uses colored functions but because it is statically compiled language, there is no chance of doing this wrong as that would simply be a compile error. The way suspend functions work in Kotlin is that it is essentially syntactic sugar for async await where calling a suspend function implicitly awaits it; which you can only do in a so-called CoroutineScope (for example another suspend function). It's actually very similar to what go does. Instead of go foo(), Kotlin makes you write something like CoroutineScope(CoroutineName("myscopename")).launch { foo() }. It's actually very similar to what go does as well. Instead of go foo(), Kotlin makes you write something like CoroutineScope(CoroutineName("myscopename")).launch { foo() } The difference is that a coroutine scope is a thing with a name and some behavior. For example, you can control how the coroutine is dispatched for example. Some dispatchers might do everything on the main thread. Other dispatchers use thread pools. Very useful if you are integrating some blocking IO framework where you don't want to block the main thread. There's more to it but it's a pretty well designed solution for what is a tough problem space.
TheMode 5y ago
I remember this post being referenced by Brian Goetz, which ultimately leaded to the development of project Loom in the JVM (green threads handled by the runtime)
- vips7L 5y ago
  Can’t wait til we get it!
jasode 5y ago
The different philosophies of "explicitly annotate async functions" vs "implicit async" vs "create os threads with polling" etc will always continue because they all have tradeoffs. So, the "simplification" or a "unification" of a concurrency model always comes at a cost that some programmers don't want to pay. Whenever you see, "Language <X> has solved the color problem so it's not an issue." -- the reflexive skepticism should be, "At what price, and do I want to pay it?" I think it's instructive that Rust had a clean-sheet start to design a new language and so far, they've rejected Go style "channels" as Rust core syntax and instead, adopted the async/await paradigm. You don't have to program in Rust to get the gist of the following previous comments about their thought process: https://news.ycombinator.com/item?id=21475154 https://news.ycombinator.com/item?id=21475154 https://lobste.rs/s/bfsxsl/ocaml_4_03_will_if_all_goes_well_support#c_whrcmk https://lobste.rs/s/bfsxsl/ocaml_4_03_will_if_all_goes_well_... https://lobste.rs/s/y3fsrm/what_is_zig_s_colorblind_async_await#c_wsg7e0 https://lobste.rs/s/y3fsrm/what_is_zig_s_colorblind_async_aw...
- gpsx 5y ago
  There are definitely tradeoffs. In javascript I can cavalierly code knowing that a sequence of lines will not be interrupted by another thread. (Unfortunately, I still have some mental overhead because I keep thinking, "This program will be screwed if they ever introduce multi-threading".)
stevebmark 5y ago
I dislike all of Ruby's magic, metaprogramming, and that it's almost impossible to statically analyze, however I have come to appreciate how nice it is in Ruby not to have functions with 'color' like this article mentions. I certainly hope TC39 can find a way to bake in similar async-agnostic syntax to Javascript. I'm not sure if it's possible with syntax alone, or if it would require a new language.
- brasic 5y ago
  Consider yourself fortunate to have (forgive the assumption!) never worked in a ruby codebase that uses eventmachine or promise.rb. It’s less common than in some other languages but ruby is perfectly capable of a callback/promise-based model for performing async io or query batching, for example. And it creates exactly the “function color” problem that this article describes.
nonbirithm 5y ago
I wish there were a statically typed language with Lua's stackful coroutines and the ability to manually yield. It seems that Lua and Ruby are the only two well-known languages with that kind of stackful coroutine. And also, LuaJIT has proven that they can be implemented in a performant way, so I'm not sure if performance is a valid obstacle to implementing them more often. Even languages like Go and Java with proper fibers don't allow you to yield where you want to, and the yield points are inserted by the compiler/runtime instead. A use case which I can't seem to reimplement without stackful coroutines is "inline" blocking input, as found in the source code of old roguelikes. It makes the game logic easier to follow since you don't have to rearchitect the entire engine to be event-based, and can instead yield between an "update" and "draw" coroutine each frame. msg("Do you want your possessions identified?) local yes = yes_or_no() if yes then identify_possessions() end I was honestly very surprised when I first realized that this is not how most graphical turn-based games are programmed.
- gpderetta 5y ago
  I'm a huge fan of lua-like coroutines. Even though I have never written a line of lua I read all their papers on the topic. Among the statically compiled languages, there are a few options in C++ for stackful coroutines, for example boost.context. There is still a chance that they might make it into the standard at some point.
still_grokking 5y ago
Scala got a really nice solution to this whole problem area lately. Its new "cps-async" implementation has "auto coloring" as a highlight feature. The Scala compile will just do the right thing in an async context. Details can be found here: https://rssh.github.io/dotty-cps-async/ https://rssh.github.io/dotty-cps-async/ One of the more interesting parts is also that "cps-async" works fine with arbitrary effect monads. There are integrations with Cats' IO and ZIO and some more.
nerdponx 5y ago
One thing I don't fully understand is how async-without-yield is different from the "bad" green threads as described in e.g. https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html Are these two fundamentally opposing viewpoints? Or can these two sets of ideas be reconciled somehow? The article that I posted specifically calls out Java as an example of why you need explicit await, yet the article in the OP praises Java for not having it!

129 points by xiaodai 5y ago

deleted 5y ago

[deleted]

tonetheman 5y ago

All my functions ... every one I have ever written in my entire life: the color ass or doo doo brown.

dang 5y ago
Can you please not post like this to HN? We're trying for higher-quality discussion here, to the extent possible. It's especially important not to post unsubstantive stuff to a fresh thread, because threads are super sensitive to initial conditions. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

dang 5y ago

Previous related threads: What Color Is Your Function? (2015) - https://news.ycombinator.com/item?id=23218782 https://news.ycombinator.com/item?id=23218782 - May 2020 (85 comments) What Color is Your Function? (2015) - https://news.ycombinator.com/item?id=16732948 https://news.ycombinator.com/item?id=16732948 - April 2018 (45 comments) What Color Is Your Function? - https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648 - Feb 2015 (143 comments) What Color Is Your Function? - https://news.ycombinator.com/item?id=8982494 https://news.ycombinator.com/item?id=8982494 - Feb 2015 (3 comments)

nvader 5y ago

Oh, I didn't realize it was a retro night at HN

jedisct1 5y ago

Zig’s functions are colorblind: https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/

BiteCode_dev 5y ago
That's an interesting design, it seems like the best of both worlds. Zip has really neat ideas. I wish it didn't go the "case as you want road" though.
- messe 5y ago
  > I wish it didn't go the "case as you want road" though. That's almost inevitable given its desire for close interop with C (as in, the Zig compiler includes a C compiler and can directly #include C headers). The standard library has a relatively consistent case style though, which the Zig guide recommends.
daurnimator 5y ago
Indeed, partially thanks to this article! It was the key piece of material that helped me convince Andrew to make zig's async colour-less.
- meowface 5y ago
  The internet is pretty cool.
XorNot 5y ago
This doesn't seem as useful as it implies though with the proviso statement that you can wind up with deadlocks in sync mode that you wouldn't have in async mode. Sounds like more of a footgun you can hand yourself.
jolux 5y ago
This is interesting, it seems like a global mode that kind of turns normal Zig code into Go or Erlang-like non-blocking code. It does mean you have to hold non-local context in your head while reading functions, but maybe that's a worthy tradeoff?
- messe 5y ago
  > This is interesting, it seems like a global mode that kind of turns normal Zig code into Go or Erlang-like non-blocking code Nope! There's no language magic going on, and the global switch is just a global variable in the root module. It only affects a few functions in the standard library that check that variable. Zig relies heavily on compile time code executing for metaprogramming, and your code can read symbols from the root source file via @import("root"). All that pub const io_mode = .evented; does is set a variable that the standard library then checks. When io_mode is set to .evented, the library sets up an event loop and makes async calls internally. When it isn't, it makes blocking calls as you would expect. Unless you explicitly use 'async' to call functions, then they're going to act as if they block as they will be immediately await-ed upon, so there's no change.

quickthrower2 5y ago

You can call an async from a non async function of course. You just can’t return it’s result synchronously. Nitpick aside, I have got to the point where I care little! Every language has its boilerplatish things, be it Elm’s JSON parsing, GO’s “if (err != nil)” or Haskells myriad language extension declarations. I don’t think asyncs on a chain of functions, or converting a callback function into a promise based one is that bad.

skybrian 5y ago
The sticky part is when a callback returns a value and you can’t compute it without doing I/O.
ddek 5y ago
The issue is if you need to add an async call to an existing synchronous code base. If you introduce async in this situation, you’ll probably need to rewrite a huge chunk of your code to allow it. To prevent inconsistencies, this leads to a practice where programmers return promises or tasks by default, even in synchronous functions; which can cause a myriad of problems beyond simply looking really ugly.
- dragonwriter 5y ago
  > The issue is if you need to add an async call to an existing synchronous code base. What it seems is needed is a blue function that wraps a red function with (in the simple case, though the actual inplementation needs to be more complicated to identify when thr sinple case applies vs when you have to do something more complicated) a dedicated single-item event loop and returns its result [0] (in async-await, I’d want to have a keyword “sync” used much like “await” but in an otherwise synchronous context to wrap a red function in this blue function.) JS—or at least browsers specifically—might avoid having this, or the tools to do it, to avoid encouraging use of async constructs in a sync context which could kill UI liveness, but for other red/blue async/sync environments it would be useful to have, whether in function or keyword form. [0] “async_to_sync" in this article is an example (for Python, but the general idea would be the same un any environment): https://www.aeracode.org/2018/02/19/python-async-simplified/ https://www.aeracode.org/2018/02/19/python-async-simplified/

isaacimagine 5y ago

In previous threads I haven't seen an in-depth discussion of algebraic effects as a solution to this problem, i.e. composing arbitrarily-many 'red' functions. Thoughts?

skybrian 5y ago
The danger is that declaring too much at the API boundary makes the problem worse. Declared effects result in more ways that API’s can be incompatible and fewer ways that an implementation can change without breaking compatibility.
Ezku 5y ago
I’m thinking your reference to composing red functions doesn’t seem congruent with your desire to discuss algebraic effects. Composing arbitrarily many red functions, or blue functions, is a solved problem. (I won’t mention the lingo knowing many in the audience to be allergic.) Composing functions with heterogenous, or completely arbitrary, colors is much less so. That’s what the article is about, too, is it not - combining blue and red? But I might well be missing something deeper here, perhaps you’ll enlighten the reader :) I would be really keen on seeing a mainstream-ish language that does algebraic effects or that can even implement them in a legitimate sense (discounting React’s implementation here, for instance). Purescript tried, but are doing something different these days. https://purescript-resources.readthedocs.io/en/latest/eff-to-effect.html https://purescript-resources.readthedocs.io/en/latest/eff-to...
- isaacimagine 5y ago
  By 'red' I was referring to anything effectful, whether that be concurrency, IO, state, or otherwise. Composing any one effect is a solved problem (e.g. monad); composing arbitrarily many effects is a lot harder. Recent languages like Koka describe effects over the free monad; because they are described over a single monad, and this single monad composes, effects compose. I am by no means an expert in this area, but I hope this clarifies the intent of my original comment. What transformations and/or constraints exist for composable heterogeneous effects? Koka: http://koka-lang.org/ http://koka-lang.org/

JayRoni 5y ago

7 Simple Techniques To Improve Website Ranking https://seospecialistssrilanka.com/2021/09/24/website-ranking/ https://seospecialistssrilanka.com/2021/09/24/website-rankin...

superjan 5y ago

There can be very specific exceptions. In C# Winforms code I wrote, I have data loading code which I made async, called from UI code that was not. You need a running messageloop which was provided by a modal progress bar. Users could cancel the async code from the dialog. In the real world it is a quite a typical example where you want to call async code from non-async.

habibur 5y ago

Two years after this article was written, async-await was introduced in Javascript as a solution to this problem. As far as I understand.

dragonwriter 5y ago
Nope, async-await isn’t a solution to the problem, its just a slightly less annoying syntax sugar for defining and calling “red” functions, to which everything in the article applies.
brazzy 5y ago
The article specifically mentions the async/await feature of C# and explains how it does not solve the problem, just make it less ugly to deal with.
hsn915 5y ago
async/await in js exhibits the exact problem mentioned in the article.
- BiteCode_dev 5y ago
  Not really, you can call any async function in JS in sync code. It is automatically scheduled, and it will return a result immediately. This result is a Promise, that you can pass sync callbacks to, and keep the whole code away from async/await. You can also decide that async/await is nice, and just use that instead. I'm a Python dev, but that part is better in JS.

vermilingua 5y ago

I might be showing my inexperience here, but my understanding of async/await and promises in JS is that they create threads under the hood to solve the problem the author is talking about. I'm afraid I just don't quite understand why async/await don't solve the problem and instead sweep it under the rug, could someone explain it?

dkdbejwi383 5y ago
If you want to use ‘await’, the calling scope needs to be an async function, which means any other function which calls the now async caller also needs to be async (well, if you need the resolved value. You can just orphan promises but then you’ll probably have unintended consequences). You’re forced to propagate it up the chain, which one may argue is a leaky abstraction.
yuchi 5y ago
Absolutely not. JS is single thread and async/await is more or less a syntax sugar around promise APIs.
brazzy 5y ago
> my understanding of async/await and promises in JS is that they create threads under the hood to solve the problem the author is talking about. No, that is wrong in multiple ways. Javascript is fundamentally designed around an event loop processed by a single thread. The only multitreading facility it has are web workers, which don't share memory with the main thread. > I'm afraid I just don't quite understand why async/await don't solve the problem and instead sweep it under the rug, could someone explain it? Both are really only syntactic sugar over callbacks. Under the hood, async/await splits the await-ing function and passes the second part as a callback to the await-ed function. And it requires the await-ing function itself to be async. Promises are just a nicer way to do callbacks. It's impossible to get a result from a promise (or an ansync function) synchronously. If there were a way, it would block the main thread and kill performance completely.

BiteCode_dev 5y ago

In Python at least, you can easily call an async function from a sync function: result = asyncio.run_until_complete(func) Also, having explicit async/await are a wait to tell you: here are the point in your code that have something doing unreliable side effects down there. You can then use this information to: - move that code and group it, so you get the rest of the code sansIO style - put that in a try/except, because this code WILL fail at some point - think about the perfs of this part of your code, which is what you are doing async io the first place I do wish that python coro would schedule themself automatically, like JS does, and return a future that you can await explicitly. But that ship as sailed.

Joker_vD 5y ago
You can do that in pretty much every language, it's just that in most of implementations, blocking on an async call can easily dead-lock, .NET is particularly notorious in this regard: you call GetResult(), the scheduler tries to schedule the task to the thread that have just called the GetResult() on it -- and this thread is now deadlocked, because it waits for itself. But since a threadpool has many threads, such degradation takes some time until the whole of the system deadlocks, and it's not even guaranteed to happen: the task scheduling in .NET is very convoluted.
- corey_moncure 5y ago
  I've been burned by this so many times that I just simply will never use async in any code that is beyond trivial complexity and/or has performance requirements attached. I'll recommend breaking your solution into small pieces that execute separately and pass data to one another via messaging, to isolate the "colors" from one another at the process level.
  - omnicognate 5y ago
    I'm not a huge fan of async/await (though I have to write a fair amount of it) but that's throwing the baby out with the bathwater. The way to avoid "sync-over-async" deadlocks is to not do sync-over-async, i.e. don't call blocking methods/properties like Wait() and Result on tasks from threadpool threads - it's not what they're for (though they have valid, necessary uses and therefore can't simply be removed). I've never written code that's deadlocked in this way. I've fixed a ton of such deadlocks, though, and every one boils down to somebody not following this simple rule, and more generally not thinking clearly about what their multithreaded code will actually do when it runs. (Encouraging people to be sloppy in their thinking about these issues is one of the reasons I mildly dislike C#'s concurrency facilities - but nonetheless it's perfectly possible to write code with them that isn't riddled with deadlocks.)

rlass 5y ago

Async implementations like Python's have roughly the same speed as Bernstein's tcpserver, which forks a process for every connection and as such has perfect isolation between connections. Progress, I guess.

jayd16 5y ago
How does the memory usage compare?

jillesvangurp 5y ago

Kotlin's co-routines are pretty nice. Technically it uses colored functions but because it is statically compiled language, there is no chance of doing this wrong as that would simply be a compile error. The way suspend functions work in Kotlin is that it is essentially syntactic sugar for async await where calling a suspend function implicitly awaits it; which you can only do in a so-called CoroutineScope (for example another suspend function). It's actually very similar to what go does. Instead of go foo(), Kotlin makes you write something like CoroutineScope(CoroutineName("myscopename")).launch { foo() }. It's actually very similar to what go does as well. Instead of go foo(), Kotlin makes you write something like CoroutineScope(CoroutineName("myscopename")).launch { foo() } The difference is that a coroutine scope is a thing with a name and some behavior. For example, you can control how the coroutine is dispatched for example. Some dispatchers might do everything on the main thread. Other dispatchers use thread pools. Very useful if you are integrating some blocking IO framework where you don't want to block the main thread. There's more to it but it's a pretty well designed solution for what is a tough problem space.

TheMode 5y ago

I remember this post being referenced by Brian Goetz, which ultimately leaded to the development of project Loom in the JVM (green threads handled by the runtime)

vips7L 5y ago
Can’t wait til we get it!

jasode 5y ago

The different philosophies of "explicitly annotate async functions" vs "implicit async" vs "create os threads with polling" etc will always continue because they all have tradeoffs. So, the "simplification" or a "unification" of a concurrency model always comes at a cost that some programmers don't want to pay. Whenever you see, "Language <X> has solved the color problem so it's not an issue." -- the reflexive skepticism should be, "At what price, and do I want to pay it?" I think it's instructive that Rust had a clean-sheet start to design a new language and so far, they've rejected Go style "channels" as Rust core syntax and instead, adopted the async/await paradigm. You don't have to program in Rust to get the gist of the following previous comments about their thought process: https://news.ycombinator.com/item?id=21475154 https://news.ycombinator.com/item?id=21475154 https://lobste.rs/s/bfsxsl/ocaml_4_03_will_if_all_goes_well_support#c_whrcmk https://lobste.rs/s/bfsxsl/ocaml_4_03_will_if_all_goes_well_... https://lobste.rs/s/y3fsrm/what_is_zig_s_colorblind_async_await#c_wsg7e0 https://lobste.rs/s/y3fsrm/what_is_zig_s_colorblind_async_aw...

gpsx 5y ago
There are definitely tradeoffs. In javascript I can cavalierly code knowing that a sequence of lines will not be interrupted by another thread. (Unfortunately, I still have some mental overhead because I keep thinking, "This program will be screwed if they ever introduce multi-threading".)

stevebmark 5y ago

I dislike all of Ruby's magic, metaprogramming, and that it's almost impossible to statically analyze, however I have come to appreciate how nice it is in Ruby not to have functions with 'color' like this article mentions. I certainly hope TC39 can find a way to bake in similar async-agnostic syntax to Javascript. I'm not sure if it's possible with syntax alone, or if it would require a new language.

brasic 5y ago
Consider yourself fortunate to have (forgive the assumption!) never worked in a ruby codebase that uses eventmachine or promise.rb. It’s less common than in some other languages but ruby is perfectly capable of a callback/promise-based model for performing async io or query batching, for example. And it creates exactly the “function color” problem that this article describes.

nonbirithm 5y ago

I wish there were a statically typed language with Lua's stackful coroutines and the ability to manually yield. It seems that Lua and Ruby are the only two well-known languages with that kind of stackful coroutine. And also, LuaJIT has proven that they can be implemented in a performant way, so I'm not sure if performance is a valid obstacle to implementing them more often. Even languages like Go and Java with proper fibers don't allow you to yield where you want to, and the yield points are inserted by the compiler/runtime instead. A use case which I can't seem to reimplement without stackful coroutines is "inline" blocking input, as found in the source code of old roguelikes. It makes the game logic easier to follow since you don't have to rearchitect the entire engine to be event-based, and can instead yield between an "update" and "draw" coroutine each frame. msg("Do you want your possessions identified?) local yes = yes_or_no() if yes then identify_possessions() end I was honestly very surprised when I first realized that this is not how most graphical turn-based games are programmed.

gpderetta 5y ago
I'm a huge fan of lua-like coroutines. Even though I have never written a line of lua I read all their papers on the topic. Among the statically compiled languages, there are a few options in C++ for stackful coroutines, for example boost.context. There is still a chance that they might make it into the standard at some point.

still_grokking 5y ago

Scala got a really nice solution to this whole problem area lately. Its new "cps-async" implementation has "auto coloring" as a highlight feature. The Scala compile will just do the right thing in an async context. Details can be found here: https://rssh.github.io/dotty-cps-async/ https://rssh.github.io/dotty-cps-async/ One of the more interesting parts is also that "cps-async" works fine with arbitrary effect monads. There are integrations with Cats' IO and ZIO and some more.

nerdponx 5y ago

One thing I don't fully understand is how async-without-yield is different from the "bad" green threads as described in e.g. https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html Are these two fundamentally opposing viewpoints? Or can these two sets of ideas be reconciled somehow? The article that I posted specifically calls out Java as an example of why you need explicit await, yet the article in the OP praises Java for not having it!