14 ms·

Rust’s async isn’t colored

• acjohnson55 6y ago
  I'm not a Rust user, but this sounds similar to the situation in Scala, where you can call Await.result to block the thread on a Future to get its value. The problem is that now, any callers of the function that blocks are blocking. So one form of coloring, "async", is converted to another, "blocking". You can use multithreading and synchronization to break of this trap, but now you're in the world we built all of this to escape in the first place. I think this is similar to what the author is describing, in which case, I'd say that function color contagion is a feature of continuation passing style programming, and its sugared variants (futures, async/await), rather than being something that you can say absolutely about a language or a runtime.
  • thinkharderdev 6y ago
    Yeah, Scala doesn't have async/await but you have the same basic issue where if you have one async function (something that returns a Future/IO/Task/etc) then every caller has to also be async (or block on the result and potentially cause all manner of mayhem). I think it's generally less of a problem in Scala though because most libraries/frameworks etc are async by default which makes sense. It's much easier to take a sync effect and fit it into an async process (just wrap the result in Future.successful/IO.pure/etc) than it is to fit an async call into a sync process (you can always do Await.result/IO.unsafeRunSync but it can have some very perverse effects on runtime behavior).
• txdv 6y ago
  The articles title F#king has nothing to do with the programming language F#, it is actually just masking a swear word. I was confused in the beginning
  • scribu 6y ago
    Yeah, better to leave out the swear word completely from the HN entry, rather than write a confusing euphemism.
    • johncolanduoni 6y ago
      It was censored in the original with #@. I’m guessing HN doesn’t allow @ in post titles.
      • OJFord 6y ago
        The space and capitalisation makes it even more confusing, `f#king` wouldn't have been so bad; `f##king` or `f*king` (or the more common vowel-omission style `f*cking`) better.
      • nirui 6y ago
        Well, it worked 7 months ago for this item: https://news.ycombinator.com/item?id=23895789 https://news.ycombinator.com/item?id=23895789
• deleted 6y ago
  [deleted]
• mseepgood 6y ago
  What's an F# king? The spouse of the C# queen?
  • v-yadli 6y ago
    Those who _champion_ F# features :p
• pron 6y ago
  > However this is not a problem in Rust! In Rust, you can easily confine the coloring to exactly the part of the code you want to, and doing it is pretty straightforward... Of course you shouldn't do both of those things all the time, or you will lose most of the benefits of using async in the first place! This is true for most languages with async/await, and if you call the two colours differently and to get the full benefit you shouldn't mix them even though with some effort you could, because mixing them has a cost that's high enough to require two distinct sets of API, then guess what? You have coloured functions! For an article that shows that async/await -- in Rust as in almost any other language (some languages employ the terminology async/await for a different feature) -- is very much coloured, the exasperation at something that is obviously true is perplexing. Perhaps the author believes that function colouring is not a big hindrance for them in Rust, or that function colouring is not a significant downside of async/await in general, but it is very much there. It's an odd title for an article that says, async is definitely coloured, but I can live with it.
  • kuschku 6y ago
    Javascript (where the color analogy is from) does not allow in any way calling promise/async/callback based functions as if they were blocking. Every language where you can call an async function as if it was blocking has no colored functions, and none of the issues of it.
    • BenoitP 6y ago
      Well, making a blocking call -say writing to a log for auditing purposes- in a tight loop can dramatically decrease performance. There will be no issue in terms of program correctness, but there will be one in terms of performance assumptions. And performance might actually have been why the programmer chose Rust in the first place? Seems like Rust's async are not correctness-colored, but they are performance-colored.
      • gpderetta 6y ago
        > There will be no issue in terms of program correctness there is if it leads to deadlocks.
• johncolanduoni 6y ago
  I’m a big fan of Rust’s Future model, but it is still colored in the same way as most languages that offer a “block until future/promise/whatever is complete” (C#, Java, etc.). You can do that, but depending on your executor model it can vary from inefficient (e.g. blocking a thread on the thread pool that is supposed to be running futures) to deadlocking (e.g. a single-threaded executor, or blocking on all threads of a thread pool). Rust’s futures library actually contains a defense mechanism against precisely this behavior: a function “futures::executor::enter” that returns a guard that causes an immediate panic if another “futures::executor::enter” call is made on the same thread while it is alive. The idea is that every executor calls this when it takes over execution on a thread, so you get an immediate panic if you try to block on a future inside another future instead of risking a deadlock.
• pansa2 6y ago
  Despite the title, the article accepts that function colours exist in Rust. The question is, does it matter? Here’s a specific scenario to think about. If I have synchronous functions `f`, `g` and `h`, `f` can call `g(h)` and `g` can in turn call its parameter `h` - no problems. Now if I have async functions `f2` and `h2`, can `f2` call `g(h2)` without the need to write an async version of `g`? In a language like Lua (with stackful coroutines), the answer is “yes” - `g` can be called from and can call functions of either colour. But AFAIK in JavaScript (with callbacks, promises and async/await based on promises), and in Python (with async/await based on stackless coroutines), the answer is “no” - the colour of `g` must match that of its caller and callee. What’s the answer for Rust? Can we answer “yes” by following this article’s recommendations of “spawn [g] as a blocking task” and “throw [h2] on an executor“?
  • majewsky 6y ago
    Instead of f2(g(h2())) you'd write something like f2(async { g(h2().await) }).await if f2 takes a future, or f2(g(h2().await)).await if it takes a value. (The latter is more common.) The equivalent in JS would be await f2(g(await h2()))
    • pansa2 6y ago
      The example uses `g(h2)`, not `g(h2())` - `g` takes the function `h2` as an argument, and the call(s) to `h2` are made from within `g` itself. If calling `h2` requires an `await` if and only if `h2` is async, then the function `g` can’t be used for both the synchronous and async cases. AFAIK this is the main objection to having two function colours.
      • gpderetta 6y ago
        exactly this. Unless the language allows abstracting over the async/non-async calls.
  • moron4hire 6y ago
    In JavaScript, you can await a non-promise returning function and it's a NO-OP. So if you are taking a function as a parameter, you can await it safely, regardless of whether it's async or not. function giveMe3() { return 3; } function giveMe3Async() { return Promise.resolve(3); } async function logger(func) { return console.log(await func()); } await logger(giveMe3); await logger(giveMe3Async);
• dfgdghdf 6y ago
  F# Async functions are colored, in case people are wondering. However, you can "un-color" them like this: doAsyncThing |> Async.RunSynchronously
  • dathinab 6y ago
    In rust `block_on(asyncFuncion())`. Through as async in rust is implemented in libraries (just the async=>Future transformation + some shared traits/utility is provided by rust-lang) you need to import `block_on`. E.g. `smol::block_on(asyncFunction())`.
• hannofcart 6y ago
  Quoting this article's opinion on the 'What colour is your function' article > One the one hand it's a great article that hits the nail on the head why using async/await in JS (and other languages) is very painful > Rust's async/await even though - you might have guessed it - it doesn't apply! So the author starts off by declaring that the colored functions problem only applies to other languages but not to rust. Soon after, he takes exception with the claim "You can only call a red function from within another red function". His primary argument in support of this is that in rust you can execute an async function blockingly in an executor. And to do vice versa, i.e. call a sync function in an async loop, that you can either block the event loop or hand it off to a different machinery. The fact is, both of these things can be done in other async runtimes as well. For eg. it's a near identical scenario in python async. Like others have pointed out in this comment thread, this is true in Scala too. Perhaps the author is unaware of this? So yes, rust does indeed have the colored function dichotomy, as any async runtime would, the author's angry assertions to the contrary notwithstanding. (I say this as a fan of both rust as well as async runtimes, which I feel are a nice fit for certain use cases.)
  • EugeneOZ 6y ago
    Indeed. Also, "very painful" is subjective. I never had issues with async code in JS (well, I had bugs, but it's not a fault of the language). There's nothing painful in async JS.
    • 1f60c 6y ago
      Having to use an async immediately invoked function expression in Node scripts is kind of a pain, but I agree.
      • cpfohl 6y ago
        I haven't found it painful, but i avoid iife in favor of a named function I call... ``` async function main() { await stuff(); } if (module === require.main) main(); ```
      • infensus 6y ago
• EugeneOZ 6y ago
  Rust's async just doesn't worth the hassle. At the moment when you have enough requests to get the benefits from async over thread::spawn, you'll HAVE TO run multiple instances/servers, at least for fail-safety reasons. And when the load is balanced, all the async benefits are not enough to be noticed, so there are no reasons to step into this mess.
• mratsim 6y ago
  Aren't all statically typed languages colored anyway? With something called a type. What's the point of using a type system if "color" is not a feature? It eases maintenance and working as a team. And regarding complains about requiring to add or removing await everywhere, this is equivalent to saying "you shouldn't tell everyone that your proc can suspend and all assumptions about the real control flow are off".
  • dathinab 6y ago
    This speaks about a very different kind of coloring which applies to more or less any language (IMHO) but definetly not just statically typed languages. What it speaks about a "coloring" of the call context! BLUE => Blocking I/O call context. RED => Async I/O call context. And calling a function which expects a blocking I/O call context in a function running under a Async I/O call context (or the other way around) is sub-optimal. Depending on the language this can cause anything from it works fine over performance problems and increased likelihood of dead locks to compiler errors or even super nasty runtime bugs. The important part is it's 100% language depended. But people use arguments like "that's colored so it's bad" (because of the color) or "that has a different color so you can't call it here" (because of the color). Even through it might very well not be bad that it's colored and you might very well be perfectly fine calling that there even through it has a different color then the calling function. I.e. people use the argument in the same way people argue that static typing is fundamentally bad because there is some language where it brings more drawbacks then benefits.
  • gpderetta 6y ago
    Most languages allow abstracting over types (for example via parametric polymorphism), but not over sync/async (although some might).
• cb321 6y ago
  I think the resolution to the author's consternation is that Rust's/some/most async run-times have "good enough" (in his view) color management. You still need to manually manage color (aka blocking vs non-blocking, sync vs async). E.g., it impacts how you write the calls. So, the coder must be aware of the colors -- a big chunk of the problem / annoyance even if compilers / type systems can perhaps flag / catch errors. I think what happened here was that the author found such awareness and management personally easy enough to use that he wanted to dismiss the entire coloring complaint. Then, in his zeal for dismissal, he made logical errors / mischaracterizations / exaggeration. It happens. Probably best to just move on. (EDIT: Focusing on such situations doesn't help perceptions of programming language evangelism/propaganda.)
• david_draco 6y ago
  The original post was about Node.js, and doesn't mention Rust at all. Not sure why the author is so emotional.
  • hobofan 6y ago
    Because miraculously once a comment section about async Rust is big enough, some people always start throwing around the coloring argument, and I'm getting tired of explaining why this doesn't apply to Rust, and thought I'd write something I can refer to in the future. In particular this was prompted by the article from yesterday[0], that in its opening paragraph blindly asserts that the coloring problems apply to Rust. [0]: https://theta.eu.org/2021/03/08/async-rust-2.html https://theta.eu.org/2021/03/08/async-rust-2.html discussed at https://news.ycombinator.com/item?id=26406989 https://news.ycombinator.com/item?id=26406989
• zozbot234 6y ago
  Rust has its own colored functions problems due to the whole Fn/FnMut/FnOnce distinction, based on the properties of closures' captured state. It's bad enough that some folks would say Rust functions are not genuinely first class, and it's also why some features that are common in other functional languages (including monads as generally understood in FP) are challenging in Rust. It may be possible to come up with some more elegant solution to the overall issue by adding linearity to the Rust type system, but clearly this comes with its own issues wrt. ergonomics. All of these are really complex, research-level problems, and it's not clear whether they can be addressed sufficiently in an overtly "practical" PL design.
• leosarev 6y ago
  If Rust async isn't colored, so C#'s.
  • bouke 6y ago
    In C# one has to take the synchronization context into account. Calling an async method in a synchronous method while blocking, will lead to deadlocks sooner or later on various implementations. For example ASP.NET (non-Core) only allows 1 thread to enter the context. Now you could sprinkle some ConfigureAwait(false) over your code, but that means you can no longer access HttpContext inside your code. Not sure if Rust does something similar, but C# is definitely coloured.
    • dathinab 6y ago
      > but C# is definitely coloured. And so is Rust and IMHO any language (Go is mono red ;) ). The more relevant take away is that not all drawbacks associated with coloring does apply to all languages equally. IMHO the only constant about coloring is that calling functions of different color is somewhat harder then calling the same color (but how much depends on language). This also means if your language default to blue/red then red/blue is likely to be generally slightly harder to use.
• MichaelBurge 6y ago
  Last time I tried you couldn't call block_on inside another call to block_on, so you can't always just "throw it on the executor". But it's been a year since I tried so maybe executors are more likely to support arbitrarily-nested async -> sync -> async -> sync ... chains now.
• jackhalford 6y ago
  > I personally don't mind it much, as that explicitness is one of my favourite things about Rust. Rust has operator overloading, so the + operator may call a function, I don't call that explicit. For me the 2 first checkmarks of being coloured are what matter most, the author argues that you can work around the third point, but it seems like a huge pain to do routinely. I'm still in the camp that non colored async runtimes are more useful.
  • nindalf 6y ago
    What are some examples of non coloured async runtimes?
    • majewsky 6y ago
      One of the more widely deployed ones is Go. Code looks like with synchronous IO, but everything runs in green threads that can be suspended when IO would block. So if you do a File.Read, after it returns, you might be running on a different OS thread because you were suspended mid-Read and later woken up on a different thread. It's generally nice to work with, unless you need to interface with a non-Go library that expects you to use a single thread throughout (e.g. for callbacks), in which case suddenly everything turns into a royal mess.
      • pansa2 6y ago
        > everything runs in green threads that can be suspended when IO would block AFAIK the main objection to this model is that you have to write your code so that it can handle being suspended at any point. OTOH when using async-await, the only potential suspension points are the places marked with `await`.
        • gpderetta 6y ago
          I don't think that's a very strong guarantee though. Trivially you can call any function that can block. Less trivially, any function that you can can resume any other suspended coroutine. So in practice there isn't much you can rely on. In general you can not rely on state that has escaped your function to be unmodified across function calls (although here of course rust has tools to prevent that, but they would work even without the sync/async distinction). It is true that other code can only run at function call boundaries, but that's true for any cooperatively scheduled runtime, even with stackfull coroutines.
• cygx 6y ago
  Disagreed: As far as I can tell, to get a value out of an async function, you use block_on() if called from a regular function, but .await if called from another async one. That's an example of function coloring, and it does not have to be this way (see eg Raku - formerly known as Perl6 - and, if I'm not mistaken, Julia as well[1]). Now, you might argue that it is an innocuous example of function coloring that does not come with the negative implications of Javascript's version, but that doesn't mean the coloring isn't there. [1] even in these examples, there's still a bit of colouring going on insofar that there's a difference between returning promises or values which Zig avoids, but their approach looks a bit too action-at-a-distancy for my taste...
• antender 6y ago
  If we follow the logic of the article's author to the end, then JS doesn't have this problem either. You can await on non-Promise values, so can just say that everything is red, add await to every function call and call it a day.
• dathinab 6y ago
  I would argue rust is colored (and I'm not sure if there are any languages which are truly uncolored) but I would argue the original article this links to is quite specific to JS and more specific pre-async/await(promise) JS!! So you can't at all carry over many of the arguments made there just like that. Mainly in a more programming language agnostic way I would say: 1. We need at least three categories, instead of two: GRAY/UNCOLORED - function is fully call context independent; BLUE - function is expected to be called in a blocking/os-threaded context; RED - function is expected the be called in some form of async calling context. 1.1. The async context might be layered on top of a normal os context. 1.2. Function can potentially be compiler and or runtime generic over the calling context 1.3. There can be more calling contexts then RED and BLUE, e.g. interrupts and signal handlers. 2. You need to call the function in the right context, but if the way you call it changes at all is fully language specific. Furthermore type systems can prevent you from using it the "wrong way". Just because you have green thread support or similar doesn't mean you don't have colored functions. Especially when doing FFI function coloring often still is important to consider. Through languages might hide away one color as good as they can (which is always a best effort compromise, at least as long as you don't run a OS specifically made for the given prog. lang and even then hardware might still get in your way...) 3. You need to call functions in the right context, but that doesn't mean you can't change or layer the context inside of an function. E.g. the border between gray/uncolored and blue functions is vague so often everything is run in a blue context and a red context is in some places layered on top of a blue context. Through a language could decide to not do so! Still while you can run them in each other by ad-hook creating the right calling context, it is often not recommended due to performance aspects or increased likelihood of things like dead locks. 3.1. In rust we layer a red context on-top of a blue one, furthermore you can always create a red context inside of a blue function ad-hook. It's not perfect but super easy. 3.2. Lastly to some degree the whole red context in rust is just syntax/library sugar around using a blue context with certain conventions. 3.3. Any constraint about higher-order functions is not coloring but language specific. E.g. Rust currently supports functions as values (incl. closures). But it only has very limited support for higher kindred functions (or types) which are needed for having more than just basic higher-order functions. Which can currently impose some restrictions wrt. higher-ordered functions in an async context. The pending support for generic associated types should solve this problem. 4. Red functions need to be called in a red context, if this is more painful depends 100% on the language in question. Many languages default to a blue context, furthermore red context often allow MUCH more control over the flow of execution then blue contexts so it's not uncommon to have a bit different syntax around them. But if they are more painful to call is inherently a aspect of the programming language and not coloring on itself. 4.1. In rust you need to use the `.await` operator to call them. While you could argue that it's more painful, you can also argue it's less painful as it makes many thinks much less painful then a implicit await would. 4.1.1. A a side not I would argue in rust a `Future` is a red function and a `async` function or closure is red in source code but then get's compiled to a gray/uncolored function returning a red function. Which is more complex, but is also inherently what gives you more control over the execution of read functions. 5. Again super language dependent and not at all function coloring specific. Any language should either make the core libaris fully generic over the color, or provide both. 5.1. In rust all of core (and std) is gray/uncolored or blue. Read is in the end "just" layered on top and in core/std only supported as far as providing a (generator-like) code transformation and a view shared traits/utility types to allow impl. red context's as a library. If you consider such library a a core component of rust then rust has taken the path of providing a separate blue/red functions for all colored (non-gray) functionality. So what is the takeaway? I would say the takeaway is that using arguments like "it's bad because of coloring" or well any argument using coloring as a reason why something is good or bad is fundamentally flawed. In the end it's not the coloring but the languages implementation of it which might or might not cause problems. This also means there are inherently more ways to fix a problem then there would be if coloring was actually at fault!
• the_gipsy 6y ago
  Doesn't the "colors" concept also apply to concepts like error handling? E.g. Result, Option, exceptions... You have to "add colors", e.g. special map()s or wraps. A Result<a, err> cannot be chained with a pure fn and an Option<a> without adding colors. Or a fn that throws cannot be called from a pure fn that never throws, without adding colors. Js promises do wrap try/catch in most cases, but I recall some unusual constructor that doesn't. Rust doesn't have exceptions, so Result<> is standard. It's painless to mix .map() with .and_then(), however I do think this is a case of "coloring". You also have to "color" when chaining with Option<>, which sometimes is conceptually between a pure fn and a Result fn.
  • pornel 6y ago
    It doesn't, because you can wrap and unwrap Results at will, so you can change "color" of such functions, and you're never irrecoverably stuck with just one color. The original article's key point was about async forcing refactoring of all functions in the entire call graph just because a leaf function needed it. This doesn't apply to async or options in Rust, because Rust functions can use these features without exposing that fact to their callers.
    • the_gipsy 6y ago
      That's what I'm saying, you can change the "color" of all these including async, but you still have to do that. There is no single Monad type from which you can build and chain, you always have to color some calls.
      • pornel 6y ago
        No, you can abstract over "color". You can implement a trait that lets you call both sync and async functions in the same way. You can with the same code and same syntax, get results of either `Fn()->T` or `Fn()->Future<Output=T>`.
        • the_gipsy 6y ago
          You are right, it doesn't "poison" your code, you can always wrap or map it. I guess I was just going off some tangent.
• deleted 6y ago
  [deleted]
• stevemk14ebr 6y ago
  Calling a sync function from an async when using the tokio executor is literally documented as a thing you should not do. Furthermore if you want to then spawn a task or whatever it's called, which is the documented way to do this safely, then you have to capture with static lifetime. That is a hard guarantee to provide for sufficiently complex code! Rust is definitely colored, and it's not as simple as just spawning an executor
  • stuhood 6y ago
    To spawn a sync function from an async context you use https://docs.rs/tokio/1.3.0/tokio/task/fn.spawn_blocking.html https://docs.rs/tokio/1.3.0/tokio/task/fn.spawn_blocking.htm....
• butterisgood 6y ago
  Colors seem to be about how functions compose, and the way functions compose is a way we can control effects. "Throw it on an executor" doesn't change the fact that you had to change the way you evaluate different colored functions. So I guess it's still colored? (How are function "colors" not just monads without saying "monads"? Maybe if you understand function coloring you already understand what monads are and just don't know it yet.)
• recursivedoubts 6y ago
  I am building an experimental front end language that does not have coloured functions: https://hyperscript.org https://hyperscript.org The runtime works everything out at the expression level to handle promises so developers don't need to deal with them: https://hyperscript.org/docs/#async https://hyperscript.org/docs/#async
• travisgriggs 6y ago
  Disclaimer: I am not a hard core consumer of async utilities and idioms. Does anyone remember Aspect Oriented Programming? Past the peak of OOP excitement, there was a set of problems that OOP hadn't just made magically simple, and aspects were the solution. I recall it was always the same basic three examples that were used to illustrate why everyone needed them. Compilers were enhanced, libraries were written, lots of papers submitted, lots of conference sessions and talks. And then it all kind of faded away. Is the async movement going to be a case of history rhyming with itself? I see some similarities (and of course some differences). (If iOS Grand Central Dispatch is to be considered an async implementation, I consume a bit of async that way I guess)
  • pjmlp 6y ago
    AOP is pretty much alive in each Spring application.
• brundolf 6y ago
  > However sometimes you don't really have a choice, and this super cool crate you want to use is async or sync (obviously always the version you don't need right now), and you have to make it work. Doesn't the whole crate get colored, though, because now it requires an async runtime? Or is it possible to synchronously run an async function without a runtime? Genuinely asking; I haven't used this stuff yet firsthand
• skyde 6y ago
  the article say “ Want to call a sync function from an async function? -> Either just call in normally, or if it would block your async function spawn it as a blocking task” What “spawn it as a blocking task” mean? Is it ok to create a task using blocking non-async function?
• jnwatson 6y ago
  I ported a large Python codebase from sync to async. There's a temptation to partially move and keep a mixed async/sync structure. That temptation is wrong though. There's a worse secondary effect of mixing async and sync code by using executors (async->sync) or posting to the async loop (sync->async). The first effect: it means that your sync functions that were previously non-blocking are now blocking, so all async callers are now forced to make calls into them with executors. Even worse, if you have chains of async -> executor -> sync -> post -> async -> executor -> sync (as you might do by mechanically converting sync code), each executor call takes up an entire worker in the pool, and the Python thread pool has a maximum number of workers. You need as many workers as the sum of the maximum number of outstanding async->sync transition stacks in all of your async tasks. The penalty for underestimating this is deadlock.
• breatheoften 6y ago
  why do people think function coloring is bad? In my experience it's a very good thing ... An async function call is a massively different abstraction than a sync function call ... think about what logic is allowed to live on the other side of the async function call -- it can be literally anything without changing the caller ... sync function calls imply a very different set of expectations exist between function callers and function implementations and distinguishing between those expectations is a pure win. If you are finding function coloring to be a big issue then the problem probably mainly lives in how you've organized your logic or codebase ... separating the colors so that things which should compose well do happens naturally in codebases which organize around data transformations (which most applications should aim toward)
  • mywittyname 6y ago
    Function coloring is bad because, as you say, the two function call types are massively different abstractions, while being almost interchangable. If two things are massively different, they shouldn't be interchangable. Javascript handled async in probably the worst possible way. So now, effectively, everybody writes exclusively async functions, and calls them with await, regardless of whether it is necessary, because the introduction of an async call anywhere in the call stack will require every function in that stack to be changed to async in order to use the damn await keyword. And that change can have implications all over the codebase. It looks like rust managed to avoid that tomfoolery by not introducing such a stupid rule.
• throwawaygulf 6y ago
  Can we not use the word "colored"? It's a racist term from colonial times used by white supremacists to dehumanize BIPOCs.
• phibz 6y ago
  I think of async functions in rust as regular functions that happen to return a Future. Sure to do something useful you need to use an executor (or follow the semantics of one) which uses a context to track the progress through a state machine until success or failure. But it isn't from a different "universe" (color) of functions. That's the big difference here from say Javascript. At least that's how I explain it to myself.