8 ms·

Blocking code is a leaky abstraction

140 points by zdw 2y ago

plorkyeran 2y ago
The author has a very strange understanding of the idea of a "leaky abstraction". AppKit requiring you to call methods on the main thread is just not an abstraction at all, leaky or otherwise.
paulyy_y 2y ago
Or just use fibers and avoid icky futures.
- aphantastic 2y ago
  Doesn’t change the core issue, which is that if you want to do multiple operations simultaneously that take time to execute, you’re going to need to write stuff to handle that which you wouldn’t otherwise. Whether it’s channel piping or async awaiting is semantics.
  - PaulDavisThe1st 2y ago
    It's not just semantics. With synchronous code, the blocking aspects are invisible in when reading code: size_t rcnt = read (fd, buf, sizeof (buf)); /* nobody knows if we will block */ With some async code, the fact that there will be a wait and then a continuation is visible from just reading the code.
nemetroid 2y ago
The author seems to reinforce the original point, that the async paradigm ends up working best in an all-or-nothing deal. Whether the difficulties in interfacing the paradigms should be attributed to the blocking part or the async part does not really matter for the practical result: if calling blocking code from async code is awkward and your project's main design is async, you're going to end up wanting to rewrite blocking code as async code.
PaulDavisThe1st 2y ago
Sometimes, things need to leak. For example, the fact that code actually takes time to execute ... this is an abstraction that should almost certainly leak. The fact that some data you want is not currently available ... whether you want to obscure that a wait is required or not is up for debate and may be context dependent. "I want to always behave as though I never have to block" is a perfectly fine to thing to say. "Nobody should ever have to care about needing to block" is not.
- fargle 2y ago
  > this is an abstraction that should almost certainly leak. that's not a leak, that is the abstraction for procedural/imperative code, or at least an important part of it. essentially: "each step occurs in order and the next step doesn't happen until the former step is finished"
  - PaulDavisThe1st 2y ago
    That's not a particularly interesting description for multithreaded code. The procedural/imperative element of it may be true of any given thread's execution, but with the right user-space or kernel scheduler, you may or may not care what the precise sequencing of things is across threads. This is true whether you're using lightweight user-space threads ("fibers") or kernel threads.
    - fargle 2y ago
      of course, but in that case it's not the base imperative abstraction that's leaky - it's the multi-thread/multi-core abstraction built on top of it - it's almost as easy as multiple threads happening in parallel. until it isn't. the multi-thread abstraction attempts to be: "hey, you know how that imperative model works? now we can have more than one of these models (we'll call them threads!) running in parallel. neat, huh?" it's things like memory barriers and race conditions, etc. that are the "leaks". but it's the threading/multi-core abstractions that are leaky not (back to the article) the "blocking" code. i can write single-thread "blocking" code all day long that does all kinds of interesting non-trivial things and never have to worry about leaks wrt. blocking, in-order execution, etc. even in the presence of out-of-order execution processors and optimizing compilers, VMs, multi-user OS'es, etc. effects are observably identical to the naive model. the author didn't do a good job of clearly defining anything, but i bristle at the idea that the basic "blocking" code abstraction whats leaky - it's the async, threads, event-driven models, etc. that are (necessarily) a bit leaky when they break the basic imperative blocking model or require accommodation of it.
klodolph 2y ago
What is the difference between code that blocks waiting for I/O and code that performs a lengthy computation? To the runtime or scheduler, these are very different. But to the caller, maybe it does not matter why the code takes a long time to return, only that it does. Async only solves one of these two cases. I’d like to draw an analogy here to ⊥ “bottom” in Haskell. It’s used to represent a computation that does not return a value. Why doesn’t it return a value? Maybe because it throws an exception (and bubbles up the stack), or maybe because it’s in an infinite loop, or maybe it’s just in a very long computation that doesn’t terminate by the time the user gets frustrated and interrupts the program. From a certain perspective, sometimes you don’t care why ⊥ doesn’t return, you just care that it doesn’t return. Same is often true for blocking calls. You often don’t care whether a call is slow because of I/O or whether it is slow because of a long-running computation. Often, you just care whether it is slow or how slow it is. (And obviously, sometimes you do care about the difference. I just think that the “blocking code is a leaky abstraction” is irreparably faulty, as an argument.)
- rileymat2 2y ago
  Async is worse than leaky it often goes viral because many other parts need to be async to call it.
  - gwbas1c 2y ago
    It's better to think of "async" as indicating that a code will do something that blocks, and we're allowing our process to manage its blocking (via Futures) instead of the operating system (via a context switch mid-thread.) I would argue a few things: First: You need to be aware, in your program, of when you need to get data outside of your process. This is, fundamentally, a blocking operation. If your minor refactor / bugfix means that you need to add "async" a long way up the stack, does this mean that you goofed on assuming that some kind of routine could work only with data in RAM? Instead: A non-async function should be something that you are confident will only work with the data that you have in RAM, or only perform CPU-bound operations. Any time you're writing a function that could get data from out of process, make it async.
worik 2y ago
Asynchronous code is the bees knees Async/await is a horrible fit for Rust. > Some of this criticism is valid. async code is a little hard to wrap your head around, but that’s true with many other concepts in Rust, like the borrow checker and the weekly blood sacrifices. Many popular async libraries are explicitly tied to heavyweight crates like tokio and futures, which aren’t good picks for many types of programs. There are also a lot of language features that need to be released for async to be used without an annoying amount of Boxing dynamic objects. Yes. So don't do it It is entirely possible to write asynchronous code without using asyc/await. async/await is a fun paradigm for garbage collected systems. It is a horrible mess in Rust
- 0x1ceb00da 2y ago
  It's the only real solution if you need tens of thousands of concurrent tasks. Real threads use too much memory.
  - smw 2y ago
    Or you use something like goroutines or Elixir processes where green threads are preemptively scheduled.
  - worik 2y ago
    That is a false dichotomy. There are other options than async/await and threads. In my heart that is my main objection to async/await in Rust. It has "poisoned the well" for asynchronous programming. Where people think it is async/await or threads We have been doing asynchronous programming for many decades.
Spivak 2y ago
It seems like the magic for the author is in the annotation. There's still two kinds of functions "fearlessly run in the UI thread" and "must be run in a worker" and async is great in that it greatly expands the number of functions in that first pool but it's effectively the same thing as the [blocking] annotation but the other way and not totally complete. Because there are async-safe but not async functions. So to me it seems like it all leaks because in all cases what's desired is some external signal about how the function behaves. A "badly written" async function can stall the loop and that isn't conveyed only by the async tag. It's rare to do this but you can and there might even be times where this makes sense[1]. But it happens that by convention async and ui-thread-safe line up but there's a secret other axis that async is getting mixed up in. [1] Say running a separate event loop in a background thread used for CPU intensive tasks that occasionally perform IO and you want to make the most use of your thread's time slice.
wavemode 2y ago
> Frankly, I don’t think async code is leaky at all, and the ways that it does leak are largely due to library problems. Huh? One could argue the exact same in opposite direction - blocking code isn't leaky at all, just wrap it in a Future. And if doing that is difficult because of !Send data, then those are just "library problems". You could always just use a threadsafe library instead, right?
rtpg 2y ago
"asynchronous code does not require the rest of your code to be synchronous" fails the smell test. Many APIs are shipping async-only. You can't stick calls to those APIs in the middle of your sync code in many cases (unless you have an event loop implementation that is re-entrant). So... you gotta convert your sync code to async code. I am curious as to what a principled definition of a "blocking" function would be. I suppose it's a call to something that ends up calling `select` or the like. Simply saying "this code needs to wait for something" is not specific enough IMO (sometimes we're just waiting for stuff to move from RAM to registers, is that blocking?), but I'm sure someone has a clear difference. If you care about this problem enough, then you probably want to start advocating for effect system-like annotations for Rust. Given how Rust has great tooling already in the language for ownership, it feels like a bit of tweaking could get you effects (if you could survive the eternal RFC process)
- PaulDavisThe1st 2y ago
  > I am curious as to what a principled definition of a "blocking" function would be. It's one where the OS puts your (kernel) thread/task to sleep and then (probably) context switches to another thread/task (possibly of another process), before eventually resuming execution of yours after some condition has been satisfied (could be I/O of some time, could be a deliberate wait, could be several other things).
  - akoboldfrying 2y ago
    That seems a necessary but not sufficient condition, since a pre-emptively multitasking OS may do this after almost any instruction. Not only that, but any OS with virtual memory will almost certainly context-switch on a hard page fault (and perhaps even on a soft page fault, I don't know). So it would seem that teading memory is sufficient to be "blocking", by your criterion.
    - PaulDavisThe1st 2y ago
      1) I deliberately left it open to including page faults by design. If you do not understand that reading memory might lead to your process blocking, you have much to learn as a programmer. However, I would concede that that this is not really part of the normal meaning of "blocking", despite the result being very similar. 2) I did not intend to include preemption. I should reword it to make it clear that the "condition" must be something of the process' own making (e.g. reading from a file or a socket) rather than simply an arbitrary decision by the kernel.
samatman 2y ago
There's a conflation going on here which is a blocker, shall we say, for understanding the two sides of this coin. The antonym of async is not blocking, it is synchronous. The antonym of blocking I/O is evented I/O. Stackless coroutines à la Rust's async are a perfectly reasonable control flow primitive to use in 'blocking' contexts, which properly aren't blocking at all since they aren't doing I/O, I'm thinking of generator patterns here. It's also entirely possible, and not even difficult, to use evented I/O in synchronous function-calls-function style code. You poll. That's the whole deal, somewhere in the tick, you check for new data, if it's there, you eat it. This does not require async. I found the "what if the real abstraction leak was the friends we made along the way" portion of the argument. Unconvincing. When people complain about async leaking, what they mean is that they would like to be able to use libraries without have to start a dummy tokio in the background. That's what's getting peas in the mashed potatoes, that's what annoys people. "Abstraction leak" is a sort of idiomatic way to complain about this. Focusing on the slogan does nothing to address the substance, and "have you considered... that, a function, might take a while, and perhaps that, that is an 'abstraction leak'? eh, eh?" is. Well it probably seemed clever when it was being drafted.
skybrian 2y ago
This is the opposite in JavaScript (and similar single-threaded languages) where sync code is atomic and normally can't do I/O.
- akira2501 2y ago
  node.js has a whole plethora of synchronous IO methods. they're actually really nice. or have I misunderstood your meaning?
  - maronato 2y ago
    Node uses libuv[1] for all its IO operations[2]. The sync versions still use libuv, but block the main thread until they receive the result. [1] https://libuv.org https://libuv.org [2] https://nodejs.org/en/learn/asynchronous-work/overview-of-blocking-vs-non-blocking https://nodejs.org/en/learn/asynchronous-work/overview-of-bl...
- throwitaway1123 2y ago
  You can perform file I/O in JavaScript synchronously. The localStorage API in the browser is synchronous (and in Node via the --experimental-webstorage option), and of course requiring a CommonJS module is also synchronous (and there are many other sync filesystem APIs in Node as a sibling comment pointed out). You just can't perform network I/O synchronously. Although a network attached file system allows for both network and file I/O technically, but that's a really pedantic point.
  - Izkata 2y ago
    > You just can't perform network I/O synchronously. Sure you can, you just shouldn't ever do it because it blocks the UI: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest_API/Synchronous_and_Asynchronous_Requests#synchronous_request https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequ...
    - throwitaway1123 2y ago
gwbas1c 2y ago
> Dependency Dog: If you want to see a good example of a leaky abstraction, consider AppKit. Not only is AppKit thread-unsafe to the point where many functions can only safely be called on the main() thread, it forces you into Apple’s terrifying Objective-C model. Basically any program that wants to have working GUI on macOS now needs to interface in Apple’s way, with basically no alternatives. Uhm, every OS that I've developed a GUI for has a similar limitation. The "only call this thing from this thread" is a well-known thread safety pattern. It might not be the pattern that this author prefers, but it is an industry standard pattern for UI.
- koito17 2y ago
  I agree with your comment. Every UI framework I have used (AppKit, Swing, JavaFX, etc.) is not thread-safe. It requires all UI code in the main thread or, in the case of JavaFX, a dedicated UI thread (not necessarily the main thread, but it must be a single thread where all UI operations occur).
  - etcd 2y ago
    WPF was the same IIRC
    - solarkraft 2y ago
      yup!
    - gwbas1c 2y ago
      That's how winforms did it; and I suspect that's how the underlying Windows OS needs it done too.
  - mike_hearn 2y ago
    Technically in JavaFX you can work with UI objects off the main thread, for instance to load or build up UI in parallel. The single thread requirement only kicks in when UI is attached to a live window (stage).
  - hinkley 2y ago
ianhooman 2y ago
Godël covers how it’s leaky abstraction all the way down. If you consider a specific chunk of useful async code it still doesn’t compose enough of system to do much of use; it can’t answer many questions alone or any about itself. We never just have Map() but other functions. Its constant composition of symbolic logic we vaguely agree means something relative to an electrical system of specific properties. A useful system is “blocked” on continuous development of that system. These posts are too specific to code, miss the forest for a tree. A more generalized pedantry (like electrical state of systems rather than dozens of syntax sugars to simulate the same old Turing machine) and more frequent spot checks for stability and correctness relative to the ground truth would be great. Way too much circumlocution around code. Map is not the terrain. I for one am having blast training a state generator that acts like a REPL and (for example) infers how to generate Firefox from a model of its code (among other things). Excited about the potential to peel away layers of abstraction by generalizing the code with automated assistants.
akira2501 2y ago
I can't buy this premise. The leaky abstraction is that your selected libraries are attempting to be as efficient and as fast as possible while also trying to be fully general. As a result you get the full abstraction layer necessary to do this, which, as you've noted, for any one project is too much. The complication you seem to be experiencing is other projects have come to rely on these libraries and their particular abstractions and so you've actually got a dependency management problem due to the design of the "core" libraries here or due to a lack of language features to /constrain/ all of them. A critical point of abstraction totally lacking in this ecosystem. In any case if you look at any language that has contexts that can be externally canceled you should realize this actual solution must be implemented at a higher level than these "async primitives."
accelbred 2y ago
I have the opposite experience, working in embedded (C, not Rust...). Building a synchronous API on top of an async one is hell, and making a blocking API asynchronous is easy. If you want blocking code to run asynchronously, just run it on another task. I can write an api that queues up the action for the other thread to take, and some functions to check current state. Its easy. To build a blocking API on top of an async one, I now need a lot of cross thread synchronization. For example, nimBLE provides an async bluetooth interface, but I needed a sync one. I ended up having my API calls block waiting for a series of FreeRTOS task notifications from the code executing asynchronously in nimBLE's bluetooth task. This was a mess of thousands of lines of BLE handling code that involved messaging between the threads. Each error condition needed to be manually verified that it sends an error notification. If a later step does not execute, either through library bug or us missing an error condition, then we are deadlocked. If the main thread continues because we expect no more async work but one of the async functions are called, we will be accessing invalid memory, causing who knows what to happen, and maybe corrupting the other task's stack. If any notification sending point is missed in the code, we deadlock.
- hackit2 2y ago
  The trick with synchronization/parallelism systems is to only communicate over a known yield point this is normally done via queues. It is the only way you get deterministic behavior from your sub-systems or multi-threaded environments.
- jcranmer 2y ago
  Making an asynchronous task into a synchronous task is easy in exactly one scenario: when there is no pre-existing event loop you need to integrate with, so the actual thing the synchronous task needs to do is create the event loop, spin it until it's empty, and then continue on its merry way. Fall off this happy path, and everything is, as you say, utterly painful. In the opposite direction, it's... always easy (if not entirely trivial). Spin a new thread for the synchronous task, and when it's done, post the task into the event loop. As long as the event loop is capable of handling off-thread task posting, it's easy. The complaint in the article that oh no, the task you offload has to be Send is... confusing to me, since in practice, you need that to be true of asynchronous tasks anyways on most of the common async runtimes.
deleted 2y ago
[deleted]
anacrolix 2y ago
This is some bullshit. And it doesn't help that everyone is using async to mean different things. Blocking/synchronous system calls are not friendly to userspace async runtimes. Explicit async runtimes are a pain in the ass. The best solution imo is a blocking userspace interface that maps into the system however it likes. Go and Haskell do this. Erlang and probably all BEAM languages do too. Nim and Zig let you choose. Runtimes like Go and Haskell can map onto code that makes blocking system calls for compatibility reasons when needed, through explicit request to the runtime, or detecting that a green thread has blocked.
- smw 2y ago
  This is the right answer! Preemptively scheduled n:m green threads >>>> any async design.
  - hinkley 2y ago
    I’ve always thought of async await as an ergonomic relaxation (as in constraints) of green threads. People had more problems doing green threads right than I’ve seen with async. Do some of the remaining problems look exactly like green thread problems? Absolutely. But it’s a step toward something more hygienic.
Joker_vD 2y ago
> Blocking code is a leaky abstraction > Asynchronous code does not require the rest of your code to be asynchronous. I can’t say the same for blocking code. Well, you can also say that "file systems are a leaky abstractions. Sector-addressing disk access does not require the rest of your code to access disk sectors directly. I can't say the same for the code that uses files". First of all, asynchronous code does require the rest of your code to be asynchronous: because, as you've yourself said, blocking code doesn't mesh well with the asynchronous. Second, when your are trying to work with two different levels of abstractions at the same time, of course you will find the presence of the higher-level abstraction (the blocking code, the file system, etc) to be inconvenient: the higher-level abstraction has to make some opinionated choices, or it would not be an abstraction but merely an interface. So yeah, I guess blocking code is a leaky abstraction, just like file systems, TCP streams, garbage-collected memory, and sequential processor instruction execution. It's still a higher-level abstraction though.
nurettin 2y ago
block_on and spawn_blocking are tokio's async abstractions, there is no "blocking code's abstraction" that can leak. What the article intentionally omits is that I can defer io by getting a handler and polling it in a select loop. And right after that, I can also poll my mpsc queue if I want to. This is how "blocking code" has worked for the last few decades.
fargle 2y ago
the author is misusing the "leaky abstraction" idea. in the section "what's in a leak" a rather muddy argument is made that a leak is that which forces you to bend your program to accommodate it. so `leak => accommodation`. then it immediately conflates the need for that accommodation (difficulty of calling one type of code with another) with "leakiness". so essentially "calling blocking => more accommodation" (e.g. from event loops that shouldn't block). - that's logically incorrect, even in the author's own argument (A => B, B, therefore A is the affirming the consequent fallacy). - that's not what a "leaky abstraction" is. not everything that doesn't fit perfectly or is hard is due a "leaky abstraction". rather it's when the simplistic model (abstraction) doesn't always hold in visible ways. a "leaky abstraction" to "blocking" code might be if code didn't actually block or execute in-order in all situations. until you get to things like spinlocks and memory barriers this doesn't happen. but that's more a leak caused by naively extending the abstraction to multi-core/multi-thread use, not the abstraction of blocking code itself. i love the passion for `async` from this fellow, but i think he's reacting to "async is a leaky abstraction" as if it were a slur or insult, while misunderstanding what it means. then replies with a "oh yeah, well your construct is twice as leaky" retort.
- dcow 2y ago
  I actually think what the author is arguing is that async formalizes the idea of structuring your code to yield execution when it needs to "block". So block is leaky because it lacks a formal definition outside of an async annotated world. I find myself sympathetic to that take, despite my frustration with Rust's implementation of async, and the ecosystem/language shortcomings, etc.
amelius 2y ago
They mean a viral abstraction.
netbsdusers 2y ago
I am surprised by some of the comments claiming to find it very hard to make an async API out of a sync one. I write a hobby kernel with a profoundly asynchronous I/O system. Since it's nice to have synchronous I/O in some places (reading in data in response to a page fault, for example, or providing the POSIX read/write functions), I have to turn it into synchronous I/O in some cases. Asynchronous I/O notifies its completion in a configurable-per-I/O-packet (called "IOP") way. The default is to signal an event object. You can wait on an event object and when it's signalled, you're woken up. The result is that synchronously sending an IOP is hardly any more work than: iop_send(iop); wait_for_event(iop->completion_event); return iop->result; I would be surprised to hear of any system where you can't do similar. Surely any environment has at least semaphores available, and you can just have the async I/O completion signal that semaphore when it's done?
- rileymat2 2y ago
  I don't think the problem comes so much from async v. sync, it comes from the implementation language/framework. If you were to wait_for_event (or its equivalent) in the node main thread you will stall the 1 thread handling requests. This whole problem is not just async/sync, it is the environment you live in. What we should be questioning is why high level languages in high level frameworks make the programmer even consider the distinction at all. If I am handling an http request, the async nature of this is an optimization because threads (or worse, processes, I once easily dos'ed an apache system that created a new process for each request) are too expensive at scale. It should not concern the programmer whether a thread is launched for each request or we are using cooperative multitasking (async). The programmer/application needs that information from that call before that one piece can move forward.
  - vacuity 2y ago
    The lower level async-as-state-machine (or coroutine or whatever) should be more standard, but that's an argument for another day. For now, you're right and the opaque threading model is basically what Go does. On the other hand, it's not suitable for Rust because in some ways Rust is lower level. C# went and did it the Rust way, which I personally appreciate but I understand fits into the problem you're describing.
hinkley 2y ago
What I like about async and promises is there’s just enough bookkeeping to keep people from violating Keenighan’s law by too wide a margin. A little goes a long way, especially when any process could be handling several requests at once and make better use of the dead space between steps. But a friend recently showed me how coroutines work in Kotlin and from my rather extensive experience helping people debug code that’s exceeded their Kernighan threahold, this seems a lot less footgun than nodejs. It’s far too easy to inject unhandled promise rejection in Nodejs. I think await should be the default not the exception, and Kotlin is closer to that.
- wruza 2y ago
  How do they work in Kotlin? If it’s just: result = co_func() Then that creates an invisible yield point which messes with your “is this a critical section” sense. Suddenly anything may yield and your naive pt.x = getX() pt.y = getY() turns into the concurrency hell wrt pt’s consistency. I think await should be the default not the exception, and Kotlin is closer to that. I somewhat agree in that it should be an await/nowait keyword pair required at async calls. Not realistic though even in typescript, considering that Promise{} is just a continuation shell that may be passed around freely and doesn’t represent a specific function’s asynchronicity.
  - hinkley 2y ago
    The biggest problem with the Node/Javascript solution is that if I call a function and don’t need the result, if that function is changed to async (or I missed that it always was) end up with bugs because we didn’t await the result. When I say things like this people typically claim I’m bringing up some hypothetical, so let me be clear: I’m tired of fixing this bug. Not in my code (mostly), but in the code base. It happens too often both in a numerical sense and in a sense that it’s so stupid that it happens at all. It’s how I started thinking about this problem in the first place. The solution we have isn’t working very well.
jongjong 2y ago
Async/await has been a game-changer. You can tell that this is the case by how much shorter and more reliable code with async/await is compared to alternatives (e.g. which rely on callbacks). Callbacks are a leaky abstraction IMO because they force you to write a lot of additional logic to handle results in the correct sequence. Obtaining the result of multiple long-running function calls on the same line is insanely messy with callbacks. I've found that async/await helps to write code which stays close to the business domain. You can design it with multiple layers which take care of a distinct level of abstraction so you can understand what's happening just by reading the main file top to bottom. You don't have to jump around many files to read the functions to understand what they are doing and how their results are combined. Async/await makes the concurrency aspect as succinct as possible. That said, I've seen many people writing code with async/await as though they were writing using callbacks... Some people haven't adapted correctly to it. If your async/await code doesn't look radically different from how you used to write code, then you're probably doing something seriously wrong. Ensure that you familiarize yourself with the async/await utility functions of your language like (in JS) Promise.all, Promise.race... Notice how try-catch works with the await keyword...
- dboreham 2y ago
  > compared to alternatives (e.g. which rely on callbacks). This is a browser-centric view, which makes no sense in the context of Rust (which runs on a proper OS with threads and absolutely no need for callbacks).
  - jongjong 2y ago
    I work a lot with Node.js on the server too.
tombert 2y ago
I was debating writing a blog post like "Blocking IO Considered Harmful" or something cheeky like that. I'm not as familiar with Rust or C as other people, but I've done a lot of Java, and JDBC has made my life more difficult than it needs to be due to its semantics. It has been hard for me to get decent thread utilization with it because of all the blocking it does behind the scenes. I've moved most of my SQL stuff to Vert.x SQL, which has given me better performance overall, and I can do most of it with a single thread. At least in Java land, it's trivial to convert a non-blocking call into a blocking one (in Vert.x you can do myFuture.toCompletionStage().toCompletableFuture().get()), but it's pretty hard to efficiently go the other direction. I really wish JDBC would create something like "executeAsync" that returns a CompletableFuture or something similar.
- hackit2 2y ago
  In defense of Java blocking calls, in the 1990s to the 2010's the industry mostly considered Asynchronous programming as coordinating multiple threads over know yield points - such as using atomic locks, or mutexs. The main push by the industry was all about abstractions and OOP, and moving away from static C/C++ programming languages to dynamic programming languages such as Java/Python/Visual Basic or C#. It is only until recently with the popularity of Node.js, Python and the explosion of the web that the developer have to deal with single threaded languages that blocking calls are a significant impact to the users experience. This push for Async in the web has had a cascade effect to system programming languages where old time developers are only first learning about it.
  - BalinKing 2y ago
    Could you clarify the "static C/C++" vs "dynamic Java" distinction? I'm used to thinking of "static" and "dynamic" in the context of type systems, but do the terms have overloaded meanings in concurrency/parallelism?
  - hinkley 2y ago
    It was also pushed by the C10K movement.
lalaithion 2y ago
> There are a substantial number of async crates out there that run on top of tokio. They use tokio’s primitives, tokio’s executor, and tokio’s I/O semantics. Because of this, they rely on tokio’s runtime to be running in the background. If you try the above strategy for a crate that relies on tokio, it will fail at runtime with a panic. This is the worst design choice in the broader Rust ecosystem right now. It’s insane that people are exporting public functions from packages that rely on their caller running the right thing in the background. Imagine a library that didn’t work if you didn’t create a global mutable variable for it to modify, and that variable didn’t even need to be passed into the library explicitly. Tokio should have a marker type that’s Copy and returned from functions that start the runtime, and require a parameter of that type when calling functions which expect a runtime to already exist. Yes this is backwards incompatible. It should be done anyway.
- Arnavion 2y ago
  If the value is Copy that value will continue to be usable after the runtime has been stopped and dropped, defeating the purpose. If the value borrows from the runtime to prevent that, futures that hold on to that value stop being `'static`, which brings its own problems, chief among them being breaking `tokio::spawn()` and causing a self-reference (runtime contains futures which contain borrows of itself). If the value holds on to (a refcount of) the runtime internally to prevent the runtime from being dropped until the value has been dropped, that will prevent programs from exiting as long as even one future is still running on the runtime, which is undesirable.
  - Filligree 2y ago
    Does anyone realistically shut down the Tokio runtime once it's been started? It wouldn't be a problem if the runtime is immortal once created; the runtime reference becomes more of a token demonstrating that, yes, this is running under Tokio. Though I don't think this is such a huge problem. Yes, it's a runtime crash that could potentially be eliminated at compile-time, but realistically that will only happen once during development, and it will be an easy fix.
perryizgr8 2y ago
> Asynchronous code does not require the rest of your code to be asynchronous. Seems like in many languages the async keyword is viral. And that is the most pointless work I've ever done, where I've to go around and mark everything async just because of one async function. So this is a strange statement to me.
dboreham 2y ago
Really frustrating that Rust became infected with the async disease. This article is pure nonsense fwiw. It says: you can always block on the Future returned by an async function. Ugh. Ok, but let's just not have any async at all please, and then there's be no need to add that code and no behind the scenes context switching done.
- vacuity 2y ago
  Async usage is overblown in some cases, but not in others. The mistake of async Rust is not that it's unnecessary, just that it's being pushed for general purpose usage. Which is probably impractical for any async abstraction, or for any abstraction in general.
vacuity 2y ago
I posit that while more programmer friendly async abstractions (such as in Go) are good in the short-term, ideally async should be far less scary even in the lower levels. Programmers should shy away from async not because it is hard, but when they have carefully considered if the architecture can afford to be sync instead. If they aren't doing async, it's likely that they rely on other code that should be handling async well. Alan Kay said (paraphrased roughly) that programmers use multithreading when they don't understand how to write programs as the state machines that they are. Then you get all the surprise and suspicion about distributed locking (featured on HN today!) and whatnot. There is much essential complexity to be tackled here. If robust software is a priority (it really should be), we need to be deliberate in whether async is exposed at a given layer of abstraction. We shouldn't be awed into writing subpar programs.
- hinkley 2y ago
  Was Kay thinking of local code or distributed code? For local I can see his point. But once we start talking to networks or even really hard drives you need scatter-gather semantics. Otherwise you’re running lots of threads, which undercuts his point, or you’re writing your own multitasking state machine which is just a poor implementation of concurrency where you’re responsible for all the bugs and deadlocks, not just some of them.
  - vacuity 2y ago
    (I should caveat what I'm about to say that I'm primarily concerned about writing robust and highly performant programs, and while I believe it should be a focus broadly, it's a practical niche.) That's the thing, though. It's arguably even more important for distributed code. If we abstract away the state machine too much, it becomes difficult to reason about the code precisely because of abstraction. The complexity that was present explicitly in the state machine will just cause confusing behavior in the abstracted version. Using lightweight threads or another high level abstraction that approximates blocking code will allow getting a program out faster, but lower quality at that. Two examples to illustrate my point: first, you mention scatter-gather, but the base concept is orthogonal to sync/async. However, I/O is characteristically async, and therefore the underlying mechanisms are async anyways. Second, io_uring is showing that async can be good for performance while not being a difficult interface. Sync code makes things easier to reason about but also kinda not. I think the big issues with async are that the OS hasn't done a good job allocating responsibility of async interfaces and the fundamental difficulty. The former makes async seem less efficient than it could be, which is true in that a sync-over-async interface is better than an async-over-sync-over-async interface, but we should have async interfaces accessible. The latter probably feeds into a bias to not even touch async where mixing async and sync would be the best blend of performance and programmability.
CJefferson 2y ago
I realise I'm picking one sentence out of the middle of this, but this is the one thing I don't feel I understand about async: "But well-formed Futures complete quickly" Does this mean people writing async code don't ever write code which takes a while to compute? I write lots of code which takes 10s of milliseconds, or 100s of milliseconds, or hours. And none of these feel like a good fit for async, as even a 10s of milliseconds piece of work would be enough to start to limit throughput and add unacceptable lag to other connections? This is fine in a multi-threaded server, one thread per connection, you make a difficult request, the server spins for half a second before responding to you -- but it doesn't really fit in a async program?
- hinkley 2y ago
  Definitely not going to solve a c10k problem with 10ms pauses, that’s for sure.
  - CJefferson 2y ago
    True, but some problems just require that long to get the answer the user wants! Some problems take 60 CPU years, but those you don’t expect the user to actively wait while you distribute their task and wait for the results, you send them an email in 3 months or so! :)
- dcow 2y ago
  The same problem comes up at the end with the proposed `#[blocking]` solution. There is no formal definition of blocking outside of async. It's a colloquial term meaning "takes a relatively longer time to finish". Async is the formalization of making a distinction between grinding the cpu and yielding until something else wakes you up and tells you to keep going. So I guess a well formed Future always adds yield points between hot loops? And then you just trust the runtime to manage execution context such that your yields are noops if there's no contention for execution time and/or to avoid needless context switching.
  - CJefferson 2y ago
galaxyLogic 2y ago
Is having the 'await' keyword really necessary? What if every time I call an async function the interpreter would automatically "await" for its result?
- vacuity 2y ago
  Perhaps (?) it wouldn't be an issue for correctness in most languages, but it would certainly add a performance and debuggability cost. It could be useful in some niche domains, probably.
sqeaky 2y ago
Does the author not know what a "leaky abstraction" is? The first time they referred to memory leaks I thought is a clever joke nested in a jab a hypothetical coder who didn't know, but then in the end they double down in a clearly non-joking way. Then even make a chart showing that calling a normal function from an async function is trivial, that is the opposite of a leaky abstraction. An abstraction is a "leaky abstraction" when it forces some external condition or behavior onto code (ex: calling code that uses the network doesn't work when the network is disconnected). In this case the hurdles of calling async code from normal code, and this author tries to hand waive that with talk of dependencies. There are real benefit and drawbacks to async code and styles. Even if I don't think it is a good abstraction I want to discuss it on the real engineering terms of its actual merits and drawbacks. This article is talking past the engineering decisions and seems like it didn't mean to. maybe I am just being a pedantic jerk.
- lalaithion 2y ago
  What I think this comes down to, is that in Rust: It is easy but dangerous to call a non-async function from an async function (since the non-async function could block, forcing all your async code to block on this function). It is hard but safe to call an async function from a non-async function. Note that this is language specific; if Rust had “async”, “pure”, and “blocking” as three different function colors, then we would have a very different design. Or, for example, in JavaScript: It is easy and safe to call a non-async function from an async function, since JavaScript exposes no blocking primitives. It is impossible to call an async function from a non-async function, since there is no way to block on the result.
kelnos 2y ago
I don't think the author's understanding of what a "leaky abstraction" is jives with how I've ever used it or heard it used. To me, a leaky abstraction is when there's hidden behavior underneath that makes the observed behavior when using the abstraction to change in ways that aren't intuitive based on the API contract. For example, "everything is a file" on UNIX is a leaky abstraction: read() and write() can behave in wildly different ways, and have significantly different constraints, depending on what the file descriptor refers to (regular file, character device, socket, memfd, etc.). Hell, C stdio/unistd functions themselves are leaky abstractions even when we're only considering regular files on disk. For example, even if your open()+write()+close() succeeds, the data may have not yet hit the disk, and might be lost if the machine crashes right after close() returns. And in some cases an fsync() might not even be enough, if the disk lies to the OS about the state of its own buffers and cache. Put another way, abstractions are about hiding (hopefully) unnecessary implementation details from the person using your abstraction. An abstraction is leaky when those hidden details turn out to actually be important, and the person using your abstraction needs to understand them anyway. The "Dependency Dog" sidebar block that has a mini-rant about AppKit on macOS is further evidence that the author doesn't know what a leaky abstraction is. You might not like AppKit, but every windowing system has canonical APIs to make windows/surfaces and draw things and define behavior and handle input. You just... have to use them, or some abstraction built on top of them. That doesn't make them leaky. (Also, pretty much every UI toolkit out there has restrictions on multi-threading; seems like a weird thing to complain about.) The author's use of "blocking" was also a little confusing. To me, "blocking" means making a call that's going to sit and spin while waiting on I/O (or some other slow/contended resource), but the author uses it to mean "a function that is not declared async". The opposite of async is sync, not blocking.
- lalaithion 2y ago
  Rust doesn’t have a way in the type system to tell if a function that isn’t declared async is going to block or not. Their point is that any of those functions could block, and having them block wouldn’t be an obvious breaking change, and so you need to know about their internals to know if they’re okay to call.
WiSaGaN 2y ago
All abstractions are leaky, but some are useful. You can't have a perfect abstraction. Your abstraction is always a compromise of various goals you want to achieve.
nice_byte 2y ago
just document your api's thread safety and let me handle it. don't make me deal with this async/await bullshit. multithreaded programming is hard enough as it is.
hesus_ruiz 2y ago
The author must have specified in the title: "Blocking code IN RUST is a leaky abstraction". I am so happy to be using Go for most programs! It has one of the best concurrency implementations that I have ever seen (Erlang permitting ...). And for most of my programs, the garbage collection is not a problem. It's the opposite: simplifies programming incredibly. No blocking versus async problems. No libraries for async are needed. Just concurrency (which has its own problems, but this is another story ...).
lfmunoz4 2y ago
Articles like this remind me how Kotlin is probably 10 times easier and concise than Rust and C++. Kind of off topic but has anyone noticed difference in code quality and documentation from projects in C++ / Rust / Kotlin / JavaScript libraries that have the same features? Recently was working with Seastar which is Tokio / Kotlin Coroutine equivalent in C++ and seems crazy how far behind it is from Kotlin Coroutines. The documentation the example, the boiler plate etc. Don't think it is the language themselves but the language culture. Seems like C++ libraries are just designed to be more difficult. Less documentation, no comments, flat folder structures with each file having with 1000s of lines of code. They expect you to go read through source code etc.

140 points by zdw 2y ago

The author has a very strange understanding of the idea of a "leaky abstraction". AppKit requiring you to call methods on the main thread is just not an abstraction at all, leaky or otherwise.

paulyy_y 2y ago

Or just use fibers and avoid icky futures.

aphantastic 2y ago
Doesn’t change the core issue, which is that if you want to do multiple operations simultaneously that take time to execute, you’re going to need to write stuff to handle that which you wouldn’t otherwise. Whether it’s channel piping or async awaiting is semantics.
- PaulDavisThe1st 2y ago
  It's not just semantics. With synchronous code, the blocking aspects are invisible in when reading code: size_t rcnt = read (fd, buf, sizeof (buf)); /* nobody knows if we will block */ With some async code, the fact that there will be a wait and then a continuation is visible from just reading the code.

nemetroid 2y ago

The author seems to reinforce the original point, that the async paradigm ends up working best in an all-or-nothing deal. Whether the difficulties in interfacing the paradigms should be attributed to the blocking part or the async part does not really matter for the practical result: if calling blocking code from async code is awkward and your project's main design is async, you're going to end up wanting to rewrite blocking code as async code.

PaulDavisThe1st 2y ago

Sometimes, things need to leak. For example, the fact that code actually takes time to execute ... this is an abstraction that should almost certainly leak. The fact that some data you want is not currently available ... whether you want to obscure that a wait is required or not is up for debate and may be context dependent. "I want to always behave as though I never have to block" is a perfectly fine to thing to say. "Nobody should ever have to care about needing to block" is not.

fargle 2y ago
> this is an abstraction that should almost certainly leak. that's not a leak, that is the abstraction for procedural/imperative code, or at least an important part of it. essentially: "each step occurs in order and the next step doesn't happen until the former step is finished"
- PaulDavisThe1st 2y ago
  That's not a particularly interesting description for multithreaded code. The procedural/imperative element of it may be true of any given thread's execution, but with the right user-space or kernel scheduler, you may or may not care what the precise sequencing of things is across threads. This is true whether you're using lightweight user-space threads ("fibers") or kernel threads.
  - fargle 2y ago
    of course, but in that case it's not the base imperative abstraction that's leaky - it's the multi-thread/multi-core abstraction built on top of it - it's almost as easy as multiple threads happening in parallel. until it isn't. the multi-thread abstraction attempts to be: "hey, you know how that imperative model works? now we can have more than one of these models (we'll call them threads!) running in parallel. neat, huh?" it's things like memory barriers and race conditions, etc. that are the "leaks". but it's the threading/multi-core abstractions that are leaky not (back to the article) the "blocking" code. i can write single-thread "blocking" code all day long that does all kinds of interesting non-trivial things and never have to worry about leaks wrt. blocking, in-order execution, etc. even in the presence of out-of-order execution processors and optimizing compilers, VMs, multi-user OS'es, etc. effects are observably identical to the naive model. the author didn't do a good job of clearly defining anything, but i bristle at the idea that the basic "blocking" code abstraction whats leaky - it's the async, threads, event-driven models, etc. that are (necessarily) a bit leaky when they break the basic imperative blocking model or require accommodation of it.

klodolph 2y ago

What is the difference between code that blocks waiting for I/O and code that performs a lengthy computation? To the runtime or scheduler, these are very different. But to the caller, maybe it does not matter why the code takes a long time to return, only that it does. Async only solves one of these two cases. I’d like to draw an analogy here to ⊥ “bottom” in Haskell. It’s used to represent a computation that does not return a value. Why doesn’t it return a value? Maybe because it throws an exception (and bubbles up the stack), or maybe because it’s in an infinite loop, or maybe it’s just in a very long computation that doesn’t terminate by the time the user gets frustrated and interrupts the program. From a certain perspective, sometimes you don’t care why ⊥ doesn’t return, you just care that it doesn’t return. Same is often true for blocking calls. You often don’t care whether a call is slow because of I/O or whether it is slow because of a long-running computation. Often, you just care whether it is slow or how slow it is. (And obviously, sometimes you do care about the difference. I just think that the “blocking code is a leaky abstraction” is irreparably faulty, as an argument.)

rileymat2 2y ago
Async is worse than leaky it often goes viral because many other parts need to be async to call it.
- gwbas1c 2y ago
  It's better to think of "async" as indicating that a code will do something that blocks, and we're allowing our process to manage its blocking (via Futures) instead of the operating system (via a context switch mid-thread.) I would argue a few things: First: You need to be aware, in your program, of when you need to get data outside of your process. This is, fundamentally, a blocking operation. If your minor refactor / bugfix means that you need to add "async" a long way up the stack, does this mean that you goofed on assuming that some kind of routine could work only with data in RAM? Instead: A non-async function should be something that you are confident will only work with the data that you have in RAM, or only perform CPU-bound operations. Any time you're writing a function that could get data from out of process, make it async.

worik 2y ago

Asynchronous code is the bees knees Async/await is a horrible fit for Rust. > Some of this criticism is valid. async code is a little hard to wrap your head around, but that’s true with many other concepts in Rust, like the borrow checker and the weekly blood sacrifices. Many popular async libraries are explicitly tied to heavyweight crates like tokio and futures, which aren’t good picks for many types of programs. There are also a lot of language features that need to be released for async to be used without an annoying amount of Boxing dynamic objects. Yes. So don't do it It is entirely possible to write asynchronous code without using asyc/await. async/await is a fun paradigm for garbage collected systems. It is a horrible mess in Rust

0x1ceb00da 2y ago
It's the only real solution if you need tens of thousands of concurrent tasks. Real threads use too much memory.
- smw 2y ago
  Or you use something like goroutines or Elixir processes where green threads are preemptively scheduled.
- worik 2y ago
  That is a false dichotomy. There are other options than async/await and threads. In my heart that is my main objection to async/await in Rust. It has "poisoned the well" for asynchronous programming. Where people think it is async/await or threads We have been doing asynchronous programming for many decades.

Spivak 2y ago

It seems like the magic for the author is in the annotation. There's still two kinds of functions "fearlessly run in the UI thread" and "must be run in a worker" and async is great in that it greatly expands the number of functions in that first pool but it's effectively the same thing as the [blocking] annotation but the other way and not totally complete. Because there are async-safe but not async functions. So to me it seems like it all leaks because in all cases what's desired is some external signal about how the function behaves. A "badly written" async function can stall the loop and that isn't conveyed only by the async tag. It's rare to do this but you can and there might even be times where this makes sense[1]. But it happens that by convention async and ui-thread-safe line up but there's a secret other axis that async is getting mixed up in. [1] Say running a separate event loop in a background thread used for CPU intensive tasks that occasionally perform IO and you want to make the most use of your thread's time slice.

wavemode 2y ago

> Frankly, I don’t think async code is leaky at all, and the ways that it does leak are largely due to library problems. Huh? One could argue the exact same in opposite direction - blocking code isn't leaky at all, just wrap it in a Future. And if doing that is difficult because of !Send data, then those are just "library problems". You could always just use a threadsafe library instead, right?

rtpg 2y ago

"asynchronous code does not require the rest of your code to be synchronous" fails the smell test. Many APIs are shipping async-only. You can't stick calls to those APIs in the middle of your sync code in many cases (unless you have an event loop implementation that is re-entrant). So... you gotta convert your sync code to async code. I am curious as to what a principled definition of a "blocking" function would be. I suppose it's a call to something that ends up calling `select` or the like. Simply saying "this code needs to wait for something" is not specific enough IMO (sometimes we're just waiting for stuff to move from RAM to registers, is that blocking?), but I'm sure someone has a clear difference. If you care about this problem enough, then you probably want to start advocating for effect system-like annotations for Rust. Given how Rust has great tooling already in the language for ownership, it feels like a bit of tweaking could get you effects (if you could survive the eternal RFC process)

PaulDavisThe1st 2y ago
> I am curious as to what a principled definition of a "blocking" function would be. It's one where the OS puts your (kernel) thread/task to sleep and then (probably) context switches to another thread/task (possibly of another process), before eventually resuming execution of yours after some condition has been satisfied (could be I/O of some time, could be a deliberate wait, could be several other things).
- akoboldfrying 2y ago
  That seems a necessary but not sufficient condition, since a pre-emptively multitasking OS may do this after almost any instruction. Not only that, but any OS with virtual memory will almost certainly context-switch on a hard page fault (and perhaps even on a soft page fault, I don't know). So it would seem that teading memory is sufficient to be "blocking", by your criterion.
  - PaulDavisThe1st 2y ago
    1) I deliberately left it open to including page faults by design. If you do not understand that reading memory might lead to your process blocking, you have much to learn as a programmer. However, I would concede that that this is not really part of the normal meaning of "blocking", despite the result being very similar. 2) I did not intend to include preemption. I should reword it to make it clear that the "condition" must be something of the process' own making (e.g. reading from a file or a socket) rather than simply an arbitrary decision by the kernel.

samatman 2y ago

There's a conflation going on here which is a blocker, shall we say, for understanding the two sides of this coin. The antonym of async is not blocking, it is synchronous. The antonym of blocking I/O is evented I/O. Stackless coroutines à la Rust's async are a perfectly reasonable control flow primitive to use in 'blocking' contexts, which properly aren't blocking at all since they aren't doing I/O, I'm thinking of generator patterns here. It's also entirely possible, and not even difficult, to use evented I/O in synchronous function-calls-function style code. You poll. That's the whole deal, somewhere in the tick, you check for new data, if it's there, you eat it. This does not require async. I found the "what if the real abstraction leak was the friends we made along the way" portion of the argument. Unconvincing. When people complain about async leaking, what they mean is that they would like to be able to use libraries without have to start a dummy tokio in the background. That's what's getting peas in the mashed potatoes, that's what annoys people. "Abstraction leak" is a sort of idiomatic way to complain about this. Focusing on the slogan does nothing to address the substance, and "have you considered... that, a function, might take a while, and perhaps that, that is an 'abstraction leak'? eh, eh?" is. Well it probably seemed clever when it was being drafted.

skybrian 2y ago

This is the opposite in JavaScript (and similar single-threaded languages) where sync code is atomic and normally can't do I/O.

akira2501 2y ago
node.js has a whole plethora of synchronous IO methods. they're actually really nice. or have I misunderstood your meaning?
- maronato 2y ago
  Node uses libuv[1] for all its IO operations[2]. The sync versions still use libuv, but block the main thread until they receive the result. [1] https://libuv.org https://libuv.org [2] https://nodejs.org/en/learn/asynchronous-work/overview-of-blocking-vs-non-blocking https://nodejs.org/en/learn/asynchronous-work/overview-of-bl...
throwitaway1123 2y ago
You can perform file I/O in JavaScript synchronously. The localStorage API in the browser is synchronous (and in Node via the --experimental-webstorage option), and of course requiring a CommonJS module is also synchronous (and there are many other sync filesystem APIs in Node as a sibling comment pointed out). You just can't perform network I/O synchronously. Although a network attached file system allows for both network and file I/O technically, but that's a really pedantic point.
- Izkata 2y ago
  > You just can't perform network I/O synchronously. Sure you can, you just shouldn't ever do it because it blocks the UI: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest_API/Synchronous_and_Asynchronous_Requests#synchronous_request https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequ...
  - throwitaway1123 2y ago

gwbas1c 2y ago

> Dependency Dog: If you want to see a good example of a leaky abstraction, consider AppKit. Not only is AppKit thread-unsafe to the point where many functions can only safely be called on the main() thread, it forces you into Apple’s terrifying Objective-C model. Basically any program that wants to have working GUI on macOS now needs to interface in Apple’s way, with basically no alternatives. Uhm, every OS that I've developed a GUI for has a similar limitation. The "only call this thing from this thread" is a well-known thread safety pattern. It might not be the pattern that this author prefers, but it is an industry standard pattern for UI.

koito17 2y ago
I agree with your comment. Every UI framework I have used (AppKit, Swing, JavaFX, etc.) is not thread-safe. It requires all UI code in the main thread or, in the case of JavaFX, a dedicated UI thread (not necessarily the main thread, but it must be a single thread where all UI operations occur).
- etcd 2y ago
  WPF was the same IIRC
  - solarkraft 2y ago
    yup!
  - gwbas1c 2y ago
    That's how winforms did it; and I suspect that's how the underlying Windows OS needs it done too.
- mike_hearn 2y ago
  Technically in JavaFX you can work with UI objects off the main thread, for instance to load or build up UI in parallel. The single thread requirement only kicks in when UI is attached to a live window (stage).
- hinkley 2y ago

ianhooman 2y ago

Godël covers how it’s leaky abstraction all the way down. If you consider a specific chunk of useful async code it still doesn’t compose enough of system to do much of use; it can’t answer many questions alone or any about itself. We never just have Map() but other functions. Its constant composition of symbolic logic we vaguely agree means something relative to an electrical system of specific properties. A useful system is “blocked” on continuous development of that system. These posts are too specific to code, miss the forest for a tree. A more generalized pedantry (like electrical state of systems rather than dozens of syntax sugars to simulate the same old Turing machine) and more frequent spot checks for stability and correctness relative to the ground truth would be great. Way too much circumlocution around code. Map is not the terrain. I for one am having blast training a state generator that acts like a REPL and (for example) infers how to generate Firefox from a model of its code (among other things). Excited about the potential to peel away layers of abstraction by generalizing the code with automated assistants.

akira2501 2y ago

I can't buy this premise. The leaky abstraction is that your selected libraries are attempting to be as efficient and as fast as possible while also trying to be fully general. As a result you get the full abstraction layer necessary to do this, which, as you've noted, for any one project is too much. The complication you seem to be experiencing is other projects have come to rely on these libraries and their particular abstractions and so you've actually got a dependency management problem due to the design of the "core" libraries here or due to a lack of language features to /constrain/ all of them. A critical point of abstraction totally lacking in this ecosystem. In any case if you look at any language that has contexts that can be externally canceled you should realize this actual solution must be implemented at a higher level than these "async primitives."

accelbred 2y ago

I have the opposite experience, working in embedded (C, not Rust...). Building a synchronous API on top of an async one is hell, and making a blocking API asynchronous is easy. If you want blocking code to run asynchronously, just run it on another task. I can write an api that queues up the action for the other thread to take, and some functions to check current state. Its easy. To build a blocking API on top of an async one, I now need a lot of cross thread synchronization. For example, nimBLE provides an async bluetooth interface, but I needed a sync one. I ended up having my API calls block waiting for a series of FreeRTOS task notifications from the code executing asynchronously in nimBLE's bluetooth task. This was a mess of thousands of lines of BLE handling code that involved messaging between the threads. Each error condition needed to be manually verified that it sends an error notification. If a later step does not execute, either through library bug or us missing an error condition, then we are deadlocked. If the main thread continues because we expect no more async work but one of the async functions are called, we will be accessing invalid memory, causing who knows what to happen, and maybe corrupting the other task's stack. If any notification sending point is missed in the code, we deadlock.

hackit2 2y ago
The trick with synchronization/parallelism systems is to only communicate over a known yield point this is normally done via queues. It is the only way you get deterministic behavior from your sub-systems or multi-threaded environments.
jcranmer 2y ago
Making an asynchronous task into a synchronous task is easy in exactly one scenario: when there is no pre-existing event loop you need to integrate with, so the actual thing the synchronous task needs to do is create the event loop, spin it until it's empty, and then continue on its merry way. Fall off this happy path, and everything is, as you say, utterly painful. In the opposite direction, it's... always easy (if not entirely trivial). Spin a new thread for the synchronous task, and when it's done, post the task into the event loop. As long as the event loop is capable of handling off-thread task posting, it's easy. The complaint in the article that oh no, the task you offload has to be Send is... confusing to me, since in practice, you need that to be true of asynchronous tasks anyways on most of the common async runtimes.

deleted 2y ago

[deleted]

anacrolix 2y ago

This is some bullshit. And it doesn't help that everyone is using async to mean different things. Blocking/synchronous system calls are not friendly to userspace async runtimes. Explicit async runtimes are a pain in the ass. The best solution imo is a blocking userspace interface that maps into the system however it likes. Go and Haskell do this. Erlang and probably all BEAM languages do too. Nim and Zig let you choose. Runtimes like Go and Haskell can map onto code that makes blocking system calls for compatibility reasons when needed, through explicit request to the runtime, or detecting that a green thread has blocked.

smw 2y ago
This is the right answer! Preemptively scheduled n:m green threads >>>> any async design.
- hinkley 2y ago
  I’ve always thought of async await as an ergonomic relaxation (as in constraints) of green threads. People had more problems doing green threads right than I’ve seen with async. Do some of the remaining problems look exactly like green thread problems? Absolutely. But it’s a step toward something more hygienic.

Joker_vD 2y ago

> Blocking code is a leaky abstraction > Asynchronous code does not require the rest of your code to be asynchronous. I can’t say the same for blocking code. Well, you can also say that "file systems are a leaky abstractions. Sector-addressing disk access does not require the rest of your code to access disk sectors directly. I can't say the same for the code that uses files". First of all, asynchronous code does require the rest of your code to be asynchronous: because, as you've yourself said, blocking code doesn't mesh well with the asynchronous. Second, when your are trying to work with two different levels of abstractions at the same time, of course you will find the presence of the higher-level abstraction (the blocking code, the file system, etc) to be inconvenient: the higher-level abstraction has to make some opinionated choices, or it would not be an abstraction but merely an interface. So yeah, I guess blocking code is a leaky abstraction, just like file systems, TCP streams, garbage-collected memory, and sequential processor instruction execution. It's still a higher-level abstraction though.

nurettin 2y ago

block_on and spawn_blocking are tokio's async abstractions, there is no "blocking code's abstraction" that can leak. What the article intentionally omits is that I can defer io by getting a handler and polling it in a select loop. And right after that, I can also poll my mpsc queue if I want to. This is how "blocking code" has worked for the last few decades.

fargle 2y ago

the author is misusing the "leaky abstraction" idea. in the section "what's in a leak" a rather muddy argument is made that a leak is that which forces you to bend your program to accommodate it. so `leak => accommodation`. then it immediately conflates the need for that accommodation (difficulty of calling one type of code with another) with "leakiness". so essentially "calling blocking => more accommodation" (e.g. from event loops that shouldn't block). - that's logically incorrect, even in the author's own argument (A => B, B, therefore A is the affirming the consequent fallacy). - that's not what a "leaky abstraction" is. not everything that doesn't fit perfectly or is hard is due a "leaky abstraction". rather it's when the simplistic model (abstraction) doesn't always hold in visible ways. a "leaky abstraction" to "blocking" code might be if code didn't actually block or execute in-order in all situations. until you get to things like spinlocks and memory barriers this doesn't happen. but that's more a leak caused by naively extending the abstraction to multi-core/multi-thread use, not the abstraction of blocking code itself. i love the passion for `async` from this fellow, but i think he's reacting to "async is a leaky abstraction" as if it were a slur or insult, while misunderstanding what it means. then replies with a "oh yeah, well your construct is twice as leaky" retort.

dcow 2y ago
I actually think what the author is arguing is that async formalizes the idea of structuring your code to yield execution when it needs to "block". So block is leaky because it lacks a formal definition outside of an async annotated world. I find myself sympathetic to that take, despite my frustration with Rust's implementation of async, and the ecosystem/language shortcomings, etc.

amelius 2y ago

They mean a viral abstraction.

netbsdusers 2y ago

I am surprised by some of the comments claiming to find it very hard to make an async API out of a sync one. I write a hobby kernel with a profoundly asynchronous I/O system. Since it's nice to have synchronous I/O in some places (reading in data in response to a page fault, for example, or providing the POSIX read/write functions), I have to turn it into synchronous I/O in some cases. Asynchronous I/O notifies its completion in a configurable-per-I/O-packet (called "IOP") way. The default is to signal an event object. You can wait on an event object and when it's signalled, you're woken up. The result is that synchronously sending an IOP is hardly any more work than: iop_send(iop); wait_for_event(iop->completion_event); return iop->result; I would be surprised to hear of any system where you can't do similar. Surely any environment has at least semaphores available, and you can just have the async I/O completion signal that semaphore when it's done?

rileymat2 2y ago
I don't think the problem comes so much from async v. sync, it comes from the implementation language/framework. If you were to wait_for_event (or its equivalent) in the node main thread you will stall the 1 thread handling requests. This whole problem is not just async/sync, it is the environment you live in. What we should be questioning is why high level languages in high level frameworks make the programmer even consider the distinction at all. If I am handling an http request, the async nature of this is an optimization because threads (or worse, processes, I once easily dos'ed an apache system that created a new process for each request) are too expensive at scale. It should not concern the programmer whether a thread is launched for each request or we are using cooperative multitasking (async). The programmer/application needs that information from that call before that one piece can move forward.
- vacuity 2y ago
  The lower level async-as-state-machine (or coroutine or whatever) should be more standard, but that's an argument for another day. For now, you're right and the opaque threading model is basically what Go does. On the other hand, it's not suitable for Rust because in some ways Rust is lower level. C# went and did it the Rust way, which I personally appreciate but I understand fits into the problem you're describing.

hinkley 2y ago

What I like about async and promises is there’s just enough bookkeeping to keep people from violating Keenighan’s law by too wide a margin. A little goes a long way, especially when any process could be handling several requests at once and make better use of the dead space between steps. But a friend recently showed me how coroutines work in Kotlin and from my rather extensive experience helping people debug code that’s exceeded their Kernighan threahold, this seems a lot less footgun than nodejs. It’s far too easy to inject unhandled promise rejection in Nodejs. I think await should be the default not the exception, and Kotlin is closer to that.

wruza 2y ago
How do they work in Kotlin? If it’s just: result = co_func() Then that creates an invisible yield point which messes with your “is this a critical section” sense. Suddenly anything may yield and your naive pt.x = getX() pt.y = getY() turns into the concurrency hell wrt pt’s consistency. I think await should be the default not the exception, and Kotlin is closer to that. I somewhat agree in that it should be an await/nowait keyword pair required at async calls. Not realistic though even in typescript, considering that Promise{} is just a continuation shell that may be passed around freely and doesn’t represent a specific function’s asynchronicity.
- hinkley 2y ago
  The biggest problem with the Node/Javascript solution is that if I call a function and don’t need the result, if that function is changed to async (or I missed that it always was) end up with bugs because we didn’t await the result. When I say things like this people typically claim I’m bringing up some hypothetical, so let me be clear: I’m tired of fixing this bug. Not in my code (mostly), but in the code base. It happens too often both in a numerical sense and in a sense that it’s so stupid that it happens at all. It’s how I started thinking about this problem in the first place. The solution we have isn’t working very well.

jongjong 2y ago

Async/await has been a game-changer. You can tell that this is the case by how much shorter and more reliable code with async/await is compared to alternatives (e.g. which rely on callbacks). Callbacks are a leaky abstraction IMO because they force you to write a lot of additional logic to handle results in the correct sequence. Obtaining the result of multiple long-running function calls on the same line is insanely messy with callbacks. I've found that async/await helps to write code which stays close to the business domain. You can design it with multiple layers which take care of a distinct level of abstraction so you can understand what's happening just by reading the main file top to bottom. You don't have to jump around many files to read the functions to understand what they are doing and how their results are combined. Async/await makes the concurrency aspect as succinct as possible. That said, I've seen many people writing code with async/await as though they were writing using callbacks... Some people haven't adapted correctly to it. If your async/await code doesn't look radically different from how you used to write code, then you're probably doing something seriously wrong. Ensure that you familiarize yourself with the async/await utility functions of your language like (in JS) Promise.all, Promise.race... Notice how try-catch works with the await keyword...

dboreham 2y ago
> compared to alternatives (e.g. which rely on callbacks). This is a browser-centric view, which makes no sense in the context of Rust (which runs on a proper OS with threads and absolutely no need for callbacks).
- jongjong 2y ago
  I work a lot with Node.js on the server too.

tombert 2y ago

I was debating writing a blog post like "Blocking IO Considered Harmful" or something cheeky like that. I'm not as familiar with Rust or C as other people, but I've done a lot of Java, and JDBC has made my life more difficult than it needs to be due to its semantics. It has been hard for me to get decent thread utilization with it because of all the blocking it does behind the scenes. I've moved most of my SQL stuff to Vert.x SQL, which has given me better performance overall, and I can do most of it with a single thread. At least in Java land, it's trivial to convert a non-blocking call into a blocking one (in Vert.x you can do myFuture.toCompletionStage().toCompletableFuture().get()), but it's pretty hard to efficiently go the other direction. I really wish JDBC would create something like "executeAsync" that returns a CompletableFuture or something similar.

hackit2 2y ago
In defense of Java blocking calls, in the 1990s to the 2010's the industry mostly considered Asynchronous programming as coordinating multiple threads over know yield points - such as using atomic locks, or mutexs. The main push by the industry was all about abstractions and OOP, and moving away from static C/C++ programming languages to dynamic programming languages such as Java/Python/Visual Basic or C#. It is only until recently with the popularity of Node.js, Python and the explosion of the web that the developer have to deal with single threaded languages that blocking calls are a significant impact to the users experience. This push for Async in the web has had a cascade effect to system programming languages where old time developers are only first learning about it.
- BalinKing 2y ago
  Could you clarify the "static C/C++" vs "dynamic Java" distinction? I'm used to thinking of "static" and "dynamic" in the context of type systems, but do the terms have overloaded meanings in concurrency/parallelism?
- hinkley 2y ago
  It was also pushed by the C10K movement.

lalaithion 2y ago

> There are a substantial number of async crates out there that run on top of tokio. They use tokio’s primitives, tokio’s executor, and tokio’s I/O semantics. Because of this, they rely on tokio’s runtime to be running in the background. If you try the above strategy for a crate that relies on tokio, it will fail at runtime with a panic. This is the worst design choice in the broader Rust ecosystem right now. It’s insane that people are exporting public functions from packages that rely on their caller running the right thing in the background. Imagine a library that didn’t work if you didn’t create a global mutable variable for it to modify, and that variable didn’t even need to be passed into the library explicitly. Tokio should have a marker type that’s Copy and returned from functions that start the runtime, and require a parameter of that type when calling functions which expect a runtime to already exist. Yes this is backwards incompatible. It should be done anyway.

Arnavion 2y ago
If the value is Copy that value will continue to be usable after the runtime has been stopped and dropped, defeating the purpose. If the value borrows from the runtime to prevent that, futures that hold on to that value stop being `'static`, which brings its own problems, chief among them being breaking `tokio::spawn()` and causing a self-reference (runtime contains futures which contain borrows of itself). If the value holds on to (a refcount of) the runtime internally to prevent the runtime from being dropped until the value has been dropped, that will prevent programs from exiting as long as even one future is still running on the runtime, which is undesirable.
- Filligree 2y ago
  Does anyone realistically shut down the Tokio runtime once it's been started? It wouldn't be a problem if the runtime is immortal once created; the runtime reference becomes more of a token demonstrating that, yes, this is running under Tokio. Though I don't think this is such a huge problem. Yes, it's a runtime crash that could potentially be eliminated at compile-time, but realistically that will only happen once during development, and it will be an easy fix.

perryizgr8 2y ago

> Asynchronous code does not require the rest of your code to be asynchronous. Seems like in many languages the async keyword is viral. And that is the most pointless work I've ever done, where I've to go around and mark everything async just because of one async function. So this is a strange statement to me.

dboreham 2y ago

Really frustrating that Rust became infected with the async disease. This article is pure nonsense fwiw. It says: you can always block on the Future returned by an async function. Ugh. Ok, but let's just not have any async at all please, and then there's be no need to add that code and no behind the scenes context switching done.

vacuity 2y ago
Async usage is overblown in some cases, but not in others. The mistake of async Rust is not that it's unnecessary, just that it's being pushed for general purpose usage. Which is probably impractical for any async abstraction, or for any abstraction in general.

vacuity 2y ago

I posit that while more programmer friendly async abstractions (such as in Go) are good in the short-term, ideally async should be far less scary even in the lower levels. Programmers should shy away from async not because it is hard, but when they have carefully considered if the architecture can afford to be sync instead. If they aren't doing async, it's likely that they rely on other code that should be handling async well. Alan Kay said (paraphrased roughly) that programmers use multithreading when they don't understand how to write programs as the state machines that they are. Then you get all the surprise and suspicion about distributed locking (featured on HN today!) and whatnot. There is much essential complexity to be tackled here. If robust software is a priority (it really should be), we need to be deliberate in whether async is exposed at a given layer of abstraction. We shouldn't be awed into writing subpar programs.

hinkley 2y ago
Was Kay thinking of local code or distributed code? For local I can see his point. But once we start talking to networks or even really hard drives you need scatter-gather semantics. Otherwise you’re running lots of threads, which undercuts his point, or you’re writing your own multitasking state machine which is just a poor implementation of concurrency where you’re responsible for all the bugs and deadlocks, not just some of them.
- vacuity 2y ago
  (I should caveat what I'm about to say that I'm primarily concerned about writing robust and highly performant programs, and while I believe it should be a focus broadly, it's a practical niche.) That's the thing, though. It's arguably even more important for distributed code. If we abstract away the state machine too much, it becomes difficult to reason about the code precisely because of abstraction. The complexity that was present explicitly in the state machine will just cause confusing behavior in the abstracted version. Using lightweight threads or another high level abstraction that approximates blocking code will allow getting a program out faster, but lower quality at that. Two examples to illustrate my point: first, you mention scatter-gather, but the base concept is orthogonal to sync/async. However, I/O is characteristically async, and therefore the underlying mechanisms are async anyways. Second, io_uring is showing that async can be good for performance while not being a difficult interface. Sync code makes things easier to reason about but also kinda not. I think the big issues with async are that the OS hasn't done a good job allocating responsibility of async interfaces and the fundamental difficulty. The former makes async seem less efficient than it could be, which is true in that a sync-over-async interface is better than an async-over-sync-over-async interface, but we should have async interfaces accessible. The latter probably feeds into a bias to not even touch async where mixing async and sync would be the best blend of performance and programmability.

CJefferson 2y ago

I realise I'm picking one sentence out of the middle of this, but this is the one thing I don't feel I understand about async: "But well-formed Futures complete quickly" Does this mean people writing async code don't ever write code which takes a while to compute? I write lots of code which takes 10s of milliseconds, or 100s of milliseconds, or hours. And none of these feel like a good fit for async, as even a 10s of milliseconds piece of work would be enough to start to limit throughput and add unacceptable lag to other connections? This is fine in a multi-threaded server, one thread per connection, you make a difficult request, the server spins for half a second before responding to you -- but it doesn't really fit in a async program?

hinkley 2y ago
Definitely not going to solve a c10k problem with 10ms pauses, that’s for sure.
- CJefferson 2y ago
  True, but some problems just require that long to get the answer the user wants! Some problems take 60 CPU years, but those you don’t expect the user to actively wait while you distribute their task and wait for the results, you send them an email in 3 months or so! :)
dcow 2y ago
The same problem comes up at the end with the proposed `#[blocking]` solution. There is no formal definition of blocking outside of async. It's a colloquial term meaning "takes a relatively longer time to finish". Async is the formalization of making a distinction between grinding the cpu and yielding until something else wakes you up and tells you to keep going. So I guess a well formed Future always adds yield points between hot loops? And then you just trust the runtime to manage execution context such that your yields are noops if there's no contention for execution time and/or to avoid needless context switching.
- CJefferson 2y ago

galaxyLogic 2y ago

Is having the 'await' keyword really necessary? What if every time I call an async function the interpreter would automatically "await" for its result?

vacuity 2y ago
Perhaps (?) it wouldn't be an issue for correctness in most languages, but it would certainly add a performance and debuggability cost. It could be useful in some niche domains, probably.

sqeaky 2y ago

Does the author not know what a "leaky abstraction" is? The first time they referred to memory leaks I thought is a clever joke nested in a jab a hypothetical coder who didn't know, but then in the end they double down in a clearly non-joking way. Then even make a chart showing that calling a normal function from an async function is trivial, that is the opposite of a leaky abstraction. An abstraction is a "leaky abstraction" when it forces some external condition or behavior onto code (ex: calling code that uses the network doesn't work when the network is disconnected). In this case the hurdles of calling async code from normal code, and this author tries to hand waive that with talk of dependencies. There are real benefit and drawbacks to async code and styles. Even if I don't think it is a good abstraction I want to discuss it on the real engineering terms of its actual merits and drawbacks. This article is talking past the engineering decisions and seems like it didn't mean to. maybe I am just being a pedantic jerk.

lalaithion 2y ago
What I think this comes down to, is that in Rust: It is easy but dangerous to call a non-async function from an async function (since the non-async function could block, forcing all your async code to block on this function). It is hard but safe to call an async function from a non-async function. Note that this is language specific; if Rust had “async”, “pure”, and “blocking” as three different function colors, then we would have a very different design. Or, for example, in JavaScript: It is easy and safe to call a non-async function from an async function, since JavaScript exposes no blocking primitives. It is impossible to call an async function from a non-async function, since there is no way to block on the result.

kelnos 2y ago

I don't think the author's understanding of what a "leaky abstraction" is jives with how I've ever used it or heard it used. To me, a leaky abstraction is when there's hidden behavior underneath that makes the observed behavior when using the abstraction to change in ways that aren't intuitive based on the API contract. For example, "everything is a file" on UNIX is a leaky abstraction: read() and write() can behave in wildly different ways, and have significantly different constraints, depending on what the file descriptor refers to (regular file, character device, socket, memfd, etc.). Hell, C stdio/unistd functions themselves are leaky abstractions even when we're only considering regular files on disk. For example, even if your open()+write()+close() succeeds, the data may have not yet hit the disk, and might be lost if the machine crashes right after close() returns. And in some cases an fsync() might not even be enough, if the disk lies to the OS about the state of its own buffers and cache. Put another way, abstractions are about hiding (hopefully) unnecessary implementation details from the person using your abstraction. An abstraction is leaky when those hidden details turn out to actually be important, and the person using your abstraction needs to understand them anyway. The "Dependency Dog" sidebar block that has a mini-rant about AppKit on macOS is further evidence that the author doesn't know what a leaky abstraction is. You might not like AppKit, but every windowing system has canonical APIs to make windows/surfaces and draw things and define behavior and handle input. You just... have to use them, or some abstraction built on top of them. That doesn't make them leaky. (Also, pretty much every UI toolkit out there has restrictions on multi-threading; seems like a weird thing to complain about.) The author's use of "blocking" was also a little confusing. To me, "blocking" means making a call that's going to sit and spin while waiting on I/O (or some other slow/contended resource), but the author uses it to mean "a function that is not declared async". The opposite of async is sync, not blocking.

lalaithion 2y ago
Rust doesn’t have a way in the type system to tell if a function that isn’t declared async is going to block or not. Their point is that any of those functions could block, and having them block wouldn’t be an obvious breaking change, and so you need to know about their internals to know if they’re okay to call.

WiSaGaN 2y ago

All abstractions are leaky, but some are useful. You can't have a perfect abstraction. Your abstraction is always a compromise of various goals you want to achieve.

nice_byte 2y ago

just document your api's thread safety and let me handle it. don't make me deal with this async/await bullshit. multithreaded programming is hard enough as it is.

hesus_ruiz 2y ago

The author must have specified in the title: "Blocking code IN RUST is a leaky abstraction". I am so happy to be using Go for most programs! It has one of the best concurrency implementations that I have ever seen (Erlang permitting ...). And for most of my programs, the garbage collection is not a problem. It's the opposite: simplifies programming incredibly. No blocking versus async problems. No libraries for async are needed. Just concurrency (which has its own problems, but this is another story ...).

lfmunoz4 2y ago

Articles like this remind me how Kotlin is probably 10 times easier and concise than Rust and C++. Kind of off topic but has anyone noticed difference in code quality and documentation from projects in C++ / Rust / Kotlin / JavaScript libraries that have the same features? Recently was working with Seastar which is Tokio / Kotlin Coroutine equivalent in C++ and seems crazy how far behind it is from Kotlin Coroutines. The documentation the example, the boiler plate etc. Don't think it is the language themselves but the language culture. Seems like C++ libraries are just designed to be more difficult. Less documentation, no comments, flat folder structures with each file having with 1000s of lines of code. They expect you to go read through source code etc.