17 ms·

It's a leaky abstraction (https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/ https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a.

It's a leaky abstraction (https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/ https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...), but maybe there is no helping it and for some reason it is a necessary tradeoff for performance?

mtsr 3y ago
Maybe it’s actually a non-leaky abstraction because it makes the async-nature explicit. The alternative is hiding it, but it’s still going to affect your code, making that effectively a leaky abstraction.
- luke-stanley 3y ago
  Maybe there could be something like a aprint() wrapper, if the authors wanted to make the async nature explicit? Or something else, probably not this for one of the most common things a programmer must do.
  - steveklabnik 3y ago
    Why is aprint “non leaky” but print.await “leaky”?
    - luke-stanley 3y ago
      Hey Steve, I wouldn't say that "print.await" is a leak abstraction. I think "print.await" is explicit and that's good, it communicates it's abstraction fairly clearly, presumably following a pattern used commonly in this, imagined language. I suppose that a wrapper like "aprint" (a convenience function labelled async, like with an "a" prefix), would be a bit better than having people continually try using print, not await it, and not getting the expected output in stdout (or whatever stream it's sent to), while they are in the middle of trying to test something or otherwise get something working because I'm of the opinion that common things should be easy. Maybe "people would generally expect a print function to just work and not return a promise or something" is an abstraction? "aprint" might actually be the wrong name I'm not sure I've really thought about it right.
      - steveklabnik 3y ago
        I agree with you personally on print.await; maybe I replied to the wrong person on this thread, ha!.
  - luke-stanley 3y ago
    Actually, surely you'd want an async print and a synchronous print with clear labels? aprint might be interpreted as an async print, not an awaited print, which is what I meant. Maybe this goes against "everything is async". Anyhow a better name could be print_awaited, so a "print_awaited" could be used directly without the extra syntax to await it (saving some autocomplete time?), it's still long though.
- gpderetta 3y ago
  Why is I/O so special that need to be explicitly marked across the call stack? What about memory allocation, that can arbitrarily delay a process? Should allocating functions be transitively annotated? What about functions that lock a mutex or wait on some synchronisation primitive? What about those that signal a synchronization primitive? What about floating points, that can raise exceptions? What about panicking functions? Either all side effects should be marked or none should. Ret-connecting await annotations as an useful feature instead of a necessary evil is baffling.
  - avianlyric 3y ago
    I/O tends to be the slowest operations your software can perform, and also the riskiest, because you’re dependent on so many different underlying components working correctly. Everything from the kernel syscall, to the device driver, the device itself, and potentially devices attached to device that’s attached to your computer. In short IO operations are a complete shit show of possible problems, that can all occur while your software is suspended in a syscall. Memory allocation by comparison are extremely quick, and generally very reliable. Your system’s memory subsystem isn’t a smorgasbord of different memory drivers and controllers. It one memory system, taking to one memory controller, via an API that been standardised for decades, and where every implementation of that API is basically tested to the extreme every time a computer turns on. That’s assuming your language even bother asking the OS for memory on every allocation, which it probably doesn’t. Most language runtimes request large blocks of memory from the OS, then allocate out of those block on demand. So most “allocating functions” never result in syscall at all.
    - gpderetta 3y ago
      Memory allocation can literally fail for reasons completely outside the control of the application (for example because the OS enforces a maximum virtual memory size on the application). The fact the most allocations are fulfilled via internal pools is immaterial, at some point the allocator needs to ask the OS for more memory. This parallels the way that most I/O doesn't actually performs syscalls because of buffering. Also allocations might end up performing arbitrary I/O indirectly if the OS needs to flush dirty pages to disk to free up memory.
klabb3 3y ago
To be precise: the contract depends on the implementation. Here’s an example: I write an in memory kv cache. It’s in memory so no async needed. Now I create a trait and implement a second version with file backing. Now the children are crying because async needs to be retroactively added and also why, makes no sense etc.
- naasking 3y ago
  It does make sense if you want other types of resources, like time and memory, to also be part of the contract. Async annotations let you do this but hiding the asynchrony does not.
  - klabb3 3y ago
    Make sense might be an overstatement but ok. Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences? They’re just regular functions in most languages including Rust. Perhaps anything involving syscalls should be exposed and contractual. I doubt it, but maybe it’s important for some obscure ownership-of-resources reason. But then why the inconsistency between traditional and pooled syscalls? The only difference is whether the runtime sits in the kernel or in user space. The only one who should care is the runtime folks. My take has been for years that this is throwing complexity over the fence and shaming users for not getting it. And even when they do get it, they Arc<Mutex> everything anyways in which case you are throwing the baby out with the bathwater (RAII, single ownership, static borrowing).
    - naasking 3y ago
      > Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences? They’re just regular functions in most languages including Rust. Because the kernel doesn't expose that contract, so they don't have that behaviour. > The only difference is whether the runtime sits in the kernel or in user space. In other words, what contracts you have control over and are allowed to provide. > My take has been for years that this is throwing complexity over the fence and shaming users for not getting it. I'm sure how we got here would seem baffling if you're going to just ignore the history of the C10K problem that led us to this point. You can of course paper over any platform-specific quirks and provide a uniform interface if you like, at the cost of some runtime overhead, but eliminating as much of this kind of implicit runtime overhead as possible seems like one of Rust's goals. Other languages, like Go, have a different set of goals and so can provide that uniform interface. It's probably also possible to have some of that uniform interface via a crate, if some were so inclined, but that doesn't mean it should be in the core which has a broader goal.

mtsr 3y ago

Maybe it’s actually a non-leaky abstraction because it makes the async-nature explicit. The alternative is hiding it, but it’s still going to affect your code, making that effectively a leaky abstraction.

luke-stanley 3y ago
Maybe there could be something like a aprint() wrapper, if the authors wanted to make the async nature explicit? Or something else, probably not this for one of the most common things a programmer must do.
- steveklabnik 3y ago
  Why is aprint “non leaky” but print.await “leaky”?
  - luke-stanley 3y ago
    Hey Steve, I wouldn't say that "print.await" is a leak abstraction. I think "print.await" is explicit and that's good, it communicates it's abstraction fairly clearly, presumably following a pattern used commonly in this, imagined language. I suppose that a wrapper like "aprint" (a convenience function labelled async, like with an "a" prefix), would be a bit better than having people continually try using print, not await it, and not getting the expected output in stdout (or whatever stream it's sent to), while they are in the middle of trying to test something or otherwise get something working because I'm of the opinion that common things should be easy. Maybe "people would generally expect a print function to just work and not return a promise or something" is an abstraction? "aprint" might actually be the wrong name I'm not sure I've really thought about it right.
    - steveklabnik 3y ago
      I agree with you personally on print.await; maybe I replied to the wrong person on this thread, ha!.
- luke-stanley 3y ago
  Actually, surely you'd want an async print and a synchronous print with clear labels? aprint might be interpreted as an async print, not an awaited print, which is what I meant. Maybe this goes against "everything is async". Anyhow a better name could be print_awaited, so a "print_awaited" could be used directly without the extra syntax to await it (saving some autocomplete time?), it's still long though.
gpderetta 3y ago
Why is I/O so special that need to be explicitly marked across the call stack? What about memory allocation, that can arbitrarily delay a process? Should allocating functions be transitively annotated? What about functions that lock a mutex or wait on some synchronisation primitive? What about those that signal a synchronization primitive? What about floating points, that can raise exceptions? What about panicking functions? Either all side effects should be marked or none should. Ret-connecting await annotations as an useful feature instead of a necessary evil is baffling.
- avianlyric 3y ago
  I/O tends to be the slowest operations your software can perform, and also the riskiest, because you’re dependent on so many different underlying components working correctly. Everything from the kernel syscall, to the device driver, the device itself, and potentially devices attached to device that’s attached to your computer. In short IO operations are a complete shit show of possible problems, that can all occur while your software is suspended in a syscall. Memory allocation by comparison are extremely quick, and generally very reliable. Your system’s memory subsystem isn’t a smorgasbord of different memory drivers and controllers. It one memory system, taking to one memory controller, via an API that been standardised for decades, and where every implementation of that API is basically tested to the extreme every time a computer turns on. That’s assuming your language even bother asking the OS for memory on every allocation, which it probably doesn’t. Most language runtimes request large blocks of memory from the OS, then allocate out of those block on demand. So most “allocating functions” never result in syscall at all.
  - gpderetta 3y ago
    Memory allocation can literally fail for reasons completely outside the control of the application (for example because the OS enforces a maximum virtual memory size on the application). The fact the most allocations are fulfilled via internal pools is immaterial, at some point the allocator needs to ask the OS for more memory. This parallels the way that most I/O doesn't actually performs syscalls because of buffering. Also allocations might end up performing arbitrary I/O indirectly if the OS needs to flush dirty pages to disk to free up memory.

klabb3 3y ago

To be precise: the contract depends on the implementation. Here’s an example: I write an in memory kv cache. It’s in memory so no async needed. Now I create a trait and implement a second version with file backing. Now the children are crying because async needs to be retroactively added and also why, makes no sense etc.

naasking 3y ago
It does make sense if you want other types of resources, like time and memory, to also be part of the contract. Async annotations let you do this but hiding the asynchrony does not.
- klabb3 3y ago
  Make sense might be an overstatement but ok. Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences? They’re just regular functions in most languages including Rust. Perhaps anything involving syscalls should be exposed and contractual. I doubt it, but maybe it’s important for some obscure ownership-of-resources reason. But then why the inconsistency between traditional and pooled syscalls? The only difference is whether the runtime sits in the kernel or in user space. The only one who should care is the runtime folks. My take has been for years that this is throwing complexity over the fence and shaming users for not getting it. And even when they do get it, they Arc<Mutex> everything anyways in which case you are throwing the baby out with the bathwater (RAII, single ownership, static borrowing).
  - naasking 3y ago
    > Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences? They’re just regular functions in most languages including Rust. Because the kernel doesn't expose that contract, so they don't have that behaviour. > The only difference is whether the runtime sits in the kernel or in user space. In other words, what contracts you have control over and are allowed to provide. > My take has been for years that this is throwing complexity over the fence and shaming users for not getting it. I'm sure how we got here would seem baffling if you're going to just ignore the history of the C10K problem that led us to this point. You can of course paper over any platform-specific quirks and provide a uniform interface if you like, at the cost of some runtime overhead, but eliminating as much of this kind of implicit runtime overhead as possible seems like one of Rust's goals. Other languages, like Go, have a different set of goals and so can provide that uniform interface. It's probably also possible to have some of that uniform interface via a crate, if some were so inclined, but that doesn't mean it should be in the core which has a broader goal.