7 ms·

> I wonder why we went down that whole async/await craze with so many languages? We addressed this, albeit very briefly, in the Alternatives section of the JEP

by pron 5y ago

> I wonder why we went down that whole async/await craze with so many languages? We addressed this, albeit very briefly, in the Alternatives section of the JEP: https://openjdk.java.net/jeps/425 https://openjdk.java.net/jeps/425 There are multiple reasons: 1. Languages that don't already have threads have an implicit assumption built into all existing code that program state cannot change concurrently. This is why scheduling points need to be marked explicitly, and why adding threads — whether user-mode or OS — might break existing code in some very tricky ways. That's the case of JavaScript. 2. Some languages target an IR without control over its backend, making an efficient implementation of user-mode threads difficult, if not impossible. async/await requires only changes to the frontend compiler. That's the case of Kotlin, and, perhaps to a lesser extent, Rust. 3. Some languages have technical features that make implementing user-mode threads efficiently more difficult than in others. Pointers into the stack and careful control over memory allocation make this more challenging in languages like C++ and Rust than in Java.

closeparen 5y ago
How does async/await mitigate #1? Interleaved execution is enough to give you data races; you don't need actual parallelism.
- pron 5y ago
  Yes, but it requires a special call site (transitively, all the way up the stack) that permits the interleaving, and so cannot sneak into existing code that might implicitly assume no interleaving.
  - gpderetta 5y ago
    but good old callback-based code still allows for interleaving and AFAIK JS doesn't require any callsite allocation for that.
    - pron 5y ago
      It does not allow for interleaving. Interleaving means that state can change in the same subroutine.
      - gpderetta 5y ago
        What I mean is that subroutine can observe its own state being changed even after a call of a non-sync marked function if that function directly or indirectly calls into a closure closing over that subroutine state. I.e. IMHO async offers very weak reentrancy guarantees that are better enforced via other means (rust-like lifetimes, immutability annotations, atomic contructs, etc).
noncoml 5y ago
All these are implementation details. The programmers should be seen as “users” of the language. What you give here is a list of excuses on why system X doesn’t do what is best for its users.
- pie_flavor 5y ago
  No, explicit pointers are not an implementation detail. Nor is knowing when you are on an OS thread in a language where you are likely to integrate with OS functions that depend on what thread you are on. Different languages exist for different purposes requiring them to solve problems in different ways.
  - noncoml 5y ago
    Two out of three arguments of the GP are implementation details. You can tell that they are implementation details by the way the language is used. “It’s difficult to do A given B” As a user I don’t care about B. I just want A. For your example, the concept of explicit pointers are orthogonal to threads. That’s why you can have OS threads with explicit pointers. Just because it’s difficult to make them work together doesn’t mean they are incompatible as concepts. It’s still an implementation detail. For example, during Win95 one could argue it’s impossible for a crash program not to crash the whole system. As a user I don’t care what’s going under the hood. I just don’t like it when my Windows 95 app can crash the system. It is an implementation detail
- ecnahc515 5y ago
  Implementation details are often also defacto features, because the behavior may be relied upon by users. Unless you designed the language up front to consider these things, it's often very much a challenge to tell your users that their code is broken, especially if you did not have a language specification clarifying it. For languages like Python for example, this is a big issue, and reason why alternative concurrency patterns to async/await haven't made much progress.
  - noncoml 5y ago
SemanticStrengh 5y ago
A major issue with loom is that it consume much more %CPU https://github.com/ebarlas/project-loom-comparison/blob/main/docs/comparison-plots.png https://github.com/ebarlas/project-loom-comparison/blob/main... Edit no it is more efficient although it consume surprisingly high gpus at higher throughput than the others.
- brokencode 5y ago
  Looking at the graphs, it uses less CPU for a given throughout, so it’s actually more efficient for CPU. It also provides lower latency and higher max throughout. It does seem to require more memory, though.
  - pron 5y ago
    We expect to improve the memory consumption significantly in future releases. Some things had to be cut to make this release.
    - SemanticStrengh 5y ago
      The existing data already looks excellent. I wonder if you could leverage SIMD/the vector api for speeding up some things. Or if value types will have an impact.
- didibus 5y ago
  Wouldn't that be expected when it also delivers more throughput and better latencies? It's handling more requests concurrently, so I'd expect the CPU usage to be higher, how else could it serve more requests faster otherwise?
  - SemanticStrengh 5y ago
    Yes indeed I just find the consumption increase a but abrupt after 10K
signal11 5y ago
Ron, thank you for your work on Loom, it’s very exciting and I’m looking forward to using it in production code! As an aside, Ron had a JUG talk 6 months ago which I found really helpful where he went into more detail about why they chose this approach: https://youtu.be/KmMU5Y_r0Uk https://youtu.be/KmMU5Y_r0Uk (27m20s mark, and from 2m50 there’s a more general introduction to Loom). I’m sure there are other videos/papers as well, but this was a pretty good overview of Java vs other languages’ approach to async.
kevincox 5y ago
I think you are missing a key benefit of async/await. It can be implemented incredibly efficiently. This is because it is "stackless". Or in other words you know the exact amount of "stack" space required and you can allocate this exactly instead of allocating a real stack which can be very much larger. For example if I want to implement a "sleep" with async/await I probably only need to store the wake time as state, if I want to make a virtual thread to do the same I likely need to allocate a large stack just in case I use it. Of course this can be mitigated with stack caching, small stacks, segmented stacks or other tricks. But doing this is still more expensive than knowing how much "stack" you need up-front and allocating only that.
- pron 5y ago
  That's not a benefit of async/await as the same could be done with user-mode threads. In fact, that's what we do with virtual threads. But it might be a benefit of async/await in some particular languages.
  - kevincox 5y ago
    Can you explain how this is done? Is the current stack copied onto the heap (to the size it currently is)? How are new frames allocated once a thread is suspended?
    - pron 5y ago
      A portion of the stack is copied to the heap when the virtual thread is suspended, and upon successive yields those "stack chunks" are either reused or new ones allocated and form a linked list. When resuming a virtual thread, however, we don't copy its entire stack back from the heap to the stack, but we do it lazily, by installing a "return barrier" by patching the return address, so as you return from a method, its caller (or several callers) is lazily "thawed" back from the heap. This copying of small chunks of memory into a region that's likely in the cache is very efficient. The entire mechanism is rather efficient because in Java we don't have pointers into the stack, so we don't need to pin anything to a specific address, and stacks can be freely moved around.

by pron 5y ago

closeparen 5y ago

How does async/await mitigate #1? Interleaved execution is enough to give you data races; you don't need actual parallelism.

pron 5y ago
Yes, but it requires a special call site (transitively, all the way up the stack) that permits the interleaving, and so cannot sneak into existing code that might implicitly assume no interleaving.
- gpderetta 5y ago
  but good old callback-based code still allows for interleaving and AFAIK JS doesn't require any callsite allocation for that.
  - pron 5y ago
    It does not allow for interleaving. Interleaving means that state can change in the same subroutine.
    - gpderetta 5y ago
      What I mean is that subroutine can observe its own state being changed even after a call of a non-sync marked function if that function directly or indirectly calls into a closure closing over that subroutine state. I.e. IMHO async offers very weak reentrancy guarantees that are better enforced via other means (rust-like lifetimes, immutability annotations, atomic contructs, etc).

noncoml 5y ago

All these are implementation details. The programmers should be seen as “users” of the language. What you give here is a list of excuses on why system X doesn’t do what is best for its users.

pie_flavor 5y ago
No, explicit pointers are not an implementation detail. Nor is knowing when you are on an OS thread in a language where you are likely to integrate with OS functions that depend on what thread you are on. Different languages exist for different purposes requiring them to solve problems in different ways.
- noncoml 5y ago
  Two out of three arguments of the GP are implementation details. You can tell that they are implementation details by the way the language is used. “It’s difficult to do A given B” As a user I don’t care about B. I just want A. For your example, the concept of explicit pointers are orthogonal to threads. That’s why you can have OS threads with explicit pointers. Just because it’s difficult to make them work together doesn’t mean they are incompatible as concepts. It’s still an implementation detail. For example, during Win95 one could argue it’s impossible for a crash program not to crash the whole system. As a user I don’t care what’s going under the hood. I just don’t like it when my Windows 95 app can crash the system. It is an implementation detail
ecnahc515 5y ago
Implementation details are often also defacto features, because the behavior may be relied upon by users. Unless you designed the language up front to consider these things, it's often very much a challenge to tell your users that their code is broken, especially if you did not have a language specification clarifying it. For languages like Python for example, this is a big issue, and reason why alternative concurrency patterns to async/await haven't made much progress.
- noncoml 5y ago

SemanticStrengh 5y ago

A major issue with loom is that it consume much more %CPU https://github.com/ebarlas/project-loom-comparison/blob/main/docs/comparison-plots.png https://github.com/ebarlas/project-loom-comparison/blob/main... Edit no it is more efficient although it consume surprisingly high gpus at higher throughput than the others.

brokencode 5y ago
Looking at the graphs, it uses less CPU for a given throughout, so it’s actually more efficient for CPU. It also provides lower latency and higher max throughout. It does seem to require more memory, though.
- pron 5y ago
  We expect to improve the memory consumption significantly in future releases. Some things had to be cut to make this release.
  - SemanticStrengh 5y ago
    The existing data already looks excellent. I wonder if you could leverage SIMD/the vector api for speeding up some things. Or if value types will have an impact.
didibus 5y ago
Wouldn't that be expected when it also delivers more throughput and better latencies? It's handling more requests concurrently, so I'd expect the CPU usage to be higher, how else could it serve more requests faster otherwise?
- SemanticStrengh 5y ago
  Yes indeed I just find the consumption increase a but abrupt after 10K

signal11 5y ago

Ron, thank you for your work on Loom, it’s very exciting and I’m looking forward to using it in production code! As an aside, Ron had a JUG talk 6 months ago which I found really helpful where he went into more detail about why they chose this approach: https://youtu.be/KmMU5Y_r0Uk https://youtu.be/KmMU5Y_r0Uk (27m20s mark, and from 2m50 there’s a more general introduction to Loom). I’m sure there are other videos/papers as well, but this was a pretty good overview of Java vs other languages’ approach to async.

kevincox 5y ago

I think you are missing a key benefit of async/await. It can be implemented incredibly efficiently. This is because it is "stackless". Or in other words you know the exact amount of "stack" space required and you can allocate this exactly instead of allocating a real stack which can be very much larger. For example if I want to implement a "sleep" with async/await I probably only need to store the wake time as state, if I want to make a virtual thread to do the same I likely need to allocate a large stack just in case I use it. Of course this can be mitigated with stack caching, small stacks, segmented stacks or other tricks. But doing this is still more expensive than knowing how much "stack" you need up-front and allocating only that.

pron 5y ago
That's not a benefit of async/await as the same could be done with user-mode threads. In fact, that's what we do with virtual threads. But it might be a benefit of async/await in some particular languages.
- kevincox 5y ago
  Can you explain how this is done? Is the current stack copied onto the heap (to the size it currently is)? How are new frames allocated once a thread is suspended?
  - pron 5y ago
    A portion of the stack is copied to the heap when the virtual thread is suspended, and upon successive yields those "stack chunks" are either reused or new ones allocated and form a linked list. When resuming a virtual thread, however, we don't copy its entire stack back from the heap to the stack, but we do it lazily, by installing a "return barrier" by patching the return address, so as you return from a method, its caller (or several callers) is lazily "thawed" back from the heap. This copying of small chunks of memory into a region that's likely in the cache is very efficient. The entire mechanism is rather efficient because in Java we don't have pointers into the stack, so we don't need to pin anything to a specific address, and stacks can be freely moved around.