6 ms·

I'm personally not so sure. We already had something like that before (M:N threads), and the world moved away from it, towards letting the kernel manage everyth

by cesarb 4y ago

I'm personally not so sure. We already had something like that before (M:N threads), and the world moved away from it, towards letting the kernel manage everything (1:1 threads). So I'd expect instead that operating system kernels gain whatever features are missing for scaling to a higher number of threads, and everything once again goes back to each programming language thread corresponding to one kernel thread.

kaba0 4y ago
M:N is not the interesting aspect of virtual threads at all, automagically turning blocking operations into non-blocking is - which has not really been tried before (with erlang and go being the first).
- cryptonector 4y ago
  GNU Pth had "automagically turning blocking operations into non-blocking" ages ago, and it wasn't the first. I think what you probably had in mind is that C libraries of the 90s that did M:N threading didn't turn blocking operations into non-blocking? Using blocking operations to switch contexts is really nothing new. Heck, the cooperative multi-tasking systems of the 80s (Mac, Amiga) all essentially did that for processes (not threads), and so did Unix in the 70s.
  - kaba0 4y ago
    > I think what you probably had in mind is that C libraries of the 90s that did M:N threading didn't turn blocking operations into non-blocking Yes, mostly, though my history knowledge is definitely lacking so do correct me if I’m wrong. But you are right, there was nothing fundamentally missing, probably just no good OS support for non-blocking IO calls in the early days? Though probably the IO-CPU ratio was also different, so the benefits were not as big?
    - cryptonector 4y ago
      There were bad experiences with the M:N threading of the 90s in Solaris' and others' C libraries. Making those libraries make every file descriptor non-blocking behind the programmer's back was a tricky thing. Think about inheritance of file descriptors via fork() and exec() -- you could have one threaded process sharing an FD with a non-threaded process, and now even non-threaded processes' C library would have to poll(), and now add static linking with older C libraries to the mix and it just couldn't be done. So it wasn't done.
      - cryptonector 4y ago
        Which makes me wonder why this can be done in Java or Erlang, and the answer is that those tend to be walled gardens from which one does not fork/exec.
- ghostwriter 4y ago
  > automagically turning blocking operations into non-blocking is - which has not really been tried before (with erlang and go being the first). sorry, but Haskell has had it way before Go, with proper STM too. Neither Erlang nor Go are offering the same level of ergonomics for compile-time checked M:N threading.
- severino 4y ago
  > M:N is not the interesting aspect of virtual threads at all, automagically turning blocking operations into non-blocking is I'm not very into this Loom virtual threads thing, but... what's the difference between this automagically conversion of blocking into non-blocking in a M:N model and a 1:1 one? I mean, couldn't the same be done with normal threads too?
  - kaba0 4y ago
    Well, to a degree this is also done by the OS, IO syscalls are frequent locations where the OS scheduler might decide to schedule another thread, but this is a very slow context switch (flushing caches, including TLB, the switch to kernel mode and back, and since heartbleed and alia it is even more expensive). Loom implements every IO on top of a more modern async OS calls, and these virtual thread context switches are on the order of function calls, so the overhead and number of switches that can happen are much much lower.
    - severino 4y ago
      Thanks!
mike_hearn 4y ago
The problem of scaling threads up further is fundamental and not really solvable by more kernel features. JVM virtual threads can be efficient because the runtime has complete knowledge of the executing code and stack layouts, how the heap is laid out, how the GC works and it can control how code is compiled. The kernel can't do any of these things - it has to assume a process is a black box that could do anything with its stacks, could be compiled by anything and so on. Note that this advantage obviously goes away the moment you call into native code. Then the JVM is in the same position as the kernel. It doesn't control the compiler or the stack any more, and so that's why a virtual thread becomes "pinned" at that point and you lose the efficiency (the JVM needs to acquire more kernel threads). Fortunately though the JVM ecosystem doesn't rely on native code all that much, so it should be rare in practice. This advantage can be brought to other non-Java languages too via Truffle. Truffle languages are reimplemented on top of Java and when programs call into native code they have the option of calling into JIT compiled LLVM bitcode instead of real native code (or indeed any JVM bytecode library). In that situation the JVM remains in control and so things should in theory still be Loom-able. Not sure if that's currently true in practice, but it could be.
- cesarb 4y ago
  > JVM virtual threads can be efficient because the runtime has complete knowledge of the executing code and stack layouts, how the heap is laid out, how the GC works and it can control how code is compiled. Forgive me for staying doubtful, but I recall hearing this same "the JVM can be very fast and efficient because its JIT has complete knowledge and control" spiel back in the 90s, and back then, anyone could clearly see that the JVM was not as fast compared to pre-compiled native code as it was being promised. > The kernel can't do any of these things - it has to assume a process is a black box that could do anything with its stacks, could be compiled by anything and so on. The kernel has to assume nothing; it can dictate how userspace processes behave. As an example, a process which plays too many games with its stacks, without kernel cooperation, will quickly find out that signals share the same stack unless the kernel is told to use an alternate stack. A process which uses a register declared in the platform ABI as being for kernel use will find out that it can be unpredictably overwritten on a context switch. There are things like shadow stacks and segment register bases which can only be manipulated when the kernel allows it. And so on. Of course, for compatibility reasons, the current ABI allows userspace processes to do a lot of unpredictable things, but nothing prevents a new "highly scalable threads" process ABI, with stricter rules, from being developed if necessary. Or it could be that only a few cooperative additions to the userspace to kernel ABI are necessary; we already have things like the many options to the clone() system calls, the futex system call, restartable sequences, etc.

by cesarb 4y ago

kaba0 4y ago

M:N is not the interesting aspect of virtual threads at all, automagically turning blocking operations into non-blocking is - which has not really been tried before (with erlang and go being the first).

cryptonector 4y ago
GNU Pth had "automagically turning blocking operations into non-blocking" ages ago, and it wasn't the first. I think what you probably had in mind is that C libraries of the 90s that did M:N threading didn't turn blocking operations into non-blocking? Using blocking operations to switch contexts is really nothing new. Heck, the cooperative multi-tasking systems of the 80s (Mac, Amiga) all essentially did that for processes (not threads), and so did Unix in the 70s.
- kaba0 4y ago
  > I think what you probably had in mind is that C libraries of the 90s that did M:N threading didn't turn blocking operations into non-blocking Yes, mostly, though my history knowledge is definitely lacking so do correct me if I’m wrong. But you are right, there was nothing fundamentally missing, probably just no good OS support for non-blocking IO calls in the early days? Though probably the IO-CPU ratio was also different, so the benefits were not as big?
  - cryptonector 4y ago
    There were bad experiences with the M:N threading of the 90s in Solaris' and others' C libraries. Making those libraries make every file descriptor non-blocking behind the programmer's back was a tricky thing. Think about inheritance of file descriptors via fork() and exec() -- you could have one threaded process sharing an FD with a non-threaded process, and now even non-threaded processes' C library would have to poll(), and now add static linking with older C libraries to the mix and it just couldn't be done. So it wasn't done.
    - cryptonector 4y ago
      Which makes me wonder why this can be done in Java or Erlang, and the answer is that those tend to be walled gardens from which one does not fork/exec.
ghostwriter 4y ago
> automagically turning blocking operations into non-blocking is - which has not really been tried before (with erlang and go being the first). sorry, but Haskell has had it way before Go, with proper STM too. Neither Erlang nor Go are offering the same level of ergonomics for compile-time checked M:N threading.
severino 4y ago
> M:N is not the interesting aspect of virtual threads at all, automagically turning blocking operations into non-blocking is I'm not very into this Loom virtual threads thing, but... what's the difference between this automagically conversion of blocking into non-blocking in a M:N model and a 1:1 one? I mean, couldn't the same be done with normal threads too?
- kaba0 4y ago
  Well, to a degree this is also done by the OS, IO syscalls are frequent locations where the OS scheduler might decide to schedule another thread, but this is a very slow context switch (flushing caches, including TLB, the switch to kernel mode and back, and since heartbleed and alia it is even more expensive). Loom implements every IO on top of a more modern async OS calls, and these virtual thread context switches are on the order of function calls, so the overhead and number of switches that can happen are much much lower.
  - severino 4y ago
    Thanks!

mike_hearn 4y ago

The problem of scaling threads up further is fundamental and not really solvable by more kernel features. JVM virtual threads can be efficient because the runtime has complete knowledge of the executing code and stack layouts, how the heap is laid out, how the GC works and it can control how code is compiled. The kernel can't do any of these things - it has to assume a process is a black box that could do anything with its stacks, could be compiled by anything and so on. Note that this advantage obviously goes away the moment you call into native code. Then the JVM is in the same position as the kernel. It doesn't control the compiler or the stack any more, and so that's why a virtual thread becomes "pinned" at that point and you lose the efficiency (the JVM needs to acquire more kernel threads). Fortunately though the JVM ecosystem doesn't rely on native code all that much, so it should be rare in practice. This advantage can be brought to other non-Java languages too via Truffle. Truffle languages are reimplemented on top of Java and when programs call into native code they have the option of calling into JIT compiled LLVM bitcode instead of real native code (or indeed any JVM bytecode library). In that situation the JVM remains in control and so things should in theory still be Loom-able. Not sure if that's currently true in practice, but it could be.

cesarb 4y ago
> JVM virtual threads can be efficient because the runtime has complete knowledge of the executing code and stack layouts, how the heap is laid out, how the GC works and it can control how code is compiled. Forgive me for staying doubtful, but I recall hearing this same "the JVM can be very fast and efficient because its JIT has complete knowledge and control" spiel back in the 90s, and back then, anyone could clearly see that the JVM was not as fast compared to pre-compiled native code as it was being promised. > The kernel can't do any of these things - it has to assume a process is a black box that could do anything with its stacks, could be compiled by anything and so on. The kernel has to assume nothing; it can dictate how userspace processes behave. As an example, a process which plays too many games with its stacks, without kernel cooperation, will quickly find out that signals share the same stack unless the kernel is told to use an alternate stack. A process which uses a register declared in the platform ABI as being for kernel use will find out that it can be unpredictably overwritten on a context switch. There are things like shadow stacks and segment register bases which can only be manipulated when the kernel allows it. And so on. Of course, for compatibility reasons, the current ABI allows userspace processes to do a lot of unpredictable things, but nothing prevents a new "highly scalable threads" process ABI, with stricter rules, from being developed if necessary. Or it could be that only a few cooperative additions to the userspace to kernel ABI are necessary; we already have things like the many options to the clone() system calls, the futex system call, restartable sequences, etc.