4 ms·

Kotlin adds a huge overhead for async code. I measured over 50% slower for function heavy code. The otherwise problem is that you end up with two worlds, the n

Kotlin adds a huge overhead for async code. I measured over 50% slower for function heavy code. The otherwise problem is that you end up with two worlds, the non-async ad the async ones, and they can't really call each other without a lot of problems. The new JVM features will allow you to write async code while still being able to call any functions, that'll make it actually useful.

jillesvangurp 4y ago
My servers mostly idle waiting for IO (like most servers do), so I never saw any slowdowns with this; nor would I expect to. Also this is not really an issue on e.g. Android where a lot of UI code is inherently asynchronous and co-routines are used a lot. And it's a complete non issue on kotlin-js as well where co-routines and js promises are essentially the same thing. I use it; it's fine. Kotlin's co-routines are simply syntactic sugar for simple callbacks but without the boiler plate. If it has some kind of success / fail callback mechanism, you can create a co-routine from it. Futures, Promises, a Flux, whatever. That's all that happens and the co-routine library comes with extension functions for all these. I'm not sure what you measured relative to what. But I bet it is some blocking/expensive code actually taking a while before it suspends or something similarly sub-optimal. Of course the key bottleneck with virtual threads would be the scheduler loop that runs all these co-routines. The main contribution of Loom is moving that stuff into the JDK and possibly optimizing the hell out of that. But other than that it does something very similar to what other green/light weight threads have to do necessarily. Basically any time any virtual thread does something expensive/blocking, all the other virtual threads suffer. If you see a browser app freeze for a bit, that's usually what happened. Loom adds a thing called continuations which literally have a suspend method that tell Loom that it is now safe to switch to another Fiber. Failure to use suspend correctly, will result in the same stuttering behavior on Loom: things will freeze or block until whatever is running calls suspend: only 1 virtual thread executes at the same time. That's the whole point. For non blocking IO this is fine. If you are doing computationally intensive stuff, you either need to suspend frequently or use some thread pool. Otherwise the virtual thread will hog the CPU whenever it runs. As for the two worlds, I can call any function from a suspend function. Just not the other way around without creating a co-routine context. What Loom adds is basically a better light weight thread scheduler. Under the hood it likely does many of the same things that Kotlin already did.
- Skinney 4y ago
  > Basically any time any virtual thread does something expensive/blocking, all the other virtual threads suffer. Most Java APIs like socket and file have been re-written to do this for you under the hood. You're not expected to manually call suspend. > Just not the other way around without creating a co-routine context. This goes beyond function calling though. Same restrictions apply for implementing interfaces or passing functions as arguments, right? With loom, there is just one world. A function is a function is a function. It's good that both Java and Kotlin can take advantage of this improved model. > Under the hood it likely does many of the same things that Kotlin already did. Maybe, but this also extends to debugging, heap dumps and stack traces. Having this builtin has a lot of benefits.
- lokedhs 4y ago
  It's heavily CPU intensive code that needs to perform async operations on rare occasions. Since async adds overhead in every function call, code that calls a lot of functions will be slower. The problem with the Kotlin implementation is that if you ever want to be able to call a single async function, every call higher in the stack must also be async (unless you have a separate runner, but that's not always possible, and even if it is you can have surprising results)
- fesc 4y ago
  > Kotlin's co-routines are simply syntactic sugar for simple callbacks That's only partly true. And yeah they're callbacks but not like e.g. we know from JS. As far as I know CPS basically means that the Kotlin compiler turns a function like that: suspend fun myFunction(param: String) { val asyncVal = someSuspendingFunction() val asyncVal2 = anotherSuspendingFunctio(asyncVal) return someCalculation(asyncVal2) } Into a modified method and *class* like that: // The different parts of the function get put into different ifs // The state that is normally in the JVM stack frame is holded in a custom class which is a subclass of https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines/-continuation/ // Some integer tracks where in the function we are fun myFunction(param: String, continuation: Something<MyFunctionClass>?) { continuation = continuation ?: MyFunctionContinuation(param) if (continuation.state == 0) { contination.state = 1 someSuspendingFunction(continuation) return SUSPEND } if (continuation.state == 1) { // The result mechanism is in the Contiuation interface/abstract class. It can be used to later receive the async result // of callig annother suspending function continuation.asyncVal = contiuation.getResult() anotherSuspendingFunction(continuation.asyncVal, continuation) return SUSPEND } if (continuation.state == 2) { continuation.asyncVal2 = contiuation.getResult() return FINISHED(someCalculation(continuation.asyncVal2)) } } class MyFunctionContinuation: Continuation { // Function arguments val param: String // Every suspending function needs this. It tracks where in the function we currently are val state: Int = 0 // Local variables inside the function. Those would normally live in the JVM function stack // But the Coroutine runtime needs to be able to "rehydrate" the JVM function stack once a non-blocking call finishes var asyncVal: <Don't remember> var asyncVal2: <Don't remember> } I hope the basic idea comes across. I read this sometime in some book about coroutines. But you can see that there is some overhead. The basic idea is to replicate the function call stack in a custom datastructure. This way we can be deep inside a normal JVM callstack and also have a separate Kotlin Coroutine callstack. Once the coroutine wants to suspend it can simply return from the JVM callstack but the Kotlin callstack is preserved. Later when we want to continue we can call the same function again, with the Kotlin callstack now containing the result of the async operation. If the JVM can do this natively we wouldn't need this second Kotlin coroutine callstack because the native JVM callstack would support this.

jillesvangurp 4y ago

My servers mostly idle waiting for IO (like most servers do), so I never saw any slowdowns with this; nor would I expect to. Also this is not really an issue on e.g. Android where a lot of UI code is inherently asynchronous and co-routines are used a lot. And it's a complete non issue on kotlin-js as well where co-routines and js promises are essentially the same thing. I use it; it's fine. Kotlin's co-routines are simply syntactic sugar for simple callbacks but without the boiler plate. If it has some kind of success / fail callback mechanism, you can create a co-routine from it. Futures, Promises, a Flux, whatever. That's all that happens and the co-routine library comes with extension functions for all these. I'm not sure what you measured relative to what. But I bet it is some blocking/expensive code actually taking a while before it suspends or something similarly sub-optimal. Of course the key bottleneck with virtual threads would be the scheduler loop that runs all these co-routines. The main contribution of Loom is moving that stuff into the JDK and possibly optimizing the hell out of that. But other than that it does something very similar to what other green/light weight threads have to do necessarily. Basically any time any virtual thread does something expensive/blocking, all the other virtual threads suffer. If you see a browser app freeze for a bit, that's usually what happened. Loom adds a thing called continuations which literally have a suspend method that tell Loom that it is now safe to switch to another Fiber. Failure to use suspend correctly, will result in the same stuttering behavior on Loom: things will freeze or block until whatever is running calls suspend: only 1 virtual thread executes at the same time. That's the whole point. For non blocking IO this is fine. If you are doing computationally intensive stuff, you either need to suspend frequently or use some thread pool. Otherwise the virtual thread will hog the CPU whenever it runs. As for the two worlds, I can call any function from a suspend function. Just not the other way around without creating a co-routine context. What Loom adds is basically a better light weight thread scheduler. Under the hood it likely does many of the same things that Kotlin already did.

Skinney 4y ago
> Basically any time any virtual thread does something expensive/blocking, all the other virtual threads suffer. Most Java APIs like socket and file have been re-written to do this for you under the hood. You're not expected to manually call suspend. > Just not the other way around without creating a co-routine context. This goes beyond function calling though. Same restrictions apply for implementing interfaces or passing functions as arguments, right? With loom, there is just one world. A function is a function is a function. It's good that both Java and Kotlin can take advantage of this improved model. > Under the hood it likely does many of the same things that Kotlin already did. Maybe, but this also extends to debugging, heap dumps and stack traces. Having this builtin has a lot of benefits.
lokedhs 4y ago
It's heavily CPU intensive code that needs to perform async operations on rare occasions. Since async adds overhead in every function call, code that calls a lot of functions will be slower. The problem with the Kotlin implementation is that if you ever want to be able to call a single async function, every call higher in the stack must also be async (unless you have a separate runner, but that's not always possible, and even if it is you can have surprising results)
fesc 4y ago
> Kotlin's co-routines are simply syntactic sugar for simple callbacks That's only partly true. And yeah they're callbacks but not like e.g. we know from JS. As far as I know CPS basically means that the Kotlin compiler turns a function like that: suspend fun myFunction(param: String) { val asyncVal = someSuspendingFunction() val asyncVal2 = anotherSuspendingFunctio(asyncVal) return someCalculation(asyncVal2) } Into a modified method and *class* like that: // The different parts of the function get put into different ifs // The state that is normally in the JVM stack frame is holded in a custom class which is a subclass of https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.coroutines/-continuation/ // Some integer tracks where in the function we are fun myFunction(param: String, continuation: Something<MyFunctionClass>?) { continuation = continuation ?: MyFunctionContinuation(param) if (continuation.state == 0) { contination.state = 1 someSuspendingFunction(continuation) return SUSPEND } if (continuation.state == 1) { // The result mechanism is in the Contiuation interface/abstract class. It can be used to later receive the async result // of callig annother suspending function continuation.asyncVal = contiuation.getResult() anotherSuspendingFunction(continuation.asyncVal, continuation) return SUSPEND } if (continuation.state == 2) { continuation.asyncVal2 = contiuation.getResult() return FINISHED(someCalculation(continuation.asyncVal2)) } } class MyFunctionContinuation: Continuation { // Function arguments val param: String // Every suspending function needs this. It tracks where in the function we currently are val state: Int = 0 // Local variables inside the function. Those would normally live in the JVM function stack // But the Coroutine runtime needs to be able to "rehydrate" the JVM function stack once a non-blocking call finishes var asyncVal: <Don't remember> var asyncVal2: <Don't remember> } I hope the basic idea comes across. I read this sometime in some book about coroutines. But you can see that there is some overhead. The basic idea is to replicate the function call stack in a custom datastructure. This way we can be deep inside a normal JVM callstack and also have a separate Kotlin Coroutine callstack. Once the coroutine wants to suspend it can simply return from the JVM callstack but the Kotlin callstack is preserved. Later when we want to continue we can call the same function again, with the Kotlin callstack now containing the result of the async operation. If the JVM can do this natively we wouldn't need this second Kotlin coroutine callstack because the native JVM callstack would support this.