4 ms·

What you're arguing for is task preemption, and it has a global cost. If you're willing to pay global costs, Rust is probably not the right language for you.

jlokier 5y ago
Technically it's not pre-emption. It's activating parallelism on another CPU core when a time condition is triggered, in order to reduce tail latencies of unrelated tasks and prevent starvation. Nothing is directly pre-empted by that event. But let's run with pre-emption, because that does apply on single cores. No, it doesn't work out to be a global cost, assuming you mean performance (as opposed to developer cognitive cost). I have experience with this - the game engine. We don't half-ass performance in games like that, it's a central factor. We optimise for the highest performance on middle-of-the-road hardware, and measure constantly. The Rust model is same or worse on each global performance metric that matters in that scenario. The global metrics are things like graphics throughput, tail latency and variance, interactive response, task starvation. So the "cost" of "pre-emption" ranges from zero to negative. I will grant you that pre-emption adds shared resource locking overheads if you're comparing with an async-await model that uses only a single thread for all tasks globally, so locking is not required. (I.e. JavaScript). But that's not the Rust model. Pre-emption has a small, positive local cost, in that you need to check when to pre-empt. But that can be made small. You don't need a kernel to do it, and you don't need to be constantly reading clocks either. The cost is a few nanoseconds per microtask, i.e. book-keeping like with any execution loop, and some periodic interrupts or signals, which you might have anyway. Alternatively you can use performance counters on some CPUs. The actual act of pre-empting is also cheap - it's just a coroutine jump with some book-keeping. But it's also comparatively rare compared with async task transitions, so the cost doesn't really matter anyway.
- littlestymaar 5y ago
  Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow). Go and async Rust are both cooperatively scheduled, switching back and forth only in specific yield points, which reduces the amount of state that needs to be saved. Adding such a yield point is costly too though, that's why Go didn't add them inside tight loops until 1.14.[1] Regarding scheduling and how task-switching is done, async Rust and Go are fundamentally the same model (and BTW, the authors of Tokio never hide that they took a lot of inspiration from the Go runtime). The three big differences are the following: - Go's tasks are stackful, they have their own stack, which is grow-able (the stack is copied into a bigger stack when it's full) - Go's yield points are automatically inserted by the compiler based on heuristics which vary depending on the version of the compiler. Whereas Rust yield point are manually inserted by the developer. This resonate strongly with the philosophy of both languages (best-effort automatic vs explicit manual control leading to the best performance) which can also be seen on the topic of memory allocation (automatic boxing based on escape analysis vs manual Box pointer). - Go only has green thread, which means that if the runtime is failing to keep its promises[1][2], you don't have much alternative. Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload. [1] https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958 [2] https://github.com/golang/go/issues/36365 https://github.com/golang/go/issues/36365
  - jlokier 5y ago
    > Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow). In-userspace stackful context switches are not at all slow in a sane userspace-switching runtime. They are approximately a combination of setjmp() and longjmp(), but without saving all registers. Typically a few tens of nanoseconds or less on a modern CPU. Pre-emptive context switches are nearly always stackful, even if non-pre-emptive (async) context switches are stackless in the same runtime. A pre-emptive context switch in a userspace co-operative scheduling system is slower than non-pre-emptive context switch if it is caused by a dedicated interrupt of some kind. In the case of userspace pre-emption, generally a signal and return, typically single digit microseconds. However, the cost there is mainly the signal. If the pre-emptive context switch is caused by a signal that triggered anyway for another purpose, the actual context switch is, again, cheap. In the model I described (the game environment), pre-emptive context switches are rare. It wouldn't matter if they took longer, because >= 99.99% of context switches are co-operative in that model. The important factor is that no task blows the frame budget or prevents other tasks from running at the full frame rate. In web services a similar target is tail latency, in the presence of diverse tasks you cannot predict in advance. > meaning you can choose what works best for your workload. Exactly. And as I've tried to explain, for some varying, complex workloads whatever you choose statically has poor metrics; it must be adaptive to maintain good timing metrics. > Rust yield point are manually inserted by the developer Indeed, and in the case of NPCs in a game engine, or a large program composed of many libraries written by hundreds of different authors (e.g. some web servers), that causes complex, interdependent performance characteristics, where timing behaviour in one of them ruins performance for everything, unless they are isolated using threads, in which case it's too slow. There is no "the developer" who can ensure this doesn't happen. (Aside, if Rust's type system helped with this, that would be great, the same way types help with other "programming in the large" safety issues, but it doesn't address timing characteristics as far as I know.) > leading to the best performance [..] Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload. You could summarise my point as: "For some dynamic workloads, especially in timing-sensitive large programs with many components working independently and unpredictably, neither async tasks or OS threads perform best for your workload (or even adequately sometimes). The optimal (or required) combination requires some dynamic responses, and cannot be achieved solely by static placement of yield points and thread initiations."
- 0xjnml 5y ago
  Goroutines are asynchronously preemptible since Go 1.14, released February 25th, 2020. https://blog.golang.org/go1.14 https://blog.golang.org/go1.14

jlokier 5y ago

Technically it's not pre-emption. It's activating parallelism on another CPU core when a time condition is triggered, in order to reduce tail latencies of unrelated tasks and prevent starvation. Nothing is directly pre-empted by that event. But let's run with pre-emption, because that does apply on single cores. No, it doesn't work out to be a global cost, assuming you mean performance (as opposed to developer cognitive cost). I have experience with this - the game engine. We don't half-ass performance in games like that, it's a central factor. We optimise for the highest performance on middle-of-the-road hardware, and measure constantly. The Rust model is same or worse on each global performance metric that matters in that scenario. The global metrics are things like graphics throughput, tail latency and variance, interactive response, task starvation. So the "cost" of "pre-emption" ranges from zero to negative. I will grant you that pre-emption adds shared resource locking overheads if you're comparing with an async-await model that uses only a single thread for all tasks globally, so locking is not required. (I.e. JavaScript). But that's not the Rust model. Pre-emption has a small, positive local cost, in that you need to check when to pre-empt. But that can be made small. You don't need a kernel to do it, and you don't need to be constantly reading clocks either. The cost is a few nanoseconds per microtask, i.e. book-keeping like with any execution loop, and some periodic interrupts or signals, which you might have anyway. Alternatively you can use performance counters on some CPUs. The actual act of pre-empting is also cheap - it's just a coroutine jump with some book-keeping. But it's also comparatively rare compared with async task transitions, so the cost doesn't really matter anyway.

littlestymaar 5y ago
Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow). Go and async Rust are both cooperatively scheduled, switching back and forth only in specific yield points, which reduces the amount of state that needs to be saved. Adding such a yield point is costly too though, that's why Go didn't add them inside tight loops until 1.14.[1] Regarding scheduling and how task-switching is done, async Rust and Go are fundamentally the same model (and BTW, the authors of Tokio never hide that they took a lot of inspiration from the Go runtime). The three big differences are the following: - Go's tasks are stackful, they have their own stack, which is grow-able (the stack is copied into a bigger stack when it's full) - Go's yield points are automatically inserted by the compiler based on heuristics which vary depending on the version of the compiler. Whereas Rust yield point are manually inserted by the developer. This resonate strongly with the philosophy of both languages (best-effort automatic vs explicit manual control leading to the best performance) which can also be seen on the topic of memory allocation (automatic boxing based on escape analysis vs manual Box pointer). - Go only has green thread, which means that if the runtime is failing to keep its promises[1][2], you don't have much alternative. Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload. [1] https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958 [2] https://github.com/golang/go/issues/36365 https://github.com/golang/go/issues/36365
- jlokier 5y ago
  > Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow). In-userspace stackful context switches are not at all slow in a sane userspace-switching runtime. They are approximately a combination of setjmp() and longjmp(), but without saving all registers. Typically a few tens of nanoseconds or less on a modern CPU. Pre-emptive context switches are nearly always stackful, even if non-pre-emptive (async) context switches are stackless in the same runtime. A pre-emptive context switch in a userspace co-operative scheduling system is slower than non-pre-emptive context switch if it is caused by a dedicated interrupt of some kind. In the case of userspace pre-emption, generally a signal and return, typically single digit microseconds. However, the cost there is mainly the signal. If the pre-emptive context switch is caused by a signal that triggered anyway for another purpose, the actual context switch is, again, cheap. In the model I described (the game environment), pre-emptive context switches are rare. It wouldn't matter if they took longer, because >= 99.99% of context switches are co-operative in that model. The important factor is that no task blows the frame budget or prevents other tasks from running at the full frame rate. In web services a similar target is tail latency, in the presence of diverse tasks you cannot predict in advance. > meaning you can choose what works best for your workload. Exactly. And as I've tried to explain, for some varying, complex workloads whatever you choose statically has poor metrics; it must be adaptive to maintain good timing metrics. > Rust yield point are manually inserted by the developer Indeed, and in the case of NPCs in a game engine, or a large program composed of many libraries written by hundreds of different authors (e.g. some web servers), that causes complex, interdependent performance characteristics, where timing behaviour in one of them ruins performance for everything, unless they are isolated using threads, in which case it's too slow. There is no "the developer" who can ensure this doesn't happen. (Aside, if Rust's type system helped with this, that would be great, the same way types help with other "programming in the large" safety issues, but it doesn't address timing characteristics as far as I know.) > leading to the best performance [..] Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload. You could summarise my point as: "For some dynamic workloads, especially in timing-sensitive large programs with many components working independently and unpredictably, neither async tasks or OS threads perform best for your workload (or even adequately sometimes). The optimal (or required) combination requires some dynamic responses, and cannot be achieved solely by static placement of yield points and thread initiations."
0xjnml 5y ago
Goroutines are asynchronously preemptible since Go 1.14, released February 25th, 2020. https://blog.golang.org/go1.14 https://blog.golang.org/go1.14