5 ms·

> In Rust, if you're actually doing any compute work, you're stalling out the async system. Just use threads? Or add explicit yields.

Animats 5y ago
explicit yields. That's so 1984 Macintosh. "Cooperative multitasking". The usual effect of that style of programming is stuck window managers.
- staticassertion 5y ago
  You still get preemption at the thread level.
- pcwalton 5y ago
  So use threads! That's what they're there for. Go's solution is threads, just with a particularly idiosyncratic M:N implementation. Rust threads are 1:1. But they're the same concept. You can even get M:N if you really want it (nobody does, which is why mioco gets no use).
- jjav 5y ago
  Exactly. It's painful that the same problems get revisited as new things. Computation needs to be scheduled to a CPU or core for parallel progress to occur. That means kernel threads.
- monocasa 5y ago
  There's nothing wrong with explicit yields within a single process. It's only when the kernel scheduler can't preempt you that you end up with stuck window managers.
jlokier 5y ago
If you call a library function, not written by you, where you don't know whether it's going to return in a few nanoseconds or might take a long time, you the caller can't do the right thing yourself. A thread is too inefficient for the fast calls, but no thread is disastrous for the rest of the program's concurrency for the slow calls. So you're reliant on the library, not written by you, to always choose an efficient behaviour for each call. In other words, to decide dynamically when to use a thread, and when not to use a thread. And that library is reliant on all of its transitive dependencies in the same way. It's a nice ideal, but it's not realistic. And it still doesn't work all that efficiently anyway. If you call a library function many times, or just many functions, and any of those calls (outside your knowledge) has some long-running part where a thread ought to be used, to do it efficiently a single thread would span your code's multiple calls, and not just last for one library call. Essentially, to know whether to make a library call in a thread or not, and at what level to use a thread, you need to know what the behaviour of that call is going to be in advance. In practice, even when carefully optimising programs, we almost always assume one behaviour or another. Maybe measure, and assume the measured behaviour will remain similar. This doesn't have good adaptive behaviour when the assumption is wrong or out of date, so it doesn't work well for calls to things whose duration varies a lot, and especially not for large systems of calls between different modules, each of which has unpredictable timing. I first encounted this issue in a video games engine, long before Go or Rust existed (and before async-await). NPCs usually did a tiny amount of calculation, and there where tens of thousands, called every display frame, so this had to be fast. But some of them, known only to themselves, occasionally decided to make network calls, read cached data (which might need network calls or not), or call the filesystem, or do intensive "AI" planning using a burst of CPU. To maintain the frame rate while efficiently processing all NPCs, it was necessary to detect when some deep library call from an NPC had used the NPC's "short call" time budget, and switch it over to a thread. In that video games environment, the Go model worked. No matter how each NPC was coded and by whomever did it, an NPC might slow down if it called something slow but the game would stay smooth, and all other NPCs would continue to update at the full frame rate. The Rust model would result in the whole game frame rate stalling, which is much less acceptable. The threads-when-anything-could-be-slow model was too slow, and the use-threads-when-you-will-be-slow model gave NPC code too much power to affect the system outside the NPC.
- staticassertion 5y ago
  What you're arguing for is task preemption, and it has a global cost. If you're willing to pay global costs, Rust is probably not the right language for you.
  - jlokier 5y ago
    Technically it's not pre-emption. It's activating parallelism on another CPU core when a time condition is triggered, in order to reduce tail latencies of unrelated tasks and prevent starvation. Nothing is directly pre-empted by that event. But let's run with pre-emption, because that does apply on single cores. No, it doesn't work out to be a global cost, assuming you mean performance (as opposed to developer cognitive cost). I have experience with this - the game engine. We don't half-ass performance in games like that, it's a central factor. We optimise for the highest performance on middle-of-the-road hardware, and measure constantly. The Rust model is same or worse on each global performance metric that matters in that scenario. The global metrics are things like graphics throughput, tail latency and variance, interactive response, task starvation. So the "cost" of "pre-emption" ranges from zero to negative. I will grant you that pre-emption adds shared resource locking overheads if you're comparing with an async-await model that uses only a single thread for all tasks globally, so locking is not required. (I.e. JavaScript). But that's not the Rust model. Pre-emption has a small, positive local cost, in that you need to check when to pre-empt. But that can be made small. You don't need a kernel to do it, and you don't need to be constantly reading clocks either. The cost is a few nanoseconds per microtask, i.e. book-keeping like with any execution loop, and some periodic interrupts or signals, which you might have anyway. Alternatively you can use performance counters on some CPUs. The actual act of pre-empting is also cheap - it's just a coroutine jump with some book-keeping. But it's also comparatively rare compared with async task transitions, so the cost doesn't really matter anyway.
    - littlestymaar 5y ago
      Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow). Go and async Rust are both cooperatively scheduled, switching back and forth only in specific yield points, which reduces the amount of state that needs to be saved. Adding such a yield point is costly too though, that's why Go didn't add them inside tight loops until 1.14.[1] Regarding scheduling and how task-switching is done, async Rust and Go are fundamentally the same model (and BTW, the authors of Tokio never hide that they took a lot of inspiration from the Go runtime). The three big differences are the following: - Go's tasks are stackful, they have their own stack, which is grow-able (the stack is copied into a bigger stack when it's full) - Go's yield points are automatically inserted by the compiler based on heuristics which vary depending on the version of the compiler. Whereas Rust yield point are manually inserted by the developer. This resonate strongly with the philosophy of both languages (best-effort automatic vs explicit manual control leading to the best performance) which can also be seen on the topic of memory allocation (automatic boxing based on escape analysis vs manual Box pointer). - Go only has green thread, which means that if the runtime is failing to keep its promises[1][2], you don't have much alternative. Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload. [1] https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958 [2] https://github.com/golang/go/issues/36365 https://github.com/golang/go/issues/36365

Animats 5y ago

explicit yields. That's so 1984 Macintosh. "Cooperative multitasking". The usual effect of that style of programming is stuck window managers.

staticassertion 5y ago
You still get preemption at the thread level.
pcwalton 5y ago
So use threads! That's what they're there for. Go's solution is threads, just with a particularly idiosyncratic M:N implementation. Rust threads are 1:1. But they're the same concept. You can even get M:N if you really want it (nobody does, which is why mioco gets no use).
jjav 5y ago
Exactly. It's painful that the same problems get revisited as new things. Computation needs to be scheduled to a CPU or core for parallel progress to occur. That means kernel threads.
monocasa 5y ago
There's nothing wrong with explicit yields within a single process. It's only when the kernel scheduler can't preempt you that you end up with stuck window managers.

jlokier 5y ago

If you call a library function, not written by you, where you don't know whether it's going to return in a few nanoseconds or might take a long time, you the caller can't do the right thing yourself. A thread is too inefficient for the fast calls, but no thread is disastrous for the rest of the program's concurrency for the slow calls. So you're reliant on the library, not written by you, to always choose an efficient behaviour for each call. In other words, to decide dynamically when to use a thread, and when not to use a thread. And that library is reliant on all of its transitive dependencies in the same way. It's a nice ideal, but it's not realistic. And it still doesn't work all that efficiently anyway. If you call a library function many times, or just many functions, and any of those calls (outside your knowledge) has some long-running part where a thread ought to be used, to do it efficiently a single thread would span your code's multiple calls, and not just last for one library call. Essentially, to know whether to make a library call in a thread or not, and at what level to use a thread, you need to know what the behaviour of that call is going to be in advance. In practice, even when carefully optimising programs, we almost always assume one behaviour or another. Maybe measure, and assume the measured behaviour will remain similar. This doesn't have good adaptive behaviour when the assumption is wrong or out of date, so it doesn't work well for calls to things whose duration varies a lot, and especially not for large systems of calls between different modules, each of which has unpredictable timing. I first encounted this issue in a video games engine, long before Go or Rust existed (and before async-await). NPCs usually did a tiny amount of calculation, and there where tens of thousands, called every display frame, so this had to be fast. But some of them, known only to themselves, occasionally decided to make network calls, read cached data (which might need network calls or not), or call the filesystem, or do intensive "AI" planning using a burst of CPU. To maintain the frame rate while efficiently processing all NPCs, it was necessary to detect when some deep library call from an NPC had used the NPC's "short call" time budget, and switch it over to a thread. In that video games environment, the Go model worked. No matter how each NPC was coded and by whomever did it, an NPC might slow down if it called something slow but the game would stay smooth, and all other NPCs would continue to update at the full frame rate. The Rust model would result in the whole game frame rate stalling, which is much less acceptable. The threads-when-anything-could-be-slow model was too slow, and the use-threads-when-you-will-be-slow model gave NPC code too much power to affect the system outside the NPC.

staticassertion 5y ago
What you're arguing for is task preemption, and it has a global cost. If you're willing to pay global costs, Rust is probably not the right language for you.
- jlokier 5y ago
  Technically it's not pre-emption. It's activating parallelism on another CPU core when a time condition is triggered, in order to reduce tail latencies of unrelated tasks and prevent starvation. Nothing is directly pre-empted by that event. But let's run with pre-emption, because that does apply on single cores. No, it doesn't work out to be a global cost, assuming you mean performance (as opposed to developer cognitive cost). I have experience with this - the game engine. We don't half-ass performance in games like that, it's a central factor. We optimise for the highest performance on middle-of-the-road hardware, and measure constantly. The Rust model is same or worse on each global performance metric that matters in that scenario. The global metrics are things like graphics throughput, tail latency and variance, interactive response, task starvation. So the "cost" of "pre-emption" ranges from zero to negative. I will grant you that pre-emption adds shared resource locking overheads if you're comparing with an async-await model that uses only a single thread for all tasks globally, so locking is not required. (I.e. JavaScript). But that's not the Rust model. Pre-emption has a small, positive local cost, in that you need to check when to pre-empt. But that can be made small. You don't need a kernel to do it, and you don't need to be constantly reading clocks either. The cost is a few nanoseconds per microtask, i.e. book-keeping like with any execution loop, and some periodic interrupts or signals, which you might have anyway. Alternatively you can use performance counters on some CPUs. The actual act of pre-empting is also cheap - it's just a coroutine jump with some book-keeping. But it's also comparatively rare compared with async task transitions, so the cost doesn't really matter anyway.
  - littlestymaar 5y ago
    Preemption is costly, because you need to save the entire state of the execution you're preempting, in order to restore it later (which is also, as you should know, what makes context-switches slow). Go and async Rust are both cooperatively scheduled, switching back and forth only in specific yield points, which reduces the amount of state that needs to be saved. Adding such a yield point is costly too though, that's why Go didn't add them inside tight loops until 1.14.[1] Regarding scheduling and how task-switching is done, async Rust and Go are fundamentally the same model (and BTW, the authors of Tokio never hide that they took a lot of inspiration from the Go runtime). The three big differences are the following: - Go's tasks are stackful, they have their own stack, which is grow-able (the stack is copied into a bigger stack when it's full) - Go's yield points are automatically inserted by the compiler based on heuristics which vary depending on the version of the compiler. Whereas Rust yield point are manually inserted by the developer. This resonate strongly with the philosophy of both languages (best-effort automatic vs explicit manual control leading to the best performance) which can also be seen on the topic of memory allocation (automatic boxing based on escape analysis vs manual Box pointer). - Go only has green thread, which means that if the runtime is failing to keep its promises[1][2], you don't have much alternative. Rust has both Async tasks and OS threads, meaning you can choose what works best for your workload. [1] https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958 [2] https://github.com/golang/go/issues/36365 https://github.com/golang/go/issues/36365