6 ms·
Author here. I'll be watching the comments and can answer any questions! I do want to make clear that most of the optimizations discussed are implemented in th
by carllerche 7y ago
Author here. I'll be watching the comments and can answer any questions!
I do want to make clear that most of the optimizations discussed are implemented in the Go scheduler, which is where I discovered them. I wrote the article to call them out as they were not easy to discover.
- networkimprov 7y agoThe Go team is working on an update of its scheduler to make it preemptive. Will Tokio follow suit? https://github.com/golang/go/issues/24543 https://github.com/golang/go/issues/24543
- carllerche 7y agoMy understanding of that PR is it relates to how the Go compiler does code generation. Rust takes a different approach and is about to ship `async / await` which is a different strategy. Preemption is out of scope for Tokio as we are focusing on a runtime to power Rust async functions. So, for the foreseeable, Tokio will using "cooperative preemption" via `await` points.
- deleted 7y ago[deleted]
- 15155 7y agoIIRC: one of the issues with the original Rust stdlib greenthread implementation was that it's extremely difficult to preempt execution in a systems language. It's also quite difficult to add userland execution preemption without incurring significant performance penalties. How do you preempt syscalls or FFI calls? (Go will almost certainly also have this issue with cgo modules) I would be very surprised if preemptible Futures are something Tokio can or would implement.
- wbl 7y agoWhat does systems mean here? There is a dirty trick with a NOP and self rewriting code that's quite cheap when not triggered. I think java does it that way.
- sansnomme 7y agoElaborate?
- wbl 7y agoSo you put in a NOP before each backward jump. To preempt you overwrite it with a jump to a handler that saves the state of the thread.
- jedisct1 7y agoHow multitasking used to work on Atari :) But this requires code pages to be writable. Which, from a security perspective is awful.
- heavenlyblue 7y agoHow does that resolve syscall question?
- wbl 7y agoSyscall you have to do with a signal
- nine_k 7y agoThis presumes writable code segments, which many OSes shun.
- hyperman1 7y agoIt's java. Dynamic code generated by a jit.
- adwn 7y ago
- pcwalton 7y agoPreemption isn't a concept that applies to a task scheduler like Tokio. You can't just inject more states into a finite state machine at runtime.
- Exuma 7y agoWhen does that get released into Go? That sounds cool
- jstrong 7y agoI noticed that you call `unsync_load` on the `AtomicU32`, which confused me, and I later saw that there is a custom `AtomicU32` in the loom library. Can you explain what `unsync_load` is and whether it will be provided by the atomics library in `std`?
- carllerche 7y ago`unsync_load` reads the memory as if it were a normal field (no atomic operation). The `std` atomic types already provide `get_mut` which gives access to the "raw" field when the caller has a mutable reference to the atomic. The mutable access guarantees there are no concurrent threads that can access the field, so it is safe. The `unsync_load` is used because, at that point, two things are guaranteed: - The only thread that mutates the field is the `unsync_load` caller - All other threads will only read from that field. Because of this, we can just do a "plain old" access of the field w/o the atomic overhead. As for whether or not it should be in `std`, I don't know. It probably could, but it doesn't have to. It's a pretty "advanced" access strategy.
- dunkelheit 7y agoWhat is the difference with Ordering::Relaxed?
- ComputerGuru 7y agoOrdering::Relaxed guarantees atomicity (but not coherence). On 32-bit or 64-bit platforms, a (aligned) 32-bit read or write will always be atomic anyway.
- dunkelheit 7y agoWhat do you mean by coherence? If it is the property that all updates to a single memory location are seen in a consistent order by all threads then Relaxed guarantees that.
- 7y ago
- petschge 7y agoTiny comment: I think the sentence "However, in practice the overhead needed to correctly avoid locks is greater than just using a mutex." should be part of footnote 1, not the main text.
- jmakov 7y agoToo bad FFBuffer is not implemented from the FastFlow project. They are using queues without memory fences.
- carllerche 7y agoAssuming [this](https://github.com/fastflow/fastflow/blob/master/ff/buffer.hpp#L7-L8 https://github.com/fastflow/fastflow/blob/master/ff/buffer.h...) is the FFBuffer in question, it looks like it is spsc (which would not support stealing). Also, it would need some kind of synchronization to ensure consistency between the consumer & producer.
- jmakov 7y agoYes, it's an implementation of the paper they're referencing - "An Efficient Unbounded Lock-Free Queue for Multi-core Systems". And they've build a framework around it (all queues implemented using this structure). Their benchmarks shows improvement (as shown in their paper). Might be interesting to use that in Rust. But what I miss is a benchmark for various approaches (in Rust, C++ etc.) with trying to use multi core effectively. What I found is that some of the papers use https://parsec.cs.princeton.edu/overview.htm https://parsec.cs.princeton.edu/overview.htm. Probably still better that what we have now - "X nanosec per push/pop".
- gpderetta 7y agoThe advantage of FF is that it is unbounded, but if you add the required mutual exclusion between pop and steal, it will be as expensive (if not more due to the extra complexity).
- MarkSweep 7y agoIt looks like the number of threads in the threadpool is fixed. If have 1 one thread in the thread pool, is it possible to dead lock it by queuing a Task A the blocks waiting for Task B to do something, where A starts executing before B? Do you think it would make sense to dynamically adjust the number of threads in the pool? An example of a thread pool that dynamically adjusts the number of threads is .NET. The algorithm is described by Matt Warren[1] and a paper[2]. In addition to the C++ version Matt links to, there is a C#[3] implementation. [1]: https://mattwarren.org/2017/04/13/The-CLR-Thread-Pool-Thread-Injection-Algorithm/ https://mattwarren.org/2017/04/13/The-CLR-Thread-Pool-Thread... [2]: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.487.7627&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.487... [3]: https://github.com/dotnet/corert/blob/master/src/System.Private.CoreLib/src/System/Threading/ClrThreadPool.HillClimbing.cs https://github.com/dotnet/corert/blob/master/src/System.Priv...
- carllerche 7y agoThe expectation is that all code running on the Tokio scheduler does not block and does not do anything CPU intensive. There is follow up work planned to allow annotating blocking / CPU intensive bits of code so that the scheduler can do something smarter (things like you linked). The old scheduler already had such an annotation, but in the interest of shipping, this has punted in the new scheduler.
- ulber 7y agoIs that expectation wrt. responsiveness guarantees? In other words, if I'm just interested in throughput are CPU intensive tasks a problem? For example, my expectation is that it would be appropriate to use the new Tokio scheduler to parallelize a compute intensive simulation workload (where inside a "frame" responsiveness is not a problem).
- carllerche 7y agoThe Tokio scheduler uses cooperative multitasking (non-preemptive). So, if your task runs without yielding it can prevent other runnable tasks from executing. So, if that is acceptable, then it is fine.