3 ms·
When Quasar was very young we had something very similar to Erlang's reduction-based preemption; we later disabled it in favor of preemption on communication on
by pron 11y ago
When Quasar was very young we had something very similar to Erlang's reduction-based preemption; we later disabled it in favor of preemption on communication only. The reason is this: suppose you have n cores. Now, suppose you have a single fiber that hogs the core for a while. If this occurrence is rare, reduction/time-slice preemption doesn't really help, because the scheduler has n kernel threads, and one of them would steal the tasks scheduled on the thread running the "runaway" fiber and all is well.
If, on the other hand this behavior is very common, and the number of fibers exhibiting it is k, then if k > n, then you're in trouble no matter what scheduling or preemption you do. k fibers that constantly ask for CPU they can't have (because k > n) means that work is getting delayed and you're well out of soft realtime territory.
So since n is small (say 8-60), and the number of fibers is large (10K-1M or more), the number of fibers that can reasonably often require a lot of CPU (and reduction/time-slice preemption) is very small compared to the total number of fibers, and those would be special, well-known, fibers anyway (remember that if a fiber needs a lot of CPU only occasionally, the many-threaded scheduler handles that gracefully even without preemption as long as most other fibers are well-behaved). In that case rather than base the scheduler designed for ~1M fibers around the behavior of ~10 fibers just doesn't make sense. It is far more efficient to simply run that CPU-hungry code in plain-old kernel threads, and let the kernel worry about their scheduling. In Quasar, unlike in Erlang, it's very simple to choose whether code should run in a user-mode-scheduled fiber or in a kernel-scheduled heavyweight thread, and so, after seeing that turning on reduction-based preemption had no positive effect in practice, we turned that feature off.