5 ms·
So the compilation issue is only with TLS, an hypothetical FLS would not have it. The alternative to FLS is not storing duplicated data but use TLS and making
by gpderetta 6y ago
So the compilation issue is only with TLS, an hypothetical FLS would not have it.
The alternative to FLS is not storing duplicated data but use TLS and making sure that the compiler does not miscompile your code, either because you are extremely careful or because the compiler provides an option for it. Unfortunately compiler writers do not really see a future in stackful coroutines and are not really willing to provide a 'couroutine-safe' TLS mode.
Regarding the switch overhead, the fastest switch routine I have implemented does not store or load anything from any context-specific struct, only swaps a couple of registers, so an extra load might have a cost (trivial for async io, but can be noticeable for generators). Also, the way TLS is implemented on x86-64 for example is via a segment register which would need to be changed at every coroutine switch; updating it is significantly more expensive compared to a normal register [1].
Finally, as far as I know, the kernel does not use thread local/fiber local storage, instead heavily relies on per-cpu data. I advocate the same solution for coroutines, where, if you use a 1:1 mapping from OS threads to cores, the per-cpu data would correspond to the TLS of the underlying OS thread.
[1] Agner reports a reverse throughput of ~13 cycles on Neahalem. I think he stopped testing on newer architectures as it is a rarely used instruction; it needs to be expensive because it is not just a register load but it also has to update shadow limit registers. It probably has second order effects on the TLB as well.
- jlokier 6y ago> Finally, as far as I know, the kernel does not use thread local/fiber local storage, instead heavily relies on per-cpu data. It does use FLS. Everywhere you see "current->" or "current_thread_info()" in the Linux kernel, it's FLS, not per-cpu. There's a lot of those, some of them wrapped in other macros and inlines. "current" refers to the current task, which is the kernel's name for what the article calls fibers. Per-cpu data is used in the kernel as well, and I agree with you about using the analogous thing for coroutines which is TLS for certain things. Indeed, a FLS implementation almost certainly uses TLS for its root-of-data pointer and keeping tracking of the current thread's coroutine queue. > Agner reports a reverse throughput of ~13 cycles on Neahalem. [...] it is not just a register load but it also has to update shadow limit registers You're thinking of a segment register load, but these days we would use the WRFSBASE or WRGSBASE instructions, which don't update the segment descriptor (shadow limit etc) and don't use the GDT/LDT. Windows has supported this for userspace scheduling for a long time (but Windows has good fiber support with native FLS anyway), Java coroutines want to use it for performance on Linux, and Linux kernel looks like it will enable this soon; the CPUs have supported it for years. When those instructions aren't available, since the "current coroutine" is almost certainly a TLS variable itself that must be updated on context switch, use that as the base for FLS too. Then FLS access is one extra level of pointer indirection on access, but still hoisted and cached. For portable coroutines-and-threads code, use a portable TLS variable or pthread_getspecific() for this.