3 ms·
> also doesn't allow pointers into the stack, so we can do very efficient copying "in the shadow" of cache misses How does this work?
by vacuity 3y ago
> also doesn't allow pointers into the stack, so we can do very efficient copying "in the shadow" of cache misses
How does this work?
- pron 3y agoFirst, the use-case where virtual threads offer the most benefit is servers with high concurrency (due to Little's law https://youtu.be/07V08SB1l8c?si=rwTQrnHBnp4NGrj7 https://youtu.be/07V08SB1l8c?si=rwTQrnHBnp4NGrj7), which means we're talking about a very large number of threads. That, in turn, means that the state of all those threads cannot fit in the CPU cache, so even the most efficient implementation possible, i.e. one that simply changes the sp register to point to a new stack will incur an expensive cache miss. That means that copying small sections of the stack is almost free (because a cache miss+copy isn't all that more expensive than just a cache miss). Copying portions of the stack means that you can continue executing code directly in OS threads (rather than change the stack pointer), so the overhead for FFI is zero. However, you cannot copy portions of stack if there are pointers into the stack. In .NET and in low-level languages there are pointers into the stack which makes copying inefficient. Furthermore, managing these stack portions efficiently in the heap requires extremely efficient dynamic memory management, which is something that languages with good GCs do better than low-level languages with more direct memory management.