3 ms·
First, the use-case where virtual threads offer the most benefit is servers with high concurrency (due to Little's law https://youtu.be/07V08SB1l8c?si=rwTQrnHBn
by pron 3y ago
First, the use-case where virtual threads offer the most benefit is servers with high concurrency (due to Little's law https://youtu.be/07V08SB1l8c?si=rwTQrnHBnp4NGrj7 https://youtu.be/07V08SB1l8c?si=rwTQrnHBnp4NGrj7), which means we're talking about a very large number of threads. That, in turn, means that the state of all those threads cannot fit in the CPU cache, so even the most efficient implementation possible, i.e. one that simply changes the sp register to point to a new stack will incur an expensive cache miss. That means that copying small sections of the stack is almost free (because a cache miss+copy isn't all that more expensive than just a cache miss).
Copying portions of the stack means that you can continue executing code directly in OS threads (rather than change the stack pointer), so the overhead for FFI is zero. However, you cannot copy portions of stack if there are pointers into the stack. In .NET and in low-level languages there are pointers into the stack which makes copying inefficient. Furthermore, managing these stack portions efficiently in the heap requires extremely efficient dynamic memory management, which is something that languages with good GCs do better than low-level languages with more direct memory management.