4 ms·

Yep. We do this on real hardware and never preempt. There is nothing more performant than that. Using IPIs (inter-processor interrupts) you can trigger events l

Yep. We do this on real hardware and never preempt. There is nothing more performant than that. Using IPIs (inter-processor interrupts) you can trigger events like "more work has been added" on each CPUs queue. Additionally, when you put all interrupts of a device solely on one specific CPU you won't have to lock anything. Some other things: pthreads generally have high cost, and that means C++ threads do too. pthreads have quite a few features that you regularly don't use, which you can skip completely using fibers and coroutines.

Koshkin 7y ago
> never preempt I don’t think this is a realistic expectation on Linux and, especially, Windows which runs hundreds of threads of its own you don’t want to know about. (Besides, we must remember that multithreading was invented and found quite useful in the era of ”single-core” processors.)
- emidln 7y ago
  Linux/CFS has an isolcpus feature that can be used to tell the kernel to never schedule things on a given CPU. This is useful when latency matters.
  - ncmncm 7y ago
    I learned recently that even with isolcpu, and nohz, and interrupts directed elsewhere, the kernel will still pause the thread on the isolcpu if it has mmapped a file (e.g., to report stats) and the kernel decides it's time to copy the bits to disk. If you don't want stalls, only map writable files on a tmpfs volume. To snapshot the file, copy it to another file on the same volume, and then snapshot the copy.
    - gpderetta 7y ago
      Yes, it is called TLB shootdown and it is required to preserve the integrity of the TLB across CPUs on an umap or a dirty bit change. If latency is impprtant, don't use disk backed writeable mappings. Edit: to clarify: any writeable mapping or any unmap will cause TLB shutdown interrupts to be broadcasted to all currently running threads of a process.
      - ncmncm 7y ago
        Of course one never unmaps the file, or any mapped memory, so this has nothing to do with TLB problems (which are also a thing -- another reason processes are better than threads). These pauses happen even without unmapping. Despite that no synchronization is available, so you are right in the middle of whatever, the kernel decides a static snapshot of the pages' state must be written, so write-protects the pages first, and blocks your process until the write is done. It's just rude.
        gpderetta 7y ago
        Unmapping is one case. As I said earlier, the other case is clearing the dirty bit for a page in the in the page directory. As the dirty bit is cached in the tlb (so it doesn't need to be written back every time the page is written by a cpu), the tlb need to be flushed. I do not think the kernel write protects the page is writing out, unless the architecture has no hardware dirty bit and one must be maintained by the os. Technically tlb shootdown is not a form of blocking, but it does cause periodic latency spikes.
        ncmncm 7y ago
        Multi-millisecond latency spikes in a process spinning on an atomic update cannot be blamed on TLB flush, which introduces stalls measured in, at worst, microseconds.
        gpderetta 7y ago
        [very late reply] yes, that sort of spikes are definitely not due to just interrupts. As you say, it is probably a kernel io thread taking over the cpu.
- bluetomcat 7y ago
  On a server running heterogeneous CPU-bound tasks of various users it is hardly a realistic expectation, but on a single-user device with a single application in the foreground I would say it is realistic, since most of these running processes are blocked on something most of the time. Hundreds of mostly-idle resident processes are insignificant to a one that puts the CPU through its paces.
- fwsgonzo 7y ago
  This thread is about the performance of threads, which we have established can be a high cost for some. In our case we don't run on Linux or Windows, but you can still do the same on Linux afaik, although you will have to write a kernel object for some things.
- cma 7y ago
  Mask off all those other processes to only run on core 0.
londons_explore 7y ago
How many percent performance do you think you gain by never pre-empting? If we're talking 50%, the complexity sounds worth it, but if it's 1% I think I'd prefer to stick with standard scheduling and know my program will 'just work' on any CPU or OS, and with any libraries I choose to use.
- fwsgonzo 7y ago
  Probably a lot on bare metal, but we are a special case here. It really brings down the latency to respond to network events. A single context switch is in the range of 100K-1M CPU cycles. The reason why is because we avoid all the indirect cost of context switching, which is all the various caches that has to be flushed. And also the context switching itself, of course. However, you can still do a lot on Linux to equalize things if you really want to get down to it. For anything but special cases Linux really does a good job with scheduling. After all, you are likely not running much else other than your intended service. That said, for me this thread was a slight wakeup-call that made me look more into fibers and co-routines. I have been wanting to use these for a long time for some things.
  - boulos 7y ago
    > A single context switch is in the range of 100K-1M CPU cycles. 1M cycles is roughly 300 microseconds (assume 3 GHz processor, so 3 cycles is 1 nanosecond). Eli’s graph from the post I referenced above, has a context switch in the 1-3 microsecond range [1] depending on taskset/core pinning. The high end (3 microseconds) is about 10000 cycles then. Maybe you mean fork() or pthread_create for your 1M cycles? [1] https://eli.thegreenplace.net/images/2018/plot-launch-switch.png https://eli.thegreenplace.net/images/2018/plot-launch-switch...
- jandrewrogers 7y ago
  The difference can be quite large, details are workload and software dependent. It isn't just the context-switching overhead (which is prohibitively high these days), it also significantly improves average cache locality, which is the bottleneck for many high-performance codes. Some types of software optimizations require the ability to correctly infer local CPU cache contents, which is difficult when arbitrary processes are semi-randomly stepping all over that cache.

Koshkin 7y ago

> never preempt I don’t think this is a realistic expectation on Linux and, especially, Windows which runs hundreds of threads of its own you don’t want to know about. (Besides, we must remember that multithreading was invented and found quite useful in the era of ”single-core” processors.)

emidln 7y ago
Linux/CFS has an isolcpus feature that can be used to tell the kernel to never schedule things on a given CPU. This is useful when latency matters.
- ncmncm 7y ago
  I learned recently that even with isolcpu, and nohz, and interrupts directed elsewhere, the kernel will still pause the thread on the isolcpu if it has mmapped a file (e.g., to report stats) and the kernel decides it's time to copy the bits to disk. If you don't want stalls, only map writable files on a tmpfs volume. To snapshot the file, copy it to another file on the same volume, and then snapshot the copy.
  - gpderetta 7y ago
    Yes, it is called TLB shootdown and it is required to preserve the integrity of the TLB across CPUs on an umap or a dirty bit change. If latency is impprtant, don't use disk backed writeable mappings. Edit: to clarify: any writeable mapping or any unmap will cause TLB shutdown interrupts to be broadcasted to all currently running threads of a process.
    - ncmncm 7y ago
      Of course one never unmaps the file, or any mapped memory, so this has nothing to do with TLB problems (which are also a thing -- another reason processes are better than threads). These pauses happen even without unmapping. Despite that no synchronization is available, so you are right in the middle of whatever, the kernel decides a static snapshot of the pages' state must be written, so write-protects the pages first, and blocks your process until the write is done. It's just rude.
      - gpderetta 7y ago
        Unmapping is one case. As I said earlier, the other case is clearing the dirty bit for a page in the in the page directory. As the dirty bit is cached in the tlb (so it doesn't need to be written back every time the page is written by a cpu), the tlb need to be flushed. I do not think the kernel write protects the page is writing out, unless the architecture has no hardware dirty bit and one must be maintained by the os. Technically tlb shootdown is not a form of blocking, but it does cause periodic latency spikes.
        ncmncm 7y ago
        Multi-millisecond latency spikes in a process spinning on an atomic update cannot be blamed on TLB flush, which introduces stalls measured in, at worst, microseconds.
        gpderetta 7y ago
        [very late reply] yes, that sort of spikes are definitely not due to just interrupts. As you say, it is probably a kernel io thread taking over the cpu.
bluetomcat 7y ago
On a server running heterogeneous CPU-bound tasks of various users it is hardly a realistic expectation, but on a single-user device with a single application in the foreground I would say it is realistic, since most of these running processes are blocked on something most of the time. Hundreds of mostly-idle resident processes are insignificant to a one that puts the CPU through its paces.
fwsgonzo 7y ago
This thread is about the performance of threads, which we have established can be a high cost for some. In our case we don't run on Linux or Windows, but you can still do the same on Linux afaik, although you will have to write a kernel object for some things.
cma 7y ago
Mask off all those other processes to only run on core 0.

londons_explore 7y ago

How many percent performance do you think you gain by never pre-empting? If we're talking 50%, the complexity sounds worth it, but if it's 1% I think I'd prefer to stick with standard scheduling and know my program will 'just work' on any CPU or OS, and with any libraries I choose to use.

fwsgonzo 7y ago
Probably a lot on bare metal, but we are a special case here. It really brings down the latency to respond to network events. A single context switch is in the range of 100K-1M CPU cycles. The reason why is because we avoid all the indirect cost of context switching, which is all the various caches that has to be flushed. And also the context switching itself, of course. However, you can still do a lot on Linux to equalize things if you really want to get down to it. For anything but special cases Linux really does a good job with scheduling. After all, you are likely not running much else other than your intended service. That said, for me this thread was a slight wakeup-call that made me look more into fibers and co-routines. I have been wanting to use these for a long time for some things.
- boulos 7y ago
  > A single context switch is in the range of 100K-1M CPU cycles. 1M cycles is roughly 300 microseconds (assume 3 GHz processor, so 3 cycles is 1 nanosecond). Eli’s graph from the post I referenced above, has a context switch in the 1-3 microsecond range [1] depending on taskset/core pinning. The high end (3 microseconds) is about 10000 cycles then. Maybe you mean fork() or pthread_create for your 1M cycles? [1] https://eli.thegreenplace.net/images/2018/plot-launch-switch.png https://eli.thegreenplace.net/images/2018/plot-launch-switch...
jandrewrogers 7y ago
The difference can be quite large, details are workload and software dependent. It isn't just the context-switching overhead (which is prohibitively high these days), it also significantly improves average cache locality, which is the bottleneck for many high-performance codes. Some types of software optimizations require the ability to correctly infer local CPU cache contents, which is difficult when arbitrary processes are semi-randomly stepping all over that cache.