18 ms·

Your architecture proposal seems sound assuming that you have that much control over the whole system, and you don't need to integrate with any legacy component

Your architecture proposal seems sound assuming that you have that much control over the whole system, and you don't need to integrate with any legacy components. In the common case, you need to deal with limited control (e.g. in a userspace app) and you have legacy components (e.g. a consumer OS, a Java VM, etc). If you're doing best-effort soft-real-time on a consumer OS then you probably don't have control over thread affinity to _guarantee_ one thread per CPU, and you certainly can't disable IRQs. Even on servers, you can get Linux to set up your thread-per-CPU thing, but still no IRQ masking. Also, you'd need your whole stack to do cooperative multitasking in a fully composable way before you could do as you propose. Maybe that's plausible in Go, but if you're in Java (as a lot of the HFT people seem to be, for reasons I don't understand) then probably only _some_ of your system is structured as one-thread-per-CPU cooperative multitasking, and you may still need to offload high-latency work using lock-free queues. In my domain (real-time audio on desktop OSes), neither Windows nor macOS have working deterministic priority inversion mitigation. So using lock-free queues between priority domains (e.g. between GUI thread and real-time audio thread) avoids priority inversion risk (compared to using mutexes). Granted, this is a rather specialised use-case. Another reason people are interested in these things is that there's a perception that lock-free algorithms perform better for inter-thread queuing than the alternatives. In particular, there are claims that they scale better when there are a large number of producer and consumer threads/cores. I think this is the most controversial area, since the queue algorithms may scale better than e.g. a single mutex on a single queue, but it's not necessarily the case that an application architecture that uses a MPMC queue is the best fit for purpose.

valarauca1 10y ago
If you're doing best-effort soft-real-time on a consumer OS then you probably don't have control over thread affinity to _guarantee_ one thread per CPU Which consumer OS? FreeBSD, OpenBSD, Linux, and Windows all offer the functionality. To my knowledge MacOS is the only consumer OS that doesn't. and you certainly can't disable IRQs This has been in Linux for over a decade. You can modify IRQ affinity from the root shell via the proc-fs [1]. [1] https://www.kernel.org/doc/Documentation/IRQ-affinity.txt https://www.kernel.org/doc/Documentation/IRQ-affinity.txt
- RossBencina 10y ago
  I stand corrected on the Linux IRQ point. In the past I have not found Windows thread affinity APIs to provide hard guarantees. Perhaps this has changed since I last tried it.
- Unklejoe 10y ago
  Hmm. Interesting. I would have never thought to use that within a program during runtime. I don't think that the IRQ affinity facility is meant to be used like that ("that" being the masking of interrupts for a given CPU to create critical regions with a program). For starters, it seems like you'll need to be running as root to be able to change it. That could be an issue for some applications. It also says that you can’t disable interrupts for all cores. It also seems very non portable.
  - valarauca1 10y ago
    I would have never thought to use that within a program during runtime. During Runtime is unlikely. It does have clear benefits especially when dealing with 10GbE nics. Having N-cores handle 2x NIC's the caching thrashing can bottleneck your maximum bandwidth. While dedicating 1:1:1 NIC -> Core -> IRQ solves this. Generally setting up IRQ masking should be part of your pre-startup configuration. Not setting it dynamically at run-time. It also says that you can’t disable interrupts for all cores Yes/No. One can disable all interrupts for a core via separate mechanisms. But this means you need to implement your own scheduler on that core, and remove that core from the scheduler. Also you lose the ability to fire syscalls, lose kernel level Mutexes, Futexes, IO, etc. It gets very complicated very fast. Having some interrupts is generally a good thing. Lastly you'll still be subject to memory bus interrupts (but they aren't called interrupts, they're non trivial stalls) to maintain Cache coherency and R/W ordering (in AMD64 and ARMv8). You can never opt out of these. If you want to disable all interrupts on all cores why are you even running an OS? It also seems very non portable. AMD64 Linux is literally the most used OS in data centers, but you are nonetheless correct.
    - Unklejoe 10y ago
colanderman 10y ago
Thank you, that was very informative.

valarauca1 10y ago

If you're doing best-effort soft-real-time on a consumer OS then you probably don't have control over thread affinity to _guarantee_ one thread per CPU Which consumer OS? FreeBSD, OpenBSD, Linux, and Windows all offer the functionality. To my knowledge MacOS is the only consumer OS that doesn't. and you certainly can't disable IRQs This has been in Linux for over a decade. You can modify IRQ affinity from the root shell via the proc-fs [1]. [1] https://www.kernel.org/doc/Documentation/IRQ-affinity.txt https://www.kernel.org/doc/Documentation/IRQ-affinity.txt

RossBencina 10y ago
I stand corrected on the Linux IRQ point. In the past I have not found Windows thread affinity APIs to provide hard guarantees. Perhaps this has changed since I last tried it.
Unklejoe 10y ago
Hmm. Interesting. I would have never thought to use that within a program during runtime. I don't think that the IRQ affinity facility is meant to be used like that ("that" being the masking of interrupts for a given CPU to create critical regions with a program). For starters, it seems like you'll need to be running as root to be able to change it. That could be an issue for some applications. It also says that you can’t disable interrupts for all cores. It also seems very non portable.
- valarauca1 10y ago
  I would have never thought to use that within a program during runtime. During Runtime is unlikely. It does have clear benefits especially when dealing with 10GbE nics. Having N-cores handle 2x NIC's the caching thrashing can bottleneck your maximum bandwidth. While dedicating 1:1:1 NIC -> Core -> IRQ solves this. Generally setting up IRQ masking should be part of your pre-startup configuration. Not setting it dynamically at run-time. It also says that you can’t disable interrupts for all cores Yes/No. One can disable all interrupts for a core via separate mechanisms. But this means you need to implement your own scheduler on that core, and remove that core from the scheduler. Also you lose the ability to fire syscalls, lose kernel level Mutexes, Futexes, IO, etc. It gets very complicated very fast. Having some interrupts is generally a good thing. Lastly you'll still be subject to memory bus interrupts (but they aren't called interrupts, they're non trivial stalls) to maintain Cache coherency and R/W ordering (in AMD64 and ARMv8). You can never opt out of these. If you want to disable all interrupts on all cores why are you even running an OS? It also seems very non portable. AMD64 Linux is literally the most used OS in data centers, but you are nonetheless correct.
  - Unklejoe 10y ago

colanderman 10y ago

Thank you, that was very informative.