6 ms·

I don't know about java, but when using c++ you can use the OS to start the program, set up the hardware mappings and then move it to an isolated core where you

I don't know about java, but when using c++ you can use the OS to start the program, set up the hardware mappings and then move it to an isolated core where you talk to the hardware directly without ever issuing a system call. Compilers already offer flags to optimize for specific processor generations (and use the newly available instructions); you aren't going to be able to do better than that with a custom compiler.

bboygravity 6y ago
Oh, that's cool. However, you'd still be sharing at least 1 core (and all peripheral hardware and memory access in the system) with the OS. A core, other hardware and memory that you could otherwise access directly from your application without having to wait for the OS to finish its business? I'm not sure whether it's worth it to go through this trouble (writing for bare metal vs just using an off-the-shelve OS) in a real life trading-app scenario. Hence my wonder :)
- scatters 6y ago
  Accessing other hardware isn't latency-sensitive, though; that's usually disk and the internal network, for logging, telemetry and control. Typically a trading application will keep a thread running on the shared core and send messages between that and its dedicated core thread using lock-free queues, or simple atomics. As you say, writing for bare metal is a lot of hassle and it's unnecessary for the parts of a trading application that don't need to be super fast; using a commodity OS and isolating resources gives the best of both worlds.
funcDropShadow 6y ago
The jit compiler of typical jvms automatically uses the instruction sets (with limits afaik) that are available on the hardware that it is running on.
sakex 6y ago
I've never seen that, how do you do it? Also, how do you deal with VM and scheduling?
- ericbarrett 6y ago
  Brief intro, should bootstrap your knowledge: https://codywu2010.wordpress.com/2015/09/27/isolcpus-numactl-and-taskset/ https://codywu2010.wordpress.com/2015/09/27/isolcpus-numactl...
- scatters 6y ago
  Isolating a core: isolcpus and taskset, or possibly cgroups, plus nohz and there's a patch set that offers even stronger isolation guarantees (keeping kernel threads off the core). Hardware mappings are done as a ring buffer in memory mapped to the device; a kernel module helps set up that mapping but once it's in place reads and writes to those virtual addresses go directly (well - via MMU and memory bus) to the hardware. The general term for this is user-space networking (although obviously you can do it with other hardware as well). Some addresses may be mapped to device registers that trigger hardware actions on read/write. VM - the key is to get everything mapped in at the start and avoid page faults subsequently. madvise can help with this and obviously you need to avoid memory leaks that could result in sbrk - but in any case allocating at all in the trading thread is generally unnecessary (after startup) and frowned on. This does mean that the (lock-free) queue back to the shared core needs to recycle memory and both sides need to service it regularly or you need a strategy to deal with exhaustion. Scheduling isn't an issue - you have a single thread per core and (with isolcpus) the kernel scheduler knows to avoid placing other threads on it.
  - rramadass 6y ago
    Thank you for that writeup. Can you please list some resources (links/articles/books etc.) for us to read up on the above?
- throwaway_dcnt 6y ago
  In addition to the peer comments, using JNI, these facilities can also be availed in Java. Chronicle queue does this.

bboygravity 6y ago

Oh, that's cool. However, you'd still be sharing at least 1 core (and all peripheral hardware and memory access in the system) with the OS. A core, other hardware and memory that you could otherwise access directly from your application without having to wait for the OS to finish its business? I'm not sure whether it's worth it to go through this trouble (writing for bare metal vs just using an off-the-shelve OS) in a real life trading-app scenario. Hence my wonder :)

scatters 6y ago
Accessing other hardware isn't latency-sensitive, though; that's usually disk and the internal network, for logging, telemetry and control. Typically a trading application will keep a thread running on the shared core and send messages between that and its dedicated core thread using lock-free queues, or simple atomics. As you say, writing for bare metal is a lot of hassle and it's unnecessary for the parts of a trading application that don't need to be super fast; using a commodity OS and isolating resources gives the best of both worlds.

funcDropShadow 6y ago

The jit compiler of typical jvms automatically uses the instruction sets (with limits afaik) that are available on the hardware that it is running on.

sakex 6y ago

I've never seen that, how do you do it? Also, how do you deal with VM and scheduling?

ericbarrett 6y ago
Brief intro, should bootstrap your knowledge: https://codywu2010.wordpress.com/2015/09/27/isolcpus-numactl-and-taskset/ https://codywu2010.wordpress.com/2015/09/27/isolcpus-numactl...
scatters 6y ago
Isolating a core: isolcpus and taskset, or possibly cgroups, plus nohz and there's a patch set that offers even stronger isolation guarantees (keeping kernel threads off the core). Hardware mappings are done as a ring buffer in memory mapped to the device; a kernel module helps set up that mapping but once it's in place reads and writes to those virtual addresses go directly (well - via MMU and memory bus) to the hardware. The general term for this is user-space networking (although obviously you can do it with other hardware as well). Some addresses may be mapped to device registers that trigger hardware actions on read/write. VM - the key is to get everything mapped in at the start and avoid page faults subsequently. madvise can help with this and obviously you need to avoid memory leaks that could result in sbrk - but in any case allocating at all in the trading thread is generally unnecessary (after startup) and frowned on. This does mean that the (lock-free) queue back to the shared core needs to recycle memory and both sides need to service it regularly or you need a strategy to deal with exhaustion. Scheduling isn't an issue - you have a single thread per core and (with isolcpus) the kernel scheduler knows to avoid placing other threads on it.
- rramadass 6y ago
  Thank you for that writeup. Can you please list some resources (links/articles/books etc.) for us to read up on the above?
throwaway_dcnt 6y ago
In addition to the peer comments, using JNI, these facilities can also be availed in Java. Chronicle queue does this.