3 ms·

I've never seen that, how do you do it? Also, how do you deal with VM and scheduling?

• ericbarrett 6y ago
  Brief intro, should bootstrap your knowledge: https://codywu2010.wordpress.com/2015/09/27/isolcpus-numactl-and-taskset/ https://codywu2010.wordpress.com/2015/09/27/isolcpus-numactl...
• scatters 6y ago
  Isolating a core: isolcpus and taskset, or possibly cgroups, plus nohz and there's a patch set that offers even stronger isolation guarantees (keeping kernel threads off the core). Hardware mappings are done as a ring buffer in memory mapped to the device; a kernel module helps set up that mapping but once it's in place reads and writes to those virtual addresses go directly (well - via MMU and memory bus) to the hardware. The general term for this is user-space networking (although obviously you can do it with other hardware as well). Some addresses may be mapped to device registers that trigger hardware actions on read/write. VM - the key is to get everything mapped in at the start and avoid page faults subsequently. madvise can help with this and obviously you need to avoid memory leaks that could result in sbrk - but in any case allocating at all in the trading thread is generally unnecessary (after startup) and frowned on. This does mean that the (lock-free) queue back to the shared core needs to recycle memory and both sides need to service it regularly or you need a strategy to deal with exhaustion. Scheduling isn't an issue - you have a single thread per core and (with isolcpus) the kernel scheduler knows to avoid placing other threads on it.
  • rramadass 6y ago
    Thank you for that writeup. Can you please list some resources (links/articles/books etc.) for us to read up on the above?
• throwaway_dcnt 6y ago
  In addition to the peer comments, using JNI, these facilities can also be availed in Java. Chronicle queue does this.