People that are very serious about it use ASICs/FPGAs. I'm sure Goldman does too. For slightly slower stuff it's the garbage collector and OS scheduling that's

People that are very serious about it use ASICs/FPGAs. I'm sure Goldman does too. For slightly slower stuff it's the garbage collector and OS scheduling that's the problem (as both can unpredictably inject many milliseconds pauses). No GC and good usage of low level kernel primitives is the game.