29 ms·
I’ve seen this sentiment before and worked on both a “low latency Java” team and low latency C++ teams. I have some sympathy for the idea that the JVM is bette
by vgatherps 6y ago
I’ve seen this sentiment before and worked on both a “low latency Java” team and low latency C++ teams.
I have some sympathy for the idea that the JVM is better since it means you won’t spend all your time chasing crash reports. The thing is, like another comment hinted at, is that this issue is generally more reflective of the environment you build in than the technology choice.
Here’s a good talk on the reasons for and limits of using C++ for low latency systems: https://m.youtube.com/watch?v=NH1Tta7purM https://m.youtube.com/watch?v=NH1Tta7purM
If I had to rank in terms of reliability of trading infrastructure I’ve used/worked on,
* Low Latency C / C++ infra at fully automated trading firm. Very much run by the programmers and quants and also fairly small. By far the fastest (near limits of what you could do) and also most reliable.
* Low Latency Java execution infrastructure. Pretty reliable, not that fast, had some issues with GC battling and manual memory management to avoid GC, etc. There was a pretty clear latency floor (still quite low) even when “doing everything right” that serious native infrastructure beat.
* C++ market making infra at a firm run by manual traders. It was by far the slowest and least reliable. Echos the experience of “spent hours debugging weird crashes”. The culture was very “A trader asked for this and needs it done yesterday. Also this refactoring business doesn’t sound like adding new features, drop it”.
What I saw is that if you hire people who have a good idea of what they’re doing and keep a culture of technical excellence, C++ is definitely a better choice if you care about latency. You really have to maintain a culture of high quality code, testing, and in general caring about the technology.
This is only really possible when all of the stakeholders are involved in the technology, or at least understand the benefits. Once you start down the path of “well this feature could be done a day quicker if...” this goes down the drain, and a few years later you find yourself getting run over on latency AND with an impossible to use trading system. It’s really the worst of both worlds.
- fizixer 6y agoYou missed one comparison: C infra (no C++). (well strictly speaking, additionally you could also have hand-written assembly infra).
- humanrebar 6y agoYou can write C style code in C++ and still get access to features that are still interesting in the problem domain. constexpr functions, typesafe enums, static assertions, and destructors come to mind. No need to even touch a template or inheritance if you don't want to. The main benefits of C per se are ABI stability and availability of compilers.
- deleted 6y ago[deleted]
- Roritharr 6y agoInterestingly I have seen this pattern in all kinds of software businesses, but for most it tended to be less of a core business problem. I'm still wondering if the businesses would work better if the focus on technical excellency could be instilled and especially which parts of the company would have to suffer for it and what the ultimate outcome would be. I'm convinced that at least for some companies, it simply can't be the winning strategy.
- arthurcolle 6y agoI thought the actual approach taken these days is to disable GC for any JVM based trading systems and use arena based architectures or similar for application where you need super low latency, like a LOB mirror or execution engine, no?
- dehrmann 6y agoI've heard of people doing that 10-15 years ago. I'm sure it still works, but hopefully the GC situation has also improved.
- simonh 6y agoA pause is still a pause though. One system I know of had a solution. Throw in a huge amount of RAM on the server and delay garbage collection until end of day.
- overkalix 6y agoWhy waste money hire best engineers when more ram does trick?
- 2sk21 6y agoInteresting but doesn't allocation get slower as the heap increases?
- masklinn 6y agoNormally yes but that’s mostly a factor of heap fragmentation. It the GC never runs the “old” buffers are always full so you never bother going through them.
- maccard 6y agoBut a pause that can happen when you want it is acceptable. A 3 second stall might be totally acceptable as long as it happens outside of a trade, and you can guarantee that.
- angry_octet 6y agoThe worst (messiest, laziest and most chaotic) engineer I know works at a brokerage, coding obscenely large and unreliable edifices (and in a number of languages, from C++ to Excel+VB)... He does what he is asked, and doesn't worry about telling them what they need. They never demand code that doesn't crash, they see that as a fact of life, and developers are an overhead. Basically someone else owns the risk, the brokers don't, so why should he? People who don't like this style (or the code) leave. The direction for quality has to come from the top and be supported by systematic mechanisms. In fact that would be a question to ask in interviews.
- Karellen 6y ago^^This^^ is why you shouldn't call software developers "engineers" (disclaimer, am software developer). "No-one ever explicitly specified that the bridge shouldn't fall down and kill everyone who was on it at the time", said no engineer, ever.
- zanellato19 6y agoGiven the same conditions (i.e. people don't die) the same outcome would present itself. We have a history of bridges falling down, so much so that Wikipedia has a list on that: https://en.wikipedia.org/wiki/List_of_bridge_failures https://en.wikipedia.org/wiki/List_of_bridge_failures A particular one that caught my eye: Cimarron River Rail Crossing Dover, Oklahoma Territory United States 18 September 1906 Wooden railroad trestle Washed out under pressure from debris during high water 4-100+ killed Entire span lost; rebuilt Bridge was to be temporary, but replacement was delayed for financial reasons.[8][9][10] Number of deaths is uncertain; estimates range from 4 to over 100.[11] Emphasis mine. How is that different from software engineering?
- cobbzilla 6y agoThe sarcastic/negative implication of your example is that software engineering today is where civil engineering was 100+ years ago.
- roel_v 6y agoWhich one of these firms made the most money?
- vgatherps 6y agoThe third one was definitely the least, it was losing money (and specifically to trader-invisible things that never got worked on). The whole of the trading strategy (and capital traded, return profiles, etc) between the first two was fairly different, so it’s hard to compare. I would say though that the first firm was significantly outperforming its peers in a way the second firm did not, although both were very successful. The technology wasn’t the whole story in either case, but the first firm had significantly more and better opportunities as a result of just having a better stack all around
- Ragib_Zaman 6y agoIt's generally pretty rare for a market maker to lose money over a whole year. Colour me surprised.
- hnracer 6y agoTrue but I feel that the market maker label gets applied pretty loosely nowadays, couldve been a firm that mixes in a lot of intraday position taking trades that blew up
- lordnacho 6y agoThere's costs that aren't the PnL of buy-low-sell-high. The coders, the data, the infra. All cost money, a lot of money. Add to that there's a competitive aspect in the models interacting with each other, there's no slam-dunk. There's at least a couple of MMs struggling these days.
- deleted 6y ago[deleted]
- bboygravity 6y agoAm I understanding correctly that those firms really really care about latency & have unlimited money & chose C++/JAVA with a standard off-the-shelve compiler + an OS and/or JVM?? If you'd really really care about latency and have the resources, wouldn't it be more effective to just go bare metal? Bare metal as in: no OS (talk to the hardware registers directly from code), possibly a custom compiler, use all the relevant instructions that your CPU offers you. Or at the very least mess with the compiler to optimize it more to the specific make/model of the CPU/hardware that it will run on? What are the considerations in those companies for or against this?
- layoutIfNeeded 6y agoI used to work at a market maker where they used FPGAs to beat HFTs. Of course it was a constant arms race, so I’m not sure what they would be using nowadays...
- bboygravity 6y agoI would imagine they switched to ASICs which are more optimized than FPGAs (much smaller feature size, less general reconfigurable bloat, higher clocks, etc).
- vgatherps 6y agoOnly a few have, and even then only for a few fixed things. They have extreme versions of the inflexibility problems that fpgas have, and few exchanges have low enough jitter to justify ASICS over fpgas. It might surprise people that relatively few hfts do latency arb (and the ones that do often have some other structural advantage), and at that the ones are more and more mixing in serious short-term alpha. In that context speed is simply becoming table stakes for playing the game, and not the one true race to win it all.
- scatters 6y agoI don't know about java, but when using c++ you can use the OS to start the program, set up the hardware mappings and then move it to an isolated core where you talk to the hardware directly without ever issuing a system call. Compilers already offer flags to optimize for specific processor generations (and use the newly available instructions); you aren't going to be able to do better than that with a custom compiler.
- knuthsat 6y agoFrom my experience, JVM can't do much when there's a deep stack. I remember just refactoring all of foreach-loops/iterators into for-loops and got insane speedups. Functions with inefficient iterations would get optimized if called directly but deep in the call stack and nothing happens. These kinds of things are impossible with C++. It's possible to write efficient Java but you have to not use some of the language features to do so.
- iamcreasy 6y ago> I remember just refactoring all of foreach-loops/iterators into for-loops and got insane speedups. Wow! Can you please provide a simple example when/where it happens? > It's possible to write efficient Java but you have to not use some of the language features to do so. Please tell us more.
- knuthsat 6y agoLet's say you write an Iterable class so that you can use foreach on your data structure. It's quite possible iteration won't be completely inlined and the overhead of calling hasNext & next can become non-significant. An even worse case that made massive speedups is just inlining everything. I believe modern IDEs can automatically inline all invocations of a function. You'd be surprised how much performance you can get with that. As for writing efficient Java, it comes down to using primitive and value types. Although, my experience is mostly with Java 8. I have no idea how streams or other new syntax works. After optimizing a really massive Java code base I've never used it since.
- 95th 6y agoWell there is a middle ground - Rust
- aldanor 6y agoFrom what perspective exactly is Rust a middle ground Java and C++?
- splix 6y agoMaybe dev comfort like with Java, and performance like with C++?
- pjmlp 6y agoI like Rust, but it definitely isn't dev comfort like Java.
- phildawes 6y agoMemory safety with no unpredictable gc latency
- jandrewrogers 6y agoRust improves on Java in that it doesn't have a GC, so no pauses, and the code gen is similar to C++ in terms of performance. Rust is worse than C++ in that it struggles to express some idiomatic high-performance code architectures, such as using DMA for I/O, because they violate axioms of Rust's memory safety model, or in other cases because Rust currently lacks the language features. To make it work in Rust requires writing more code and disabling safety features. Rust fixes many language design and performance issues with Java, while offering some similar types of safety, but (like Java) it is missing some elements of expressiveness of C++ that are important for high-performance architectures.
- aldanor 6y agoHm... unsafe {} + use pointers and you're back to c++-style? You've had unsound code in one language, now you have unsound code in other language, how come it's suddenly worse other than having to type unsafe explicitly? I've done my fair share of C++, including low latency stuff, and in the grand scheme of things I'd say "expressiveness of C++" is completely overshadowed by its complexity and occasional ambiguity, lack of proper type system and proper generics, lack of proper module system, lack of dependency tracking, lack of a unified build systems, etc. I'm not exactly sure what you mean by expressiveness though.
- vmception 6y agoVHDL is faster no?
- vmception 6y agoVHDL is faster no? Isn’t everything you wrote from the wrong decade?
- vgatherps 6y agoFpgas have a huge amount of operational and practical difficulties. They’re great for fixed-function stuff like feed parsing or dumb triggers your software can direct in real-time, but absolutely terrible for something where you the ability to be flexible and adaptive. Pure latency arb isn’t the only hft trade there is anyways.