10 ms·

I think I actually saw these folks present at JavaOne a couple years ago? Either that or there's more than one shop branding itself as "HFT" that uses Java. I

I think I actually saw these folks present at JavaOne a couple years ago? Either that or there's more than one shop branding itself as "HFT" that uses Java. I worked in the industry and it's always a little funny to see who calls themselves HFTs vs quants. Basically, there's a bit of a spectrum of fast vs smart. In general it's hard to do incredibly smart stuff fast enough to compete in the "speed-critical" bucket of trades and vice-versa there's barely any point in being ultra-fast in the "non-speed-critical" bucket because your alphas last for minutes to hours. Just from this read, I feel like these folks are just a hair to the right of "fast" in the [fast]---------[smart] continuum. I mostly make this appraisal based on these paragraphs: >To gain those few crucial microseconds, most players invest in expensive hardware: pools of servers with overclocked liquid-cooled CPUs (in 2020 you can buy a server with 56 cores at 5.6 GHz and 1 TB RAM), collocation in major exchange datacentres, high-end nanosecond network switches, dedicated sub-oceanic lines (Hibernian Express is a major provider), even microwave networks. >It’s common to see highly customised Linux kernels with OS bypass so that the data “jumps” directly from the network card to the application, IPC (Interprocess communication) and even FPGAs (programmable single-purpose chips). That's nice but that's where the cutting edge of the speed game was in 2007ish. Everything mentioned here is table stakes at this point (colocation, dedicated fiber, expensive switches, bypassing the kernel in network code, etc). The fact that "even FPGAs" is listed as "even" is the biggest thing I focus on. FPGA's and/or custom silicon is where the speed game is right now. Similarly, "even microwave networks" is also table stakes at this point (you can get on nasdaq's wireless[0] just by paying). This is the kind of game where capex for technology is dwarfed by the margin you're slinging around every day in trading, so you see some pretty absurd hardware justified. [0] http://n.nasdaq.com/WirelessConnectivitySuite http://n.nasdaq.com/WirelessConnectivitySuite Edit: Also shout-out to a different comment in this thread mentioning ISLD, a story I considered telling as well: https://news.ycombinator.com/item?id=24896603 https://news.ycombinator.com/item?id=24896603

oppositelock 6y ago
I discovered something amazing when working with some people who were writing HFT software. Why do you need 1TB of RAM in these machines? Because when you're Java based, you want to avoid stop-the-world GC pauses. These trading systems only have to be up from 9:30AM-4:30PM EST, so they simply disable GC altogether! At the end of a trading day, restart the app or reboot the system.
- anthonypasq 6y ago
  why not just use C++ or something and never deallocate memory then?
  - layer8 6y ago
    Malloc is usually slower than GC-based allocation (which basically just increments a pointer). Of course, one can emulate this via custom allocators in C++. My guess is that Java development is just easier and quicker, and the JIT may even result in more effective optimizations than AOT compilation.
    - mathieubordere 6y ago
      in C++ they could also try using Profile-guided optimization to optimize hot code paths.
    - amelius 6y ago
      In C++ you can link with your own version of malloc (one which just returns the next consecutive block, and can be implemented simply by adding the allocated size to a pointer).
  - oppositelock 6y ago
    You generally choose the language which has the libraries and ecosystem which solves your problem. For instance, you'd be silly to use anything but Java to work with Hadoop. This is the pragmatic choice.
    - ptx 6y ago
      But as soon as you make use of almost any library from the ecosystem, you get allocations. Doesn't that defeat the purpose? Why not isolate the Java-using part in a separate process from the non-allocating fast part?
- dcolkitt 6y ago
  Even if you completely disable GC, Java is still allocating tons of ephemeral objects on the heap. Which in turn is leading to expensive and unpredictable page faults. In contrast, C++'s default is to allocate objects on the heap unless you knowingly call new/malloc. Of course, it's possible to write Java in such a way to minimize this type of heap-thrashing. But by that point, you're already doing the equivalent of C++'s manual memory book-keeping anyway. It's not like "just provision a ton of memory and turn off GC" is a free lunch.
taion 6y ago
There are still a number of players in the HFT space that use JVM languages. I think as with any space, if you really zoom into the details, you see a lot of diversity. You're obviously right in that people who are still hitting the CPU can't compete with people that do everything on FPGA, but it seems like there's still plenty of money to be made by people who are just a bit less fast than that. I've heard people argue that they prefer being in that sort of space, too, as it gives them a bit more room to compete on their own alphas, and tends to be a bit less winner-take-all.
- ifdefdebug 6y ago
  I think GP had that covered mentioning a [fast]<--->[smart] continuum.
- cashewchoo 6y ago
  Yeah absolutely. There's money to be made across the spectrum of fast and smart. My experience is just way on the left side, where the special sauce is only barely statistics-y but extremely technical (both from a software/hardware standpoint and market microstructure saviiness standpoint). I'm not trying to gatekeep who can call themselves HFTs or not. The main thing I find funny is if you ask 10 firms if they're HFT or quant shops, it will probably not actually line up all that well with exactly how many orders they send or how speed-sensitive they are.
- deleted 6y ago
  [deleted]
- kyawzazaw 6y ago
  In a info session, IMC mentioned they use Java for some of their systems.
  - deleted 6y ago
    [deleted]
dcolkitt 6y ago
You're making good points, but one thing I want to emphasize is that latency is not the sole dimension of competition in HFT space. Certain things may just be "table-stakes", but for many strategies table-stakes is the only requirement. Simply being "fast enough" may be fine if you have a smarter model, exploit niche opportunities overlooked by others, or are willing to shoulder certain risks that other HFTs are trying to offload. To take this to the extreme, look at Renaissance's Medallion fund. It's certainly the case that much of what they're doing is "HFT-ish" in the sense that they're executing high turnover, high Sharpe strategies with short holding periods. Yet they've managed to continue to be successful from the 90s well until the age of HFT, without competing very hard on latency at all. Markets are an ecosystem, and like biological ecosystems there isn't just one trait that predicts dominance.
- unwoundmouse 6y ago
  what's your source for the fact that they are not competing on latency?
  - qeternity 6y ago
    It’s difficult to compete on latency and remain anonymous.
- vmception 6y ago
  correct, there is a peculiar nuance on financial trading topics where people conflate "computer executed trade" with "high frequency trading" you may need to rapidly adjust the expected fill price at a high frequency on a multi leg strategy being run in 100 positions, but the actual sending orders to the exchange doesn't need to be high frequency, and it is an important distinction that you wouldn't be competing with others on the frequency at this stage, it doesn't matter if this occurs in 1 millisecond of 500 milliseconds, even a couple of seconds. Very different ballgame than the wishful femtosecond game. its probably better that people don't understand that, but it is annoying that there are this many roadblocks to a nuanced conversation either way your servers still have to have all the authentication code, and various algorithms running to monitor the tape.
darksaints 6y ago
There is no singular definition of HFT. The extreme end of low latency is really only applicable for market making systems, but there are plenty of systems in the HFT sphere that do not rely on market making latencies. For example, those microwave links you're talking about? Those are specifically for cross exchange arbitrage. In that domain acceptable latencies go up quite a bit, and general purpose computing is still quite competitive.
howlin 6y ago
> Basically, there's a bit of a spectrum of fast vs smart. In general it's hard to do incredibly smart stuff fast enough to compete in the "speed-critical" bucket of trades and vice-versa there's barely any point in being ultra-fast in the "non-speed-critical" bucket because your alphas last for minutes to hours. Depending on the overall strategy, you can be smart and fast at once. If you have occasional alpha harvesting opportunities that must be acted on quickly (very low latency but also low-ish frequency), then it is possible to spend the time in between trades modeling market context and developing optimal short-term plans for how to react in case of various market triggers.
- traK6Dcm 6y ago
  I think he means smart as in "how long your model prediction takes". If your neural net (haven't actually met anyone who uses these in trading) takes 5ms to make a prediction that'll lock you out of a whole lot of trading opportunities/strategies. Speed always matters, no matter where on the smartness spectrum you are, but it's relative. If your model prediction takes 5ms you're not getting much ROI out of investing $1M into shaving off 50ns in your data processing. But if your end-to-end latency without prediction is 1ms, you better invest in getting that down.
  - howlin 6y ago
    > If your neural net (haven't actually met anyone who uses these in trading) takes 5ms to make a prediction that'll lock you out of a whole lot of trading opportunities/strategies. Let's say you have trading opportunities once every 100ms. They need to be acted upon within 2ms or they vanish. You don't have the time budget to run a NNet every time the state of the market changes. You can, however, train a NNet to output a very small decision tree that can run in under 1ms. The NNet can then decide which "micro-strategy" in the form of a much faster and more reactive decision tree is more appropriate for the current market context.
lordnacho 6y ago
That's right, the more to the fast end you are, the more your strategy is obvious. I worked at an HFT where it was literally buy-here-sell-there for one of the strategies, and of course you can't do that without being ridiculously fast. DSquare is reasonably famous in London, somehow everyone knows them. They are on the smarter end of things, because the founders were plugged into the early development of the electronic FX market at the beginning of this century. There's a sort of engineering-vs-business culture thing to this as well, but I think it blurs over time.
Spinnaker_ 6y ago
Nice overview. The fast vs smart comparison is apt. But something that gets missed in these conversations is what trading system architectures actually look like. The part where latency is measured in nanoseconds is just the tip of the iceberg. Java and 2007 tech is absolutely acceptable for most of the architecture. Another thing is the structure of Equity and Derivative markets vs FX. The former is fairly standardized. You'll need infrastructure at a handful of exchanges in 2 or 3 cities. Microwave networks and FPGA's are the norm. But for FX markets, where these guys trade, the picture is much more messy. There are countless places to trade. Co-location means different things at all of them. The entire structure is much less regulated and understood. As a result, strategies and trading system architecture looks much different.
- deleted 6y ago
  [deleted]
jloveless 6y ago
Agreed. A lot of this is all standard now - even when I wrote about it circa 2013 [1] [1] https://queue.acm.org/detail.cfm?id=2536492 https://queue.acm.org/detail.cfm?id=2536492
eneveu 6y ago
I think the JavaOne presentation was made by the LMAX devs: https://www.youtube.com/watch?v=eTeWxZvlCZ8 https://www.youtube.com/watch?v=eTeWxZvlCZ8 They built the Disruptor data structure around 2011 for their high performance financial exchange on the JVM: https://lmax-exchange.github.io/disruptor/files/Disruptor-1.0.pdf https://lmax-exchange.github.io/disruptor/files/Disruptor-1.... I used the Disruptor at a smart grid startup in 2012-2014, after LMAX open sourced it. Martin Thompson has a lot of interesting presentations on the concept of mechanical sympathy: - https://www.youtube.com/watch?v=929OrIvbW18 https://www.youtube.com/watch?v=929OrIvbW18 - Adventures with concurrent programming in Java: A quest for predictable latency by Martin Thompson - https://www.youtube.com/watch?v=03GsLxVdVzU https://www.youtube.com/watch?v=03GsLxVdVzU - Designing for Performance by Martin Thompson
anthony_barker 6y ago
Flexibility in code is important. When markets change it is good to be able to quickly rework your code. Sometimes depending on the market this is more important than pure speed. Market data is the slow monster - especially in the options market - not order entry. This is where the fpgas come in to help. Often I have seen trading groups throw money at infrastructure when it was really just that a competitor (a virtu or citadel) has a better way of internalizing order flow or managing risk so they can pay more for the same trade.

oppositelock 6y ago

I discovered something amazing when working with some people who were writing HFT software. Why do you need 1TB of RAM in these machines? Because when you're Java based, you want to avoid stop-the-world GC pauses. These trading systems only have to be up from 9:30AM-4:30PM EST, so they simply disable GC altogether! At the end of a trading day, restart the app or reboot the system.

anthonypasq 6y ago
why not just use C++ or something and never deallocate memory then?
- layer8 6y ago
  Malloc is usually slower than GC-based allocation (which basically just increments a pointer). Of course, one can emulate this via custom allocators in C++. My guess is that Java development is just easier and quicker, and the JIT may even result in more effective optimizations than AOT compilation.
  - mathieubordere 6y ago
    in C++ they could also try using Profile-guided optimization to optimize hot code paths.
  - amelius 6y ago
    In C++ you can link with your own version of malloc (one which just returns the next consecutive block, and can be implemented simply by adding the allocated size to a pointer).
- oppositelock 6y ago
  You generally choose the language which has the libraries and ecosystem which solves your problem. For instance, you'd be silly to use anything but Java to work with Hadoop. This is the pragmatic choice.
  - ptx 6y ago
    But as soon as you make use of almost any library from the ecosystem, you get allocations. Doesn't that defeat the purpose? Why not isolate the Java-using part in a separate process from the non-allocating fast part?
dcolkitt 6y ago
Even if you completely disable GC, Java is still allocating tons of ephemeral objects on the heap. Which in turn is leading to expensive and unpredictable page faults. In contrast, C++'s default is to allocate objects on the heap unless you knowingly call new/malloc. Of course, it's possible to write Java in such a way to minimize this type of heap-thrashing. But by that point, you're already doing the equivalent of C++'s manual memory book-keeping anyway. It's not like "just provision a ton of memory and turn off GC" is a free lunch.

taion 6y ago

There are still a number of players in the HFT space that use JVM languages. I think as with any space, if you really zoom into the details, you see a lot of diversity. You're obviously right in that people who are still hitting the CPU can't compete with people that do everything on FPGA, but it seems like there's still plenty of money to be made by people who are just a bit less fast than that. I've heard people argue that they prefer being in that sort of space, too, as it gives them a bit more room to compete on their own alphas, and tends to be a bit less winner-take-all.

ifdefdebug 6y ago
I think GP had that covered mentioning a [fast]<--->[smart] continuum.
cashewchoo 6y ago
Yeah absolutely. There's money to be made across the spectrum of fast and smart. My experience is just way on the left side, where the special sauce is only barely statistics-y but extremely technical (both from a software/hardware standpoint and market microstructure saviiness standpoint). I'm not trying to gatekeep who can call themselves HFTs or not. The main thing I find funny is if you ask 10 firms if they're HFT or quant shops, it will probably not actually line up all that well with exactly how many orders they send or how speed-sensitive they are.
deleted 6y ago
[deleted]
kyawzazaw 6y ago
In a info session, IMC mentioned they use Java for some of their systems.
- deleted 6y ago
  [deleted]

dcolkitt 6y ago

You're making good points, but one thing I want to emphasize is that latency is not the sole dimension of competition in HFT space. Certain things may just be "table-stakes", but for many strategies table-stakes is the only requirement. Simply being "fast enough" may be fine if you have a smarter model, exploit niche opportunities overlooked by others, or are willing to shoulder certain risks that other HFTs are trying to offload. To take this to the extreme, look at Renaissance's Medallion fund. It's certainly the case that much of what they're doing is "HFT-ish" in the sense that they're executing high turnover, high Sharpe strategies with short holding periods. Yet they've managed to continue to be successful from the 90s well until the age of HFT, without competing very hard on latency at all. Markets are an ecosystem, and like biological ecosystems there isn't just one trait that predicts dominance.

unwoundmouse 6y ago
what's your source for the fact that they are not competing on latency?
- qeternity 6y ago
  It’s difficult to compete on latency and remain anonymous.
vmception 6y ago
correct, there is a peculiar nuance on financial trading topics where people conflate "computer executed trade" with "high frequency trading" you may need to rapidly adjust the expected fill price at a high frequency on a multi leg strategy being run in 100 positions, but the actual sending orders to the exchange doesn't need to be high frequency, and it is an important distinction that you wouldn't be competing with others on the frequency at this stage, it doesn't matter if this occurs in 1 millisecond of 500 milliseconds, even a couple of seconds. Very different ballgame than the wishful femtosecond game. its probably better that people don't understand that, but it is annoying that there are this many roadblocks to a nuanced conversation either way your servers still have to have all the authentication code, and various algorithms running to monitor the tape.

darksaints 6y ago

There is no singular definition of HFT. The extreme end of low latency is really only applicable for market making systems, but there are plenty of systems in the HFT sphere that do not rely on market making latencies. For example, those microwave links you're talking about? Those are specifically for cross exchange arbitrage. In that domain acceptable latencies go up quite a bit, and general purpose computing is still quite competitive.

howlin 6y ago

> Basically, there's a bit of a spectrum of fast vs smart. In general it's hard to do incredibly smart stuff fast enough to compete in the "speed-critical" bucket of trades and vice-versa there's barely any point in being ultra-fast in the "non-speed-critical" bucket because your alphas last for minutes to hours. Depending on the overall strategy, you can be smart and fast at once. If you have occasional alpha harvesting opportunities that must be acted on quickly (very low latency but also low-ish frequency), then it is possible to spend the time in between trades modeling market context and developing optimal short-term plans for how to react in case of various market triggers.

traK6Dcm 6y ago
I think he means smart as in "how long your model prediction takes". If your neural net (haven't actually met anyone who uses these in trading) takes 5ms to make a prediction that'll lock you out of a whole lot of trading opportunities/strategies. Speed always matters, no matter where on the smartness spectrum you are, but it's relative. If your model prediction takes 5ms you're not getting much ROI out of investing $1M into shaving off 50ns in your data processing. But if your end-to-end latency without prediction is 1ms, you better invest in getting that down.
- howlin 6y ago
  > If your neural net (haven't actually met anyone who uses these in trading) takes 5ms to make a prediction that'll lock you out of a whole lot of trading opportunities/strategies. Let's say you have trading opportunities once every 100ms. They need to be acted upon within 2ms or they vanish. You don't have the time budget to run a NNet every time the state of the market changes. You can, however, train a NNet to output a very small decision tree that can run in under 1ms. The NNet can then decide which "micro-strategy" in the form of a much faster and more reactive decision tree is more appropriate for the current market context.

lordnacho 6y ago

That's right, the more to the fast end you are, the more your strategy is obvious. I worked at an HFT where it was literally buy-here-sell-there for one of the strategies, and of course you can't do that without being ridiculously fast. DSquare is reasonably famous in London, somehow everyone knows them. They are on the smarter end of things, because the founders were plugged into the early development of the electronic FX market at the beginning of this century. There's a sort of engineering-vs-business culture thing to this as well, but I think it blurs over time.

Spinnaker_ 6y ago

Nice overview. The fast vs smart comparison is apt. But something that gets missed in these conversations is what trading system architectures actually look like. The part where latency is measured in nanoseconds is just the tip of the iceberg. Java and 2007 tech is absolutely acceptable for most of the architecture. Another thing is the structure of Equity and Derivative markets vs FX. The former is fairly standardized. You'll need infrastructure at a handful of exchanges in 2 or 3 cities. Microwave networks and FPGA's are the norm. But for FX markets, where these guys trade, the picture is much more messy. There are countless places to trade. Co-location means different things at all of them. The entire structure is much less regulated and understood. As a result, strategies and trading system architecture looks much different.

deleted 6y ago
[deleted]

jloveless 6y ago

Agreed. A lot of this is all standard now - even when I wrote about it circa 2013 [1] [1] https://queue.acm.org/detail.cfm?id=2536492 https://queue.acm.org/detail.cfm?id=2536492

eneveu 6y ago

I think the JavaOne presentation was made by the LMAX devs: https://www.youtube.com/watch?v=eTeWxZvlCZ8 https://www.youtube.com/watch?v=eTeWxZvlCZ8 They built the Disruptor data structure around 2011 for their high performance financial exchange on the JVM: https://lmax-exchange.github.io/disruptor/files/Disruptor-1.0.pdf https://lmax-exchange.github.io/disruptor/files/Disruptor-1.... I used the Disruptor at a smart grid startup in 2012-2014, after LMAX open sourced it. Martin Thompson has a lot of interesting presentations on the concept of mechanical sympathy: - https://www.youtube.com/watch?v=929OrIvbW18 https://www.youtube.com/watch?v=929OrIvbW18 - Adventures with concurrent programming in Java: A quest for predictable latency by Martin Thompson - https://www.youtube.com/watch?v=03GsLxVdVzU https://www.youtube.com/watch?v=03GsLxVdVzU - Designing for Performance by Martin Thompson

anthony_barker 6y ago

Flexibility in code is important. When markets change it is good to be able to quickly rework your code. Sometimes depending on the market this is more important than pure speed. Market data is the slow monster - especially in the options market - not order entry. This is where the fpgas come in to help. Often I have seen trading groups throw money at infrastructure when it was really just that a competitor (a virtu or citadel) has a better way of internalizing order flow or managing risk so they can pay more for the same trade.