5 ms·

I've researched this some a few months back. I don't blame you for saying that Java is wrong for HFT. My first reaction was the same, but there's much more to i

by dmos62 8y ago

I've researched this some a few months back. I don't blame you for saying that Java is wrong for HFT. My first reaction was the same, but there's much more to it. Java has a few things going for it in HFT. The obvious pluses are it's mature and memory safe. What's less obvious is that you can make it low-latency. It takes a lot of work, but it's doable, at which point you have all the nice things: mature ecosystem, speed, latency, safety. It takes a lot of work, because Java was always oriented towards server use cases, as in high-throughput, not low-latency. That's changing by the way, there are two new GC engines coming out that are low-latency oriented. Also, there's been third-party JVMs with low-latency guarantees for quite a while. Of course, what's between the lines is that there isn't any easy answers for HFT people. You either choose mature safety and do gc gymnastics (because everything is throughput oriented), or you choose manual memory management, which is its own gymnastics. Anyway, that's my take. I welcome input and contradictions.

bluGill 8y ago
Mostly I agree, but there is one factor that makes C++ (or any native compiled language) better than java for HFT: the ability to lie to the optimizer about what the hot path is. in HFT you have thousands of no trades for every trade, so the java optimizer will optimize the no-trade code path as more likely, then when the trade happens java pays a CPU branch prediction miss penalty at the only time low latency matters. You have to have good algorithms optimized to the max for this to matter though.
- dmos62 8y ago
  That's interesting, never thought about that.
  - bluGill 8y ago
    Unless you write HFT code, or follow talks by those writing HFT code you probably wouldn't. When you write HFT code you have to look at profiles and think about cache misses until branch missed become something to consider. If you don't come up with that idea someone else will and they will beat you to every trade and put you out of business.
- scott00 8y ago
  I was with you until you mentioned branch prediction... isn't branch prediction a hardware feature? How do you trick the HW branch predictor into predicting the unlikely case?
  - bluGill 8y ago
    Yes it is a hardware feature. However the hardware can be given hints as to which branch is more likely. This is generally documented by the manufacturer, in one of those technical documents aimed at compiler writers. With profile guided optimization it is possible for the compiler to have much better information about branches than the CPU can guess. Java applies profile guided optimization in real time, with C++ it much more complex to apply.
    - usefulcat 8y ago
      > However the hardware can be given hints as to which branch is more likely. I don't think that is the case for modern (last 8 years or so) Intel processors. For example, I'm under the impression that gcc's __builtin_expect only affects the layout of the generated code. However I'd love to learn something new here; do you have a source or any additional info you could share?
      - bluGill 8y ago
        My understanding is the layout is the hint. Modern CPUs would not want to use cache space for any hints, not to mention all the silicon to decode the hint. Compiler writers are smart people and can layout code to give the CPU the right hints, so long as they know what the CPU does. CPU manufactures want the compilers writers to do this as the compiler is a significant factor in making one CPU faster than the competition in benchmarks.
        hyperpape 8y ago
        It seems to me that it's misleading to call that a hint. Rather, certain code paths create hard-to-predict branches, and others create predictable branches. A hint would be some kind of metadata that says "you'll want to predict this branch this way based on your algorithm, but please don't!"
- hazeii 8y ago
  So, if I want to manipulate the market I need to work out (or poison) the hot path and have a system that's tailored to being faster on profitable - if colder - paths?
- BonesJustice 8y ago
  That’s indeed a problem, but it can be mitigated. If you have a low-latency trading component written in Java, a common trick is to continuously bombard it with ‘fake’ inputs to keep the desired code paths nice and hot. The fake inputs should be virtually indistinguishable from real ones that you would normally act on. The more subtle the difference, the better, e.g., just flip the sign on the timestamp field. You can use that subtle difference to pick whether the order goes out to the real exchange or a fake exchange. The decision needs to avoid actual branching instructions, though, or the JVM will likely optimize out the ‘real’ hot path, and you’ll fall back to interpreted mode when an actionable ‘real’ event comes in. I usually use a branchless selector to index into an array ([0] goes to a real socket, [1] goes to a black hole socket). You can also use this technique to make sure you can respond to very rare events quickly. For example, you may want to respond to news signals from Bloomberg. Actionable news is rare, so if you want to keep your news parsing/analysis code warmed up and in the cache, it needs to constantly be reacting to warmup data.
  - snazz 8y ago
    Interesting. Have you ever introduced an error in an attempt to lower latencies with this method?
    - BonesJustice 8y ago
      Thankfully, no (knocks on wood). But you have to base your design around the idea that real and fake trades are indistinguishable until the last possible moment. That critical requirement needs to always be in your mind. I would never try to bolt those kinds of optimizations onto an existing system. It’d be too easy to miss something.
mavelikara 8y ago
I have been out of the HFT space for many years now, but your conclusions make sense. Here is my own take on the subject from my short tenure at an HFT firm: https://news.ycombinator.com/item?id=12053159 https://news.ycombinator.com/item?id=12053159

by dmos62 8y ago

bluGill 8y ago

Mostly I agree, but there is one factor that makes C++ (or any native compiled language) better than java for HFT: the ability to lie to the optimizer about what the hot path is. in HFT you have thousands of no trades for every trade, so the java optimizer will optimize the no-trade code path as more likely, then when the trade happens java pays a CPU branch prediction miss penalty at the only time low latency matters. You have to have good algorithms optimized to the max for this to matter though.

dmos62 8y ago
That's interesting, never thought about that.
- bluGill 8y ago
  Unless you write HFT code, or follow talks by those writing HFT code you probably wouldn't. When you write HFT code you have to look at profiles and think about cache misses until branch missed become something to consider. If you don't come up with that idea someone else will and they will beat you to every trade and put you out of business.
scott00 8y ago
I was with you until you mentioned branch prediction... isn't branch prediction a hardware feature? How do you trick the HW branch predictor into predicting the unlikely case?
- bluGill 8y ago
  Yes it is a hardware feature. However the hardware can be given hints as to which branch is more likely. This is generally documented by the manufacturer, in one of those technical documents aimed at compiler writers. With profile guided optimization it is possible for the compiler to have much better information about branches than the CPU can guess. Java applies profile guided optimization in real time, with C++ it much more complex to apply.
  - usefulcat 8y ago
    > However the hardware can be given hints as to which branch is more likely. I don't think that is the case for modern (last 8 years or so) Intel processors. For example, I'm under the impression that gcc's __builtin_expect only affects the layout of the generated code. However I'd love to learn something new here; do you have a source or any additional info you could share?
    - bluGill 8y ago
      My understanding is the layout is the hint. Modern CPUs would not want to use cache space for any hints, not to mention all the silicon to decode the hint. Compiler writers are smart people and can layout code to give the CPU the right hints, so long as they know what the CPU does. CPU manufactures want the compilers writers to do this as the compiler is a significant factor in making one CPU faster than the competition in benchmarks.
      - hyperpape 8y ago
        It seems to me that it's misleading to call that a hint. Rather, certain code paths create hard-to-predict branches, and others create predictable branches. A hint would be some kind of metadata that says "you'll want to predict this branch this way based on your algorithm, but please don't!"
hazeii 8y ago
So, if I want to manipulate the market I need to work out (or poison) the hot path and have a system that's tailored to being faster on profitable - if colder - paths?
BonesJustice 8y ago
That’s indeed a problem, but it can be mitigated. If you have a low-latency trading component written in Java, a common trick is to continuously bombard it with ‘fake’ inputs to keep the desired code paths nice and hot. The fake inputs should be virtually indistinguishable from real ones that you would normally act on. The more subtle the difference, the better, e.g., just flip the sign on the timestamp field. You can use that subtle difference to pick whether the order goes out to the real exchange or a fake exchange. The decision needs to avoid actual branching instructions, though, or the JVM will likely optimize out the ‘real’ hot path, and you’ll fall back to interpreted mode when an actionable ‘real’ event comes in. I usually use a branchless selector to index into an array ([0] goes to a real socket, [1] goes to a black hole socket). You can also use this technique to make sure you can respond to very rare events quickly. For example, you may want to respond to news signals from Bloomberg. Actionable news is rare, so if you want to keep your news parsing/analysis code warmed up and in the cache, it needs to constantly be reacting to warmup data.
- snazz 8y ago
  Interesting. Have you ever introduced an error in an attempt to lower latencies with this method?
  - BonesJustice 8y ago
    Thankfully, no (knocks on wood). But you have to base your design around the idea that real and fake trades are indistinguishable until the last possible moment. That critical requirement needs to always be in your mind. I would never try to bolt those kinds of optimizations onto an existing system. It’d be too easy to miss something.

mavelikara 8y ago

I have been out of the HFT space for many years now, but your conclusions make sense. Here is my own take on the subject from my short tenure at an HFT firm: https://news.ycombinator.com/item?id=12053159 https://news.ycombinator.com/item?id=12053159