8 ms·

Before any one panics, please actually read the article. For some context, Peter Lawry is pretty famous for being the OpenHFT guy so he (rightfully) cares at sc

Before any one panics, please actually read the article. For some context, Peter Lawry is pretty famous for being the OpenHFT guy so he (rightfully) cares at scales that have very different concerns than general application code. He's working at the scale of 67 million events per second. Yea, for him a single allocation looks like : 67.8 M event/s, 472 ns vs. 50.1 M event/s, 638 ns As always with perf: scale and context matters! An allocation in these tests at these volumes was still only 160ns and at lower volumes was less than 50ns. These are closer to the scale of cache misses then a C malloc call.

ape4 4y ago
Aiming for that speed, in addition to avoiding creating objects per-transaction, you should avoid anything else that could be time consuming.
Karellen 4y ago
Even so, I wonder what advantages Java has, that makes using it in that environment so overwhelmingly compelling over other languages. Enough that it overcomes the fact that you have to write very non-idiomatic Java if you don't want to incur unacceptable performance costs. It would be interesting to see what other languages were considered for the use case, and which cost/benefit considerations made Java come out on top.
- another2another 4y ago
  >have to write very non-idiomatic Java if you don't want to incur unacceptable performance costs. You would also, I guess have to choose any library dependencies very carefully as well. Or implement the functionality yourself, negating any advantage that might bring in speeding up development.
- chmod775 4y ago
  > Even so, I wonder what advantages Java has, that makes using it in that environment so overwhelmingly compelling over other languages. Presumably not the whole thing needs to be that fast, only the part that actually handles the individual events. Java has the advantage of being an industry standard, or at least being very common in the financial sector. So you can work in a single language while only having to write some un-idiomatic Java in places where performance is critical.
- kjeetgill 4y ago
  Unacceptable performance costs are pretty contextual though. Plenty of code becomes a unidiomatic if you have to avoid a malloc() or new at all costs for example. HFT folks in any language tend to break idioms to wring every cycle they can out of a machine. I'd bet you'd be faster in well tuned C++ code, but why not asm then?
  - peter_lawrey 4y ago
    or FPGA
- StevePerkins 4y ago
  It's not so much the language as it is the virtual machine / runtime environment. The JVM is one of the most solid and heavily invested systems in the history of business software. Notable companies who started off with PHP or Python or what have you, reached a stage in their growth where they had to either migrate, or else write their own in-house compilers to fork those languages and make them scalable. No one's ever had the need to do this with Java (although different vendors do produce their own tweaked builds in order to compete in the support space). People generally assume that a virtual machine is a disadvantage over AOT compilation. And for the "To Do List" apps that so many students and hobbyists on these forums are writing, they are correct. But JIT compilation is vastly superior for long-running server side business processes. Which is the niche that Java owns (facing some competition from .NET, which works in the same manner). It's not sexy or beginner-friendly, so the level of discussion that it gets here is disproportionately low.
  - dec0dedab0de 4y ago
    JIT compilation is vastly superior for long-running server side business processes. How is that possible? Assuming all things are equal AOT should always be better. The advantage of a virtual machine is being able to distribute the same files to different processors and operating systems. The advantage of a JIT is that it makes your virtual machine faster.... oh how I wish Python would have an official JIT, sigh. But back to my question, if a JIT compiler takes bytecode and converts it to native code at runtime, and an AOT compiler takes source code and converts it to native code before being distributed, then how could JIT be faster? Is it that many people use AOT compilers in a generalized way instead of taking advantage of hardware specific optimizations? Or is the Java JIT compiler just so good at optimizing java byte code that it doesn't matter if it's compiled ahead of time or not? Edit: I got too many answers, but thank you all. I didn't consider the extra optimizations possible with runtime data. Please keep replying if you think of other reasons.
    - peter_lawrey 4y ago
      Something not generally mentioned is that Java can inline what would be virtual methods in C++, it can also remove them as behaviour changes or code is unloaded.
- kaba0 4y ago
  It is pretty much unparalleled in terms of the product of ecosystem size and performance. The other two languages in the top 3 are javascript and python, both have serious deficits regarding parallelism. As seen, it is a pretty strong contender in performance alone as well, with perhaps the best observability, and it has a well-defined execution semantics, even regarding erroneous execution — while any sort of memory error in a lower level language may silently corrupt the heap after which no assumption can hold regarding program state. Also, primitive-only java programming is probably no worse than similarly low-level C/C++/etc.
  - nesarkvechnep 4y ago
    Java’s observability is a joke when compared to Erlang’s.
    - kjeetgill 4y ago
      I'm not that familiar with it, I'd love to know more! I'd be curious what we could bring to Java land.
- tstrimple 4y ago
  One factor I've run into is the cost of offshore development for non-Java technologies. It's so much more difficult to find .NET Core developers in India than a Java developer for example. Forget about Rust or Go. Many enterprise development operations view developers as fungible which I disagree with, but it's much easier to pretend they are fungible if you're using something like Java rather than Rust.
  - blackoil 4y ago
    I don't think HFT are hiring from body shops. Of you want the top layer, you'll find Go, Rust devs in India but they'll not be cheap.
- origin_path 4y ago
  Java in HFT is a mix of things. Firstly: memory safety. You might not be allocating but you're benefiting from optimized bounds checks and type safety. If you're trading at HFT speeds then you need your program to throw an exception if something goes wrong, and not simply make a trade with a random bit of the heap sent to the exchange in the 'quantity' field. And you need it to be easily diagnosed when that happens, without taking your whole service offline. Secondly: these programs do actually allocate sometimes. They just don't do it (much) in the hot path in prod. For instance they'll happily write normal Java code that runs at startup to load data files, and they'll happily write normal code on cold paths that aren't hit all that often. They may well do allocations in test mode for logging, test verification etc. Some HFT shops do even allocate in the hot paths but they just size the heap so it doesn't collect during the trading day, and then they run a GC once markets close. Thirdly: HotSpot is a very easy way to get PGO which is normally considered to be a 10%-25% performance win. Getting it from C++ toolchains is possible, but hard, and Java has the advantage that if market conditions suddenly change and you start going down different codepaths it'll re-compile on the fly, whereas for C++ you'd have to wait for the next rebuild to re-establish an optimal hot path. Fourthly: not allocating isn't actually all that weird or non-idiomatic. Quite a lot of apps use this "allocate up front, but not in the hot path" approach. Check out Mindustry, it's a game written in Java that hits a solid 60fps at minimum and can easily hit hundreds of fps. It's just silky smooth even on a non-gaming laptop. How? Doesn't allocate in the rendering loop. Object pooling is hardly radical, most apps with real time latency requirements do that even in C++.
- speed_spread 4y ago
  > non-idiomatic Java I'd contend that non-allocating superfast code in any language ends up looking kind of odd. High Frequency Trade programs have a lot in common with embedded microcontroller apps. Except they're also doing business transactions.

ape4 4y ago

Aiming for that speed, in addition to avoiding creating objects per-transaction, you should avoid anything else that could be time consuming.

Karellen 4y ago

Even so, I wonder what advantages Java has, that makes using it in that environment so overwhelmingly compelling over other languages. Enough that it overcomes the fact that you have to write very non-idiomatic Java if you don't want to incur unacceptable performance costs. It would be interesting to see what other languages were considered for the use case, and which cost/benefit considerations made Java come out on top.

another2another 4y ago
>have to write very non-idiomatic Java if you don't want to incur unacceptable performance costs. You would also, I guess have to choose any library dependencies very carefully as well. Or implement the functionality yourself, negating any advantage that might bring in speeding up development.
chmod775 4y ago
> Even so, I wonder what advantages Java has, that makes using it in that environment so overwhelmingly compelling over other languages. Presumably not the whole thing needs to be that fast, only the part that actually handles the individual events. Java has the advantage of being an industry standard, or at least being very common in the financial sector. So you can work in a single language while only having to write some un-idiomatic Java in places where performance is critical.
kjeetgill 4y ago
Unacceptable performance costs are pretty contextual though. Plenty of code becomes a unidiomatic if you have to avoid a malloc() or new at all costs for example. HFT folks in any language tend to break idioms to wring every cycle they can out of a machine. I'd bet you'd be faster in well tuned C++ code, but why not asm then?
- peter_lawrey 4y ago
  or FPGA
StevePerkins 4y ago
It's not so much the language as it is the virtual machine / runtime environment. The JVM is one of the most solid and heavily invested systems in the history of business software. Notable companies who started off with PHP or Python or what have you, reached a stage in their growth where they had to either migrate, or else write their own in-house compilers to fork those languages and make them scalable. No one's ever had the need to do this with Java (although different vendors do produce their own tweaked builds in order to compete in the support space). People generally assume that a virtual machine is a disadvantage over AOT compilation. And for the "To Do List" apps that so many students and hobbyists on these forums are writing, they are correct. But JIT compilation is vastly superior for long-running server side business processes. Which is the niche that Java owns (facing some competition from .NET, which works in the same manner). It's not sexy or beginner-friendly, so the level of discussion that it gets here is disproportionately low.
- dec0dedab0de 4y ago
  JIT compilation is vastly superior for long-running server side business processes. How is that possible? Assuming all things are equal AOT should always be better. The advantage of a virtual machine is being able to distribute the same files to different processors and operating systems. The advantage of a JIT is that it makes your virtual machine faster.... oh how I wish Python would have an official JIT, sigh. But back to my question, if a JIT compiler takes bytecode and converts it to native code at runtime, and an AOT compiler takes source code and converts it to native code before being distributed, then how could JIT be faster? Is it that many people use AOT compilers in a generalized way instead of taking advantage of hardware specific optimizations? Or is the Java JIT compiler just so good at optimizing java byte code that it doesn't matter if it's compiled ahead of time or not? Edit: I got too many answers, but thank you all. I didn't consider the extra optimizations possible with runtime data. Please keep replying if you think of other reasons.
  - peter_lawrey 4y ago
    Something not generally mentioned is that Java can inline what would be virtual methods in C++, it can also remove them as behaviour changes or code is unloaded.
kaba0 4y ago
It is pretty much unparalleled in terms of the product of ecosystem size and performance. The other two languages in the top 3 are javascript and python, both have serious deficits regarding parallelism. As seen, it is a pretty strong contender in performance alone as well, with perhaps the best observability, and it has a well-defined execution semantics, even regarding erroneous execution — while any sort of memory error in a lower level language may silently corrupt the heap after which no assumption can hold regarding program state. Also, primitive-only java programming is probably no worse than similarly low-level C/C++/etc.
- nesarkvechnep 4y ago
  Java’s observability is a joke when compared to Erlang’s.
  - kjeetgill 4y ago
    I'm not that familiar with it, I'd love to know more! I'd be curious what we could bring to Java land.
tstrimple 4y ago
One factor I've run into is the cost of offshore development for non-Java technologies. It's so much more difficult to find .NET Core developers in India than a Java developer for example. Forget about Rust or Go. Many enterprise development operations view developers as fungible which I disagree with, but it's much easier to pretend they are fungible if you're using something like Java rather than Rust.
- blackoil 4y ago
  I don't think HFT are hiring from body shops. Of you want the top layer, you'll find Go, Rust devs in India but they'll not be cheap.
origin_path 4y ago
Java in HFT is a mix of things. Firstly: memory safety. You might not be allocating but you're benefiting from optimized bounds checks and type safety. If you're trading at HFT speeds then you need your program to throw an exception if something goes wrong, and not simply make a trade with a random bit of the heap sent to the exchange in the 'quantity' field. And you need it to be easily diagnosed when that happens, without taking your whole service offline. Secondly: these programs do actually allocate sometimes. They just don't do it (much) in the hot path in prod. For instance they'll happily write normal Java code that runs at startup to load data files, and they'll happily write normal code on cold paths that aren't hit all that often. They may well do allocations in test mode for logging, test verification etc. Some HFT shops do even allocate in the hot paths but they just size the heap so it doesn't collect during the trading day, and then they run a GC once markets close. Thirdly: HotSpot is a very easy way to get PGO which is normally considered to be a 10%-25% performance win. Getting it from C++ toolchains is possible, but hard, and Java has the advantage that if market conditions suddenly change and you start going down different codepaths it'll re-compile on the fly, whereas for C++ you'd have to wait for the next rebuild to re-establish an optimal hot path. Fourthly: not allocating isn't actually all that weird or non-idiomatic. Quite a lot of apps use this "allocate up front, but not in the hot path" approach. Check out Mindustry, it's a game written in Java that hits a solid 60fps at minimum and can easily hit hundreds of fps. It's just silky smooth even on a non-gaming laptop. How? Doesn't allocate in the rendering loop. Object pooling is hardly radical, most apps with real time latency requirements do that even in C++.
speed_spread 4y ago
> non-idiomatic Java I'd contend that non-allocating superfast code in any language ends up looking kind of odd. High Frequency Trade programs have a lot in common with embedded microcontroller apps. Except they're also doing business transactions.