7 ms·

Speaking from experience with JVM HFT applications (we used Scala). There are a lot of tricks though to not require 1TB. And allocation in general is a bad id

Speaking from experience with JVM HFT applications (we used Scala). There are a lot of tricks though to not require 1TB. And allocation in general is a bad idea even if you don't collect because it scatters stuff all over memory and messes up cache locality. You really, really don't want to allocate in a performance sensitive jvm application if you can avoid it. It's the opposite of a lot of what I was told and taught (e.g. never do object pooling), but empirically, in my experience, allocations are the biggest slowdown. You can get an application a lot faster just by opening up the memory allocation tab in a jmc flightrecording and refactoring the biggest allocators, usually there is a lot of easy to optimize low hanging fruit that will give good performance improvements, even better than focusing on hot spots in code (in my personal experience). By far the biggest allocator in trading is going to be marketdata and calculations on it. For reading marketdata from the exchange it's best to leave raw data in memory and access it with a ByteBuffer / sun.misc.unsafe. Under this pattern classes have 1 value, the memory address to pass into sun.misc.unsafe, then everything from there on is done with offsets onto that address. For calculations it's better to write things as static functions, or use object pooling. In the course of optimizing a trading engine I wrote lots and lots of code to get allocations down to zero. It's definitely doable, but best done from the start, I refactored an existing trading engine to do that, it was not very fun.

Androider 6y ago
1TB of RAM is cheap though, versus spending engineering hours. Reminds me of the classic WTF "That would've been an option too" https://thedailywtf.com/articles/That-Wouldve-Been-an-Option-Too https://thedailywtf.com/articles/That-Wouldve-Been-an-Option...
- MangoCoffee 6y ago
  agree. hardware is getting cheaper and cheaper
  - ksec 6y ago
    Except Memory / DRAM.
- mroche 6y ago
  This was a great read, thanks for that! Never heard of The Daily WTF before.
  - petters 6y ago
    Congratulations! https://xkcd.com/1053/ https://xkcd.com/1053/
    - mroche 6y ago
      Of course there’s an XKCD for this! Ah, the wonders of the internet. The article in question was written in 2008: I was 11 years old. I guess I now fall in to the category of “heard of it by the age of 30”.
- jacques_chester 6y ago
  > 1TB of RAM is cheap though, versus spending engineering hours. It's not about the amount of RAM, it's how fast and predictable the overall system is. Note in particular the remark about cache locality; that can be the difference between nanoseconds and microseconds.
  - whizzter 6y ago
    What the GP is referring to (Misc.Unsafe) is basically going back to manual pointer reads/writes (IE writing more or less C code in Java), the benefit is you get C speed/memory layout/cache-locality but with the downside of writing it in a less suited language (Java). HOWEVER If you DO _allocate_ much then 1TB seems like a great choice since GC pauses are killers w.r.t. latency in comparison to cache issues. A cache line (often 64 bytes) would probably only hold at most a couple of Java objects (minimum is probably like 16 or more bytes for each small object) so cache locality won't be improved much by a GC (Yes, the G1 GC in newer JDK's does neighbour compacting iirc so you get a little cache locality but only if the patterns were bad from the start, but avoiding GC entirely is better)
    - apalmer 6y ago
      Can you explain why you would code java this way instead of dropping down to C?
      - whizzter 6y ago
        Probably because they started out in Java and don't want a full rewrite once they've gotten this far (might be that they don't have many C++ devs anyhow). I remember working on similar codebases 10 years back when J2ME games were still a thing (J2ME runtimes usually had horrible GC's and some people really went overboard in trying to avoid them) and it wasn't that fun at all (Even if the questDB codebase that is linked elsewhere in this thread seems to be saner). Also JNI for calling C/C++ code is fairly bad and error prone (if you want to combine code), JNA is better but i think this is one of the bigger reasons that Oracle is funding GraalVM is that it promises "seamless" interoperability and breaking up things into Java and C/C++ parts might be a good option in the future.
- deleted 6y ago
  [deleted]
- alexeiz 6y ago
  > ...a new intern named Bob. Bob raised a finger and said "Yes, I have a question... Damn those smart-ass interns!
the_only_law 6y ago
Do you know of any public examples of how this sort of Java looks? I imagine you lose out on being able to take advantage of much of the JVM ecosystem and I struggle to see what using Java even adds anymore.
- bluestreak 6y ago
  here is one: https://github.com/questdb/questdb https://github.com/questdb/questdb. Disclaimer, I work on this project. Main reason we use Java is speed of development (which increased with amount of base libraries written) and ease of testing.
  - whizzter 6y ago
    Would you do it again? Coming from C/C++ first, Java then and coming to C# it just feels so much more pragmatic with f.ex. struct(s), slices and stackalloc (and unsafe blocks with pointers in a pinch) allowing for GC less programming w/o resorting to turning pointers to integers and using function calls for memory access all around. (Noticed that you do i guess query compilation via the ASM toolkit?)
    - bluestreak 6y ago
      It is a good question. I feel there is a happy medium where boilerplate is in Java and more intricate data processing routines are in C++. Makes both worlds simpler. I like C++ better. Those things you mentioned - Java truly sucks at indeed, but you don't always need them. When it comes to IDE, testing, compilation speed, cross platform code and finding talent - Java is way easier than C++.
      - whizzter 6y ago
        Yeah i see that outlook, my comment was actually mostly about C# (been using it for the past year and these things above are in it as well as all the base being Java-ish)
- deleted 6y ago
  [deleted]
thom 6y ago
Ha, these are many of the same things one might do to make a chess engine fast on the JVM. This only strengthens my belief that the best computer science education you can get is studying chess programming.
touisteur 6y ago
I had a side project of 'flattening' classes at classloading time into large mappedbytebuffers, removing all allocations needed for ser-des and recursing on belongs-to references, copying buffer slices from sockets to the buffers (and reusing the socket buffers), and using netty for sockets (selector allocates crazy). Very fun project with javassist...

Androider 6y ago

1TB of RAM is cheap though, versus spending engineering hours. Reminds me of the classic WTF "That would've been an option too" https://thedailywtf.com/articles/That-Wouldve-Been-an-Option-Too https://thedailywtf.com/articles/That-Wouldve-Been-an-Option...

MangoCoffee 6y ago
agree. hardware is getting cheaper and cheaper
- ksec 6y ago
  Except Memory / DRAM.
mroche 6y ago
This was a great read, thanks for that! Never heard of The Daily WTF before.
- petters 6y ago
  Congratulations! https://xkcd.com/1053/ https://xkcd.com/1053/
  - mroche 6y ago
    Of course there’s an XKCD for this! Ah, the wonders of the internet. The article in question was written in 2008: I was 11 years old. I guess I now fall in to the category of “heard of it by the age of 30”.
jacques_chester 6y ago
> 1TB of RAM is cheap though, versus spending engineering hours. It's not about the amount of RAM, it's how fast and predictable the overall system is. Note in particular the remark about cache locality; that can be the difference between nanoseconds and microseconds.
- whizzter 6y ago
  What the GP is referring to (Misc.Unsafe) is basically going back to manual pointer reads/writes (IE writing more or less C code in Java), the benefit is you get C speed/memory layout/cache-locality but with the downside of writing it in a less suited language (Java). HOWEVER If you DO _allocate_ much then 1TB seems like a great choice since GC pauses are killers w.r.t. latency in comparison to cache issues. A cache line (often 64 bytes) would probably only hold at most a couple of Java objects (minimum is probably like 16 or more bytes for each small object) so cache locality won't be improved much by a GC (Yes, the G1 GC in newer JDK's does neighbour compacting iirc so you get a little cache locality but only if the patterns were bad from the start, but avoiding GC entirely is better)
  - apalmer 6y ago
    Can you explain why you would code java this way instead of dropping down to C?
    - whizzter 6y ago
      Probably because they started out in Java and don't want a full rewrite once they've gotten this far (might be that they don't have many C++ devs anyhow). I remember working on similar codebases 10 years back when J2ME games were still a thing (J2ME runtimes usually had horrible GC's and some people really went overboard in trying to avoid them) and it wasn't that fun at all (Even if the questDB codebase that is linked elsewhere in this thread seems to be saner). Also JNI for calling C/C++ code is fairly bad and error prone (if you want to combine code), JNA is better but i think this is one of the bigger reasons that Oracle is funding GraalVM is that it promises "seamless" interoperability and breaking up things into Java and C/C++ parts might be a good option in the future.
deleted 6y ago
[deleted]
alexeiz 6y ago
> ...a new intern named Bob. Bob raised a finger and said "Yes, I have a question... Damn those smart-ass interns!

the_only_law 6y ago

Do you know of any public examples of how this sort of Java looks? I imagine you lose out on being able to take advantage of much of the JVM ecosystem and I struggle to see what using Java even adds anymore.

bluestreak 6y ago
here is one: https://github.com/questdb/questdb https://github.com/questdb/questdb. Disclaimer, I work on this project. Main reason we use Java is speed of development (which increased with amount of base libraries written) and ease of testing.
- whizzter 6y ago
  Would you do it again? Coming from C/C++ first, Java then and coming to C# it just feels so much more pragmatic with f.ex. struct(s), slices and stackalloc (and unsafe blocks with pointers in a pinch) allowing for GC less programming w/o resorting to turning pointers to integers and using function calls for memory access all around. (Noticed that you do i guess query compilation via the ASM toolkit?)
  - bluestreak 6y ago
    It is a good question. I feel there is a happy medium where boilerplate is in Java and more intricate data processing routines are in C++. Makes both worlds simpler. I like C++ better. Those things you mentioned - Java truly sucks at indeed, but you don't always need them. When it comes to IDE, testing, compilation speed, cross platform code and finding talent - Java is way easier than C++.
    - whizzter 6y ago
      Yeah i see that outlook, my comment was actually mostly about C# (been using it for the past year and these things above are in it as well as all the base being Java-ish)
deleted 6y ago
[deleted]

thom 6y ago

Ha, these are many of the same things one might do to make a chess engine fast on the JVM. This only strengthens my belief that the best computer science education you can get is studying chess programming.

touisteur 6y ago

I had a side project of 'flattening' classes at classloading time into large mappedbytebuffers, removing all allocations needed for ser-des and recursing on belongs-to references, copying buffer slices from sockets to the buffers (and reusing the socket buffers), and using netty for sockets (selector allocates crazy). Very fun project with javassist...