26 ms·

For years of my life, all I thought about was stuff like this. If you've ever ran latency sensitive systems on the JVM.. man is it ever a pain. Who was it that

For years of my life, all I thought about was stuff like this. If you've ever ran latency sensitive systems on the JVM.. man is it ever a pain. Who was it that turned GC off entirely, minimized allocation and just restarted their VMs when they ran out of RAM every couple of hours, was that Netflix? Either way. It makes me excited for Rust and the languages it'll inspire, all this labor gone away.

pron 6y ago
In case you missed this post and the previous ones, here's the news: this labour is gone on the JVM. The GCs in JDK 14-15 are good enough pretty much out of the box, and they're getting better very quickly now. OTOH, if you think other languages let you do away with a GC without pretty significant extra work, especially in concurrent systems, well, then you haven't had experience with those languages.
- VHRanger 6y ago
  In D, the GC is guaranteed to only ever run if you allocate new memory on the heap. This is still a lot of work for a video game (because you never want any latency, the only way to achieve this is with an arena allocator or going full @nogc). But for apps where the latency requirement is bounded, D doesn't make it hard and the language is nice and ergonomic.
  - chrisseaton 6y ago
    > In D, the GC is guaranteed to only ever run if you allocate new memory on the heap. Isn't that how most GCs work? Why would you do anything else? To release memory to the OS? That's not really a priority in most runtime systems, and I think wanting to do that is a pretty niche requirement.
    - frant-hartm 6y ago
      I want memory allocation to run in constant time, so I can achieve low latency. That's the point of concurrent GCs - they run in the background to free up memory.
      - chrisseaton 6y ago
        I know they run in the background... but I don't think they run in the background if there's nothing to do, do they?
        imtringued 6y ago
        A JVM with default settings will happily allocate more memory without attempting to even run a single GC cycle until it hits the configured heap size. Once it hits the limit it will try to run the GC.
        deleted 6y ago
        [deleted]
    - nemothekid 6y ago
      In Java, everything is heap allocated, whereas in D you have a choice.
      - chrisseaton 6y ago
        > everything is heap allocated You can definitely write useful Java applications where nothing gets heap allocated, through using existing objects, and through using scalar-replacement-of-aggregates.
      - wokkel 6y ago
        Not entirely correct: https://shipilev.net/jvm/anatomy-quarks/18-scalar-replacement/ https://shipilev.net/jvm/anatomy-quarks/18-scalar-replacemen... . Hoi cannot force a specific allocation, but the jit does do something akin to stack allocation.
    - readittwice 6y ago
      Releasing memory is definitely useful for mobile devices or browsers. Even in some server use cases it's useful, e.g. when you pay for memory usage. I guess that's why the JVM has this -XX:SoftMaxHeapSize option. I guess it depends whether GCs are always scheduled in an allocation or can be triggered another way. Either way that should be easy to disable. I read somewhere that D doesn't have write barriers, so I would assume they have a hard time implementing more advanced GC features like generational collection or concurrent marking. It's not suprising that the GCs in the JVM achieve much better pause time.
- tjoff 6y ago
  That statement has been said for pretty much every release of any GC. I'd say the opposite is true. Relying on GC requires significant extra work. Because you always need to think about memory (exception: small script-like applications). The only thing a GC does is that it enable you to not think about it, but the moment you don't you will write bad code and realize it was a disservice all along. And by then it is too late. So in a GC language you need to constantly be aware of when you take something for granted. Which is more work than just doing it manually yourself.
  - RhodesianHunter 6y ago
    This is not really true at all though, nor is it how people who design performance intensive systems go about it. generally you design your application the way you normally would, do some testing, and then go back through the hot path and make some adjustments.
  - pron 6y ago
    First, a GC allows you to abstract over memory allocations (i.e. hide them). This makes maintaining code over time significantly easier, as allocation becomes a hidden implementation detail. Second, while there have been similar claims made in the past, they always apply for a certain target throughput and latency. GCs are constantly making great strides in that regard. Next year, ZGC will have a <1ms worst-case latency for pretty serious allocation rates, and with an acceptable hit to throughput. As you can see in this post and previous ones, G1 offers great throughput with acceptable latencies.
    - jakewins 6y ago
      This GC you speak of that hides allocations as an implementation detail sounds excellent! Every GC I've worked with - and my day job is orchstrating several thousand JVMs - simply moves the time when you need to think of allocations somewhere else. Either it moves it into PagerDuty, like G1, or it moves it into your GCP bill like Shenandoa and ZGC. The trade-off has always been latency-throughput-footprint, nothing I've seen yet has changed that. The innovation in Rust is realizing you can do all the tracing work at compile time.
- twic 6y ago
  They're good enough for some purposes, but not for others. If you need <10 us latency, you still have to jump through hoops to get that on a JVM.
  - haxen 6y ago
    For < 10 µs latency there are hoops in any language, as well as the OS, like thread-pinning, marking CPU cores unusable for OS interrupt handling, all kinds of virtual memory issues, etc. And you can't afford a single network hop or even SSD access.
    - Erlich_Bachman 6y ago
      And yet 90% of time we hear about this latency problems when speaking about a language, it's always about JVM... When it would be equally hard in any other dynamic language... What gives...
      - vips7L 6y ago
        I don't know of any other runtime that gives you the tools to tune your latency and the GC other than the JVM.
        saberience 6y ago
        Err Dotnet? :)
        exdsq 6y ago
        GHC (Haskell)?
      - therealdrag0 6y ago
        Might be a side affect of Java’s dominance/popularity.
    - PaulDavisThe1st 6y ago
      >And you can't afford a single network hop or even SSD access. 10µs ? You can't even afford that many function calls. Seriously, 10µs is not a sensible target for a general purpose OS, maybe not even for a general purpose CPU. Achievable? Perhaps. But sensible? Not really.
- manigandham 6y ago
  That's the trade-off. Predictability and determinism vs development and testing effort. The memory work remains the same, you can do it yourself or let the GC handle it. For 99% of applications, the GCs are good enough and getting better every year, but low-latency still needs predictability and would ideally choose manual memory management. The realities of the job market and IT deployments are different though and that's why we still have JVMs involved with low-latency scenarios because of talent, tooling and productivity.
- karlmdavis 6y ago
  I mean... Rust literally doesn't have a GC, in the usual sense -- certainly nothing resembling most GCs' generations, mark, sweep, etc. approaches. Steve lays it out far better than I could, coining the term "static garbage collection": https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis https://steveklabnik.com/writing/borrow-checking-escape-anal.... And to your "without pretty significant extra work" qualifier: I really don't find that to be true with Rust. The initial learning curve was a bit rough, but certainly far less so than other languages/platforms I've picked up (hello, ML). In the end, I find that it's just a nice, helpful, productive, and ludicrously performant language.
  - lostmyoldone 6y ago
    Rust is really nice, I like it a lot. It's probably also one of the easier languages to write performant code with reasonable latencies in most cases, almost all. However, automatic resource freeing/deallocation can still bite you if you need really reliable low latencies, and if you think the language will handle it all. Since the automatic deallocation in rust, per default, happens at the point where the value goes out of scope, if a resource destruction could block, or perform expensive operations on destruction, it can not be allowed to go out of scope in a latency sensitive thread. Usually not that much of an issue unless you are chasing really low latencies. But things can still happen in rust that catch you off guard. Say a value goes out of scope, and because of reasons, it does so with a destructor doing a logging call, which has accidentally become blocking on a socket send call, which it did because nobody realized that the AWS/GCP/whatever logger adaptor actually didn't perform all IO in a thread with which was only communicated with locklessly, which nobody noticed before because it was only if a buffer was full, which only happened today because .... Not a big deal, it's almost all the same things which mess upp latencies in C++ code. And that's the thing. It's not necessarily easier to get low latency in rust than in C++, but the work required for hitting a quality/performance/latency target in rust is probably still lower than for C++. Unless you are lucky to have a very mature C++ low-latency stack, together with all the utility functionality you need, which seems to be exceedingly rare. Is the work required lower than Java on a custom/tuned JVM? We'll have to wait and see. It probably is, but it's a complex balance between access to utilities, language complexity, and several more parameters which ultimately decide which platform provides the best environment for low latency code, especially if the complexity is non trivial.
dan-robertson 6y ago
The part of GC which causes the most latency issues is compaction rather than merely collection. Using a language like rust won’t help if you have memory fragmentation and indeed allocation tends to be much faster with a GC than with malloc. I think the advantages of rust are more to do with often avoiding heap allocation entirely (and predictably) or value semantics leading to fewer small allocations or the language’s semantics not forcing you to choose between more reliable allocation-heavy immutable-everything code and faster harder-to-think-about mutation-heavy code.
- gwbas1c 6y ago
  IMO, the main advantage of Rust is that it doesn't require an extensive runtime in order to have memory safety. This allows you to write a library like an image processor or embedded database without using C / C++. Otherwise, if you wrote your image processor in C# or Java, it becomes hard to call your library from Python or Node because you have to require the entire VM. Likewise, you can ship an application binary that has no requirements for a runtime. (Your application binary doesn't require a JVM, CLR, Mono, Python, Node, or some other runtime.) I've been through the Rust book twice but I'm just getting to the point of trying to write something in it. The mental model is very different. Coming from C# / Java / Javascript / Objective C; I'm wondering how many hours I need before I can get my head into Rust?
  - shpongled 6y ago
    Easiest way to get started if you're coming from a GC background is to just liberally `.clone()` everything in Rust. Once you're used to move semantics and the syntax, then you can start messing around with borrowing. It definitely has a learning curve, but I find it a breeze to write once you grok the ownership rules.
  - pjmlp 6y ago
    On the other hand it requires jumping through hoops to make borrow checker friendly architecture designs, or fiddle your code base with Rc<> types everywhere. And when their count reaches 0, you have your stop the world, unless you move the destruction into a background thread, thus manually emulating a tracing GC.
nvarsj 6y ago
> Who was it that turned GC off entirely, minimized allocation and just restarted their VMs when they ran out of RAM every couple of hours, was that Netflix? This was common practice in trading firms that got on the Java hype train. Turn off GC and just restart the JVM outside of trading hours. > Either way. It makes me excited for Rust and the languages it'll inspire, all this labor gone away. The JVM gives GC a bad name. There are plenty of GC languages which don’t have the level of pain of hotspot except in extreme cases. For the vast majority of GC languages you never even think about it. Rust / C++ are great when you need full control but it’s not necessary for most things.
- darksaints 6y ago
  I'd say it's more the language design than anything. Heap allocating everything, and then throwing in inheritance 30 levels deep makes for some very poor GC behavior.
  - wokkel 6y ago
    What relationship do you see between inheritance and GC ? Creating An instance of some object 30 levels deep in the inheritance chain does not allocate 29 extra objects.
    - darksaints 6y ago
      It's not always that simple. For example, if you inherit from a class that has private members, you are actually going to allocate both the parent and child. https://stackoverflow.com/questions/27813539/what-happens-in-the-heap-when-class-a-inherits-class-b-in-java https://stackoverflow.com/questions/27813539/what-happens-in... Also, if you don't mark your methods and classes as final, you're gonna be allocating functions everywhere as well.
      - haxen 6y ago
        > you are actually going to allocate both the parent and child. You're going to allocate a single memory block that contains all the state of that object, which of course includes the superclass state. But that has nothing to do with the depth of the class hiererarchy. > you're gonna be allocating functions everywhere as well. If by "function" you mean a "Java method", they are code and not data, and they are not allocated dynamically at all. In any case, this would have nothing to do with them being private or final. These options decide whether there will be an invokevirtual or invokespecial instruction to call them, where invokevirtual has more cost only before getting JIT-optimized.
def_true_false 6y ago
Perhaps Instagram? Or Twitch? Or Discord? https://news.ycombinator.com/item?id=23144380 https://news.ycombinator.com/item?id=23144380
- lostmyoldone 6y ago
  I know I read about some trading firm(s) doing that, but I don't know which, if it was even stated. Whomever they were, they were rotating pre-warmed jvm images with disabled GC, and were reaching quite respectable latency figures. To not have to recycle them quickly, you'll want to not generate too much garbage objects, and that's actually easier than one might think in Java. Especially if you accept to restart the jvm from time to time, as you only need to be mostly statically allocated. Rare error paths can freely use dynamic allocation as long as most of the service doesn't. Nowadays you can also get away without using strings in most places, using only char sequence flyweights over "statically" allocated char sequences. Otherwise strings were a pain, especially API's that really doesn't need a string (ownership) but had string method arguments nevertheless. Used like that, as you would on an embedded platform, theres nothing I know of that actually beats the JVM in raw performance while still being somewhat practical in terms of tooling and hiring. Rust might take that crown, we'll see, but I hope so.
manuelabeledo 6y ago
> Who was it that turned GC off entirely, minimized allocation and just restarted their VMs when they ran out of RAM every couple of hours, was that Netflix? Every single financial firm out there, using Java for sub-microsecond tasks. Really, there is no other way to keep low latencies if you have your GC messing around every few milliseconds. This may surprise some people, but Java is ubiquitous in low latency environments such as trading firms. It offers performance close enough to C++, but the developer pool is way larger. Also, when one needs to comply with extremely low latency requirements, the way to go is always dedicated hardware anyway. If you are interested in the topic, there is this project called OpenHFT that aims to provide high frequency trading tools for Java. Particularly, their Chronicle Queue implementation tries to handle the GC latency issue by storing stuff off heap. Its co-founder, Peter Lawrey, has also delivered a handful of good talks about low latency Java.
- vgatherps 6y ago
  As someone in the industry I always find the claim that Java is ubiquitous in low latency trading fairly strange. The only firm that widely uses Java over C or C++ on the hft fast path is Virtu, and the sense I get from some of my friends there is that they regret the decision (and now it’s mostly used to guide fpgas). There is a TON of Java usage in software that would have been considered HFT maybe 10 years ago, especially at big banks, but the fastest I’ve ever seen somebody make a Java trading program that actually handled the whole tick-to-trade path is 7us, which would be at best ok by C or C++ trading platform standards. I’m sure you could write some dumb trigger program operating on off-heap bytes that had similar performance characteristics to a C hft program, but unless you accomplish this via code generation then you’re just writing C in Java with a runtime actively battling your goals. Edit: Jane street uses ocaml + fpgas but they aren’t really in the HFT business in the same way that say Virtu or Tower is.
  - pjmlp 6y ago
    People say this as if it was somehow depreciative, but I rather appreciate writing safe C in Java, which is something WG14 will never take the effort to improve.
CyberDildonics 6y ago
If something is so latency sensitive and crucial making the java garbage collector such a hindrance, why not start writing parts in C++? It seems to me people end up getting into a situation where they are fighting with the java gc to try to get low latency with huge heaps and constant allocation when it really is not difficult to control memory allocation in modern C++.
OneWay43235392 6y ago
>> It makes me excited for Rust Here we go again ...
nradov 6y ago
You can also use the Real-Time Specification for Java (RTSJ). Essentially it allows you to do all your memory allocation, then shut off the GC. https://jcp.org/en/jsr/detail?id=282 https://jcp.org/en/jsr/detail?id=282

pron 6y ago

In case you missed this post and the previous ones, here's the news: this labour is gone on the JVM. The GCs in JDK 14-15 are good enough pretty much out of the box, and they're getting better very quickly now. OTOH, if you think other languages let you do away with a GC without pretty significant extra work, especially in concurrent systems, well, then you haven't had experience with those languages.

VHRanger 6y ago
In D, the GC is guaranteed to only ever run if you allocate new memory on the heap. This is still a lot of work for a video game (because you never want any latency, the only way to achieve this is with an arena allocator or going full @nogc). But for apps where the latency requirement is bounded, D doesn't make it hard and the language is nice and ergonomic.
- chrisseaton 6y ago
  > In D, the GC is guaranteed to only ever run if you allocate new memory on the heap. Isn't that how most GCs work? Why would you do anything else? To release memory to the OS? That's not really a priority in most runtime systems, and I think wanting to do that is a pretty niche requirement.
  - frant-hartm 6y ago
    I want memory allocation to run in constant time, so I can achieve low latency. That's the point of concurrent GCs - they run in the background to free up memory.
    - chrisseaton 6y ago
      I know they run in the background... but I don't think they run in the background if there's nothing to do, do they?
      - imtringued 6y ago
        A JVM with default settings will happily allocate more memory without attempting to even run a single GC cycle until it hits the configured heap size. Once it hits the limit it will try to run the GC.
        deleted 6y ago
        [deleted]
  - nemothekid 6y ago
    In Java, everything is heap allocated, whereas in D you have a choice.
    - chrisseaton 6y ago
      > everything is heap allocated You can definitely write useful Java applications where nothing gets heap allocated, through using existing objects, and through using scalar-replacement-of-aggregates.
    - wokkel 6y ago
      Not entirely correct: https://shipilev.net/jvm/anatomy-quarks/18-scalar-replacement/ https://shipilev.net/jvm/anatomy-quarks/18-scalar-replacemen... . Hoi cannot force a specific allocation, but the jit does do something akin to stack allocation.
  - readittwice 6y ago
    Releasing memory is definitely useful for mobile devices or browsers. Even in some server use cases it's useful, e.g. when you pay for memory usage. I guess that's why the JVM has this -XX:SoftMaxHeapSize option. I guess it depends whether GCs are always scheduled in an allocation or can be triggered another way. Either way that should be easy to disable. I read somewhere that D doesn't have write barriers, so I would assume they have a hard time implementing more advanced GC features like generational collection or concurrent marking. It's not suprising that the GCs in the JVM achieve much better pause time.
tjoff 6y ago
That statement has been said for pretty much every release of any GC. I'd say the opposite is true. Relying on GC requires significant extra work. Because you always need to think about memory (exception: small script-like applications). The only thing a GC does is that it enable you to not think about it, but the moment you don't you will write bad code and realize it was a disservice all along. And by then it is too late. So in a GC language you need to constantly be aware of when you take something for granted. Which is more work than just doing it manually yourself.
- RhodesianHunter 6y ago
  This is not really true at all though, nor is it how people who design performance intensive systems go about it. generally you design your application the way you normally would, do some testing, and then go back through the hot path and make some adjustments.
- pron 6y ago
  First, a GC allows you to abstract over memory allocations (i.e. hide them). This makes maintaining code over time significantly easier, as allocation becomes a hidden implementation detail. Second, while there have been similar claims made in the past, they always apply for a certain target throughput and latency. GCs are constantly making great strides in that regard. Next year, ZGC will have a <1ms worst-case latency for pretty serious allocation rates, and with an acceptable hit to throughput. As you can see in this post and previous ones, G1 offers great throughput with acceptable latencies.
  - jakewins 6y ago
    This GC you speak of that hides allocations as an implementation detail sounds excellent! Every GC I've worked with - and my day job is orchstrating several thousand JVMs - simply moves the time when you need to think of allocations somewhere else. Either it moves it into PagerDuty, like G1, or it moves it into your GCP bill like Shenandoa and ZGC. The trade-off has always been latency-throughput-footprint, nothing I've seen yet has changed that. The innovation in Rust is realizing you can do all the tracing work at compile time.
twic 6y ago
They're good enough for some purposes, but not for others. If you need <10 us latency, you still have to jump through hoops to get that on a JVM.
- haxen 6y ago
  For < 10 µs latency there are hoops in any language, as well as the OS, like thread-pinning, marking CPU cores unusable for OS interrupt handling, all kinds of virtual memory issues, etc. And you can't afford a single network hop or even SSD access.
  - Erlich_Bachman 6y ago
    And yet 90% of time we hear about this latency problems when speaking about a language, it's always about JVM... When it would be equally hard in any other dynamic language... What gives...
    - vips7L 6y ago
      I don't know of any other runtime that gives you the tools to tune your latency and the GC other than the JVM.
      - saberience 6y ago
        Err Dotnet? :)
      - exdsq 6y ago
        GHC (Haskell)?
    - therealdrag0 6y ago
      Might be a side affect of Java’s dominance/popularity.
  - PaulDavisThe1st 6y ago
    >And you can't afford a single network hop or even SSD access. 10µs ? You can't even afford that many function calls. Seriously, 10µs is not a sensible target for a general purpose OS, maybe not even for a general purpose CPU. Achievable? Perhaps. But sensible? Not really.
manigandham 6y ago
That's the trade-off. Predictability and determinism vs development and testing effort. The memory work remains the same, you can do it yourself or let the GC handle it. For 99% of applications, the GCs are good enough and getting better every year, but low-latency still needs predictability and would ideally choose manual memory management. The realities of the job market and IT deployments are different though and that's why we still have JVMs involved with low-latency scenarios because of talent, tooling and productivity.
karlmdavis 6y ago
I mean... Rust literally doesn't have a GC, in the usual sense -- certainly nothing resembling most GCs' generations, mark, sweep, etc. approaches. Steve lays it out far better than I could, coining the term "static garbage collection": https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis https://steveklabnik.com/writing/borrow-checking-escape-anal.... And to your "without pretty significant extra work" qualifier: I really don't find that to be true with Rust. The initial learning curve was a bit rough, but certainly far less so than other languages/platforms I've picked up (hello, ML). In the end, I find that it's just a nice, helpful, productive, and ludicrously performant language.
- lostmyoldone 6y ago
  Rust is really nice, I like it a lot. It's probably also one of the easier languages to write performant code with reasonable latencies in most cases, almost all. However, automatic resource freeing/deallocation can still bite you if you need really reliable low latencies, and if you think the language will handle it all. Since the automatic deallocation in rust, per default, happens at the point where the value goes out of scope, if a resource destruction could block, or perform expensive operations on destruction, it can not be allowed to go out of scope in a latency sensitive thread. Usually not that much of an issue unless you are chasing really low latencies. But things can still happen in rust that catch you off guard. Say a value goes out of scope, and because of reasons, it does so with a destructor doing a logging call, which has accidentally become blocking on a socket send call, which it did because nobody realized that the AWS/GCP/whatever logger adaptor actually didn't perform all IO in a thread with which was only communicated with locklessly, which nobody noticed before because it was only if a buffer was full, which only happened today because .... Not a big deal, it's almost all the same things which mess upp latencies in C++ code. And that's the thing. It's not necessarily easier to get low latency in rust than in C++, but the work required for hitting a quality/performance/latency target in rust is probably still lower than for C++. Unless you are lucky to have a very mature C++ low-latency stack, together with all the utility functionality you need, which seems to be exceedingly rare. Is the work required lower than Java on a custom/tuned JVM? We'll have to wait and see. It probably is, but it's a complex balance between access to utilities, language complexity, and several more parameters which ultimately decide which platform provides the best environment for low latency code, especially if the complexity is non trivial.

dan-robertson 6y ago

The part of GC which causes the most latency issues is compaction rather than merely collection. Using a language like rust won’t help if you have memory fragmentation and indeed allocation tends to be much faster with a GC than with malloc. I think the advantages of rust are more to do with often avoiding heap allocation entirely (and predictably) or value semantics leading to fewer small allocations or the language’s semantics not forcing you to choose between more reliable allocation-heavy immutable-everything code and faster harder-to-think-about mutation-heavy code.

gwbas1c 6y ago
IMO, the main advantage of Rust is that it doesn't require an extensive runtime in order to have memory safety. This allows you to write a library like an image processor or embedded database without using C / C++. Otherwise, if you wrote your image processor in C# or Java, it becomes hard to call your library from Python or Node because you have to require the entire VM. Likewise, you can ship an application binary that has no requirements for a runtime. (Your application binary doesn't require a JVM, CLR, Mono, Python, Node, or some other runtime.) I've been through the Rust book twice but I'm just getting to the point of trying to write something in it. The mental model is very different. Coming from C# / Java / Javascript / Objective C; I'm wondering how many hours I need before I can get my head into Rust?
- shpongled 6y ago
  Easiest way to get started if you're coming from a GC background is to just liberally `.clone()` everything in Rust. Once you're used to move semantics and the syntax, then you can start messing around with borrowing. It definitely has a learning curve, but I find it a breeze to write once you grok the ownership rules.
- pjmlp 6y ago
  On the other hand it requires jumping through hoops to make borrow checker friendly architecture designs, or fiddle your code base with Rc<> types everywhere. And when their count reaches 0, you have your stop the world, unless you move the destruction into a background thread, thus manually emulating a tracing GC.

nvarsj 6y ago

> Who was it that turned GC off entirely, minimized allocation and just restarted their VMs when they ran out of RAM every couple of hours, was that Netflix? This was common practice in trading firms that got on the Java hype train. Turn off GC and just restart the JVM outside of trading hours. > Either way. It makes me excited for Rust and the languages it'll inspire, all this labor gone away. The JVM gives GC a bad name. There are plenty of GC languages which don’t have the level of pain of hotspot except in extreme cases. For the vast majority of GC languages you never even think about it. Rust / C++ are great when you need full control but it’s not necessary for most things.

darksaints 6y ago
I'd say it's more the language design than anything. Heap allocating everything, and then throwing in inheritance 30 levels deep makes for some very poor GC behavior.
- wokkel 6y ago
  What relationship do you see between inheritance and GC ? Creating An instance of some object 30 levels deep in the inheritance chain does not allocate 29 extra objects.
  - darksaints 6y ago
    It's not always that simple. For example, if you inherit from a class that has private members, you are actually going to allocate both the parent and child. https://stackoverflow.com/questions/27813539/what-happens-in-the-heap-when-class-a-inherits-class-b-in-java https://stackoverflow.com/questions/27813539/what-happens-in... Also, if you don't mark your methods and classes as final, you're gonna be allocating functions everywhere as well.
    - haxen 6y ago
      > you are actually going to allocate both the parent and child. You're going to allocate a single memory block that contains all the state of that object, which of course includes the superclass state. But that has nothing to do with the depth of the class hiererarchy. > you're gonna be allocating functions everywhere as well. If by "function" you mean a "Java method", they are code and not data, and they are not allocated dynamically at all. In any case, this would have nothing to do with them being private or final. These options decide whether there will be an invokevirtual or invokespecial instruction to call them, where invokevirtual has more cost only before getting JIT-optimized.

def_true_false 6y ago

Perhaps Instagram? Or Twitch? Or Discord? https://news.ycombinator.com/item?id=23144380 https://news.ycombinator.com/item?id=23144380

lostmyoldone 6y ago
I know I read about some trading firm(s) doing that, but I don't know which, if it was even stated. Whomever they were, they were rotating pre-warmed jvm images with disabled GC, and were reaching quite respectable latency figures. To not have to recycle them quickly, you'll want to not generate too much garbage objects, and that's actually easier than one might think in Java. Especially if you accept to restart the jvm from time to time, as you only need to be mostly statically allocated. Rare error paths can freely use dynamic allocation as long as most of the service doesn't. Nowadays you can also get away without using strings in most places, using only char sequence flyweights over "statically" allocated char sequences. Otherwise strings were a pain, especially API's that really doesn't need a string (ownership) but had string method arguments nevertheless. Used like that, as you would on an embedded platform, theres nothing I know of that actually beats the JVM in raw performance while still being somewhat practical in terms of tooling and hiring. Rust might take that crown, we'll see, but I hope so.

manuelabeledo 6y ago

> Who was it that turned GC off entirely, minimized allocation and just restarted their VMs when they ran out of RAM every couple of hours, was that Netflix? Every single financial firm out there, using Java for sub-microsecond tasks. Really, there is no other way to keep low latencies if you have your GC messing around every few milliseconds. This may surprise some people, but Java is ubiquitous in low latency environments such as trading firms. It offers performance close enough to C++, but the developer pool is way larger. Also, when one needs to comply with extremely low latency requirements, the way to go is always dedicated hardware anyway. If you are interested in the topic, there is this project called OpenHFT that aims to provide high frequency trading tools for Java. Particularly, their Chronicle Queue implementation tries to handle the GC latency issue by storing stuff off heap. Its co-founder, Peter Lawrey, has also delivered a handful of good talks about low latency Java.

vgatherps 6y ago
As someone in the industry I always find the claim that Java is ubiquitous in low latency trading fairly strange. The only firm that widely uses Java over C or C++ on the hft fast path is Virtu, and the sense I get from some of my friends there is that they regret the decision (and now it’s mostly used to guide fpgas). There is a TON of Java usage in software that would have been considered HFT maybe 10 years ago, especially at big banks, but the fastest I’ve ever seen somebody make a Java trading program that actually handled the whole tick-to-trade path is 7us, which would be at best ok by C or C++ trading platform standards. I’m sure you could write some dumb trigger program operating on off-heap bytes that had similar performance characteristics to a C hft program, but unless you accomplish this via code generation then you’re just writing C in Java with a runtime actively battling your goals. Edit: Jane street uses ocaml + fpgas but they aren’t really in the HFT business in the same way that say Virtu or Tower is.
- pjmlp 6y ago
  People say this as if it was somehow depreciative, but I rather appreciate writing safe C in Java, which is something WG14 will never take the effort to improve.

CyberDildonics 6y ago

If something is so latency sensitive and crucial making the java garbage collector such a hindrance, why not start writing parts in C++? It seems to me people end up getting into a situation where they are fighting with the java gc to try to get low latency with huge heaps and constant allocation when it really is not difficult to control memory allocation in modern C++.

OneWay43235392 6y ago

>> It makes me excited for Rust Here we go again ...

nradov 6y ago

You can also use the Real-Time Specification for Java (RTSJ). Essentially it allows you to do all your memory allocation, then shut off the GC. https://jcp.org/en/jsr/detail?id=282 https://jcp.org/en/jsr/detail?id=282