4 ms·

Well it depends a bit, in my low latency application where we cant afford GC pauses, we tolerate them from 5am to 5:30 am. We do everything preallocated and if

by xwolfi 5y ago

Well it depends a bit, in my low latency application where we cant afford GC pauses, we tolerate them from 5am to 5:30 am. We do everything preallocated and if we must tolerate new heap allocation we simply never release (so we have a lot of object pools). I felt it was hard and exotic at first but now I feel it's quite logical: never free, and have a proper monitoring of your size per unit of information to size the heap right and you're done. But then it also depends why you dont want GC, us it's because clients cannot tolerate a pause (we had a socket manager creating and releasing garbage that triggered one 50ms GC pause during the last 3 seconds of trading of a particularly crazy trading day and the client threatened to leave), but often they re tolerable if you just stay reasonable with your garbage. The problem is not the collection in non critical applications, it's the garbage generation getting completely out of hand. In a C# application my team had to micro optimize because of covid-related explosive trading volume, we simply bought gigantic amount of RAM for our 200 users, only allowed GC if CPU was in low usage and run with 80GB memory usage at the end of the day for an order grid that could run with 5GB collected. That s another way: I wish Java 8 had an easier way to simply deactivate GC for us to run with hardware money.

cmrdporcupine 5y ago
One problem is that GC'd languages also tend to have programming practices that encourage the generation of garbage, while languages with manual stack/heap allocation make you think about it a bit more. The sheer number of String and other lightweight (but existent) instances in a running Java program is astounding, because the language simply doesn't encourage you to be parsimonious with creating them. For most people's applications this is totally fine. But, yeah, you can end up with a scenario where GC pauses are just totally unacceptable, or where you need absolutely predictable behaviour and then you might resort to radical things like what you're describing here and it starts to feel like you've just chosen the wrong tool for the job. I say this as a person who wrote an RTB ad server in Java, many moons ago... and never would again.
- xwolfi 5y ago
  I d simply take a noop GC in the most recent jvm (I cant yet in my conservative bank), and make people suffer through it in prod until everything is constrained correctly, but you're right, String are banned where I work, and what a pain it has to be to do everything via char buffer but it's taking a dev like maybe 10 minutes per "String" handling instead of 5 second during code writing (to massage the buffer all the way from the input signal to the output destination, be it a log file, a storage structure, a comparator or a map key), so it's a giganormous increase of dev time relatively, but in absolute time it's quite trivial over the last 20 years. Yeah we wasted time per feature on String handling but... like maybe a few hours per feature per dev and we generate billions out of the trading backend so... Maybe Java had the wrong assumption and really should just have made C-string optional first class citizens or something. One thing is sure though, our C++ colleagues in the algo team are so slow to deliver, so prone to crazy bugs, so wild and wizardy, so unable to onboard new joiners, they re getting replaced this year by... a brand new Java algo engine... So not sure what to think.
  - FpUser 5y ago
    Initially Java has been created with totally different application scope in mind comparatively to what it is now. I know it may look like a mess but maybe they should've added new optimized string type in addition to already available. As for C++ - I do just that. All my servers are C++ and while I will not claim "bug free" I do not really remember when was the last time I've had memory related bug (alloc / boundary / use after free / etc). Modern C++ along with sanitizers helps greatly in this department. All fixed bugs were logical. I did my share of Java development big while ago and got sick of trying to make it performant enough. In order for me to achieve the goal I've spent way more time caring about memory / resource related issues in Java than I would do in "dangerous languages. Granted it was high performance servers. Doing different type of task in Java could've cost way less aggravation.
  - ncmncm 5y ago
    I know exactly what to think: salaries offered for your C++ colleagues are not competitive, so you are collecting competitors' dregs. It can be very hard to come back from that, maybe impossible. In other shops, the notion of conducting trading in Java, GC'd or no, would get you laughed from the room. But Java coders certainly are cheaper.
- titzer 5y ago
  > The sheer number of String and other lightweight (but existent) instances in a running Java program is astounding, because the language simply doesn't encourage you to be parsimonious with creating them. I agree with this. Java promotes a profligate programming style where allocation (and reclamation) are considered free. (They aren't). This is baked into the JDK deeply. E.g. with standard APIs it is not even possible to parse an integer out of a string without allocating memory. Those are language and library design choices that have a huge confounding effect on trying to measure GC. For example, I wrote the Avrora AVR emulator and sensor network simulator, which is one of the DaCapo benchmarks. It's a CPU emulator for very small devices and uses almost no standard JDK stuff in its core work. As such it allocates very little and has a pretty small heap. It often shows up weird (sometimes obnoxiously small or uninteresting) in papers that study memory management because it really isn't very Java-like in its behavior.
  - _old_dude_ 5y ago
    > with standard APIs it is not even possible to parse an integer out of a string without allocating memory Not true anymore :) https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Integer.html#parseInt(java.lang.CharSequence,int,int,int) https://docs.oracle.com/en/java/javase/17/docs/api/java.base...
    - titzer 5y ago
      Wonders never cease :)
- initplus 5y ago
  I'm curious, is there a reason why GC'd languages don't use some form of escape analysis so that only allocations that can possibly live beyond the current scope need to be heap allocated?
  - kragen 5y ago
    Lots of GCed languages do use escape analysis. That's what escape analysis was invented for!
chihuahua 5y ago
C# has the option of background/concurrent GC. Did that not work for your application?
- xwolfi 5y ago
  It was less cpu intensive to buy RAM, so we did that first. We need all 20 cores for all the other stuff the desktop user uses sadly, and we can only give them 3 computers with 20 cores each. I tried lobbying for threadrippers but... not easy to onboard a new CPU in a giga corporation :(

by xwolfi 5y ago

cmrdporcupine 5y ago

One problem is that GC'd languages also tend to have programming practices that encourage the generation of garbage, while languages with manual stack/heap allocation make you think about it a bit more. The sheer number of String and other lightweight (but existent) instances in a running Java program is astounding, because the language simply doesn't encourage you to be parsimonious with creating them. For most people's applications this is totally fine. But, yeah, you can end up with a scenario where GC pauses are just totally unacceptable, or where you need absolutely predictable behaviour and then you might resort to radical things like what you're describing here and it starts to feel like you've just chosen the wrong tool for the job. I say this as a person who wrote an RTB ad server in Java, many moons ago... and never would again.

xwolfi 5y ago
I d simply take a noop GC in the most recent jvm (I cant yet in my conservative bank), and make people suffer through it in prod until everything is constrained correctly, but you're right, String are banned where I work, and what a pain it has to be to do everything via char buffer but it's taking a dev like maybe 10 minutes per "String" handling instead of 5 second during code writing (to massage the buffer all the way from the input signal to the output destination, be it a log file, a storage structure, a comparator or a map key), so it's a giganormous increase of dev time relatively, but in absolute time it's quite trivial over the last 20 years. Yeah we wasted time per feature on String handling but... like maybe a few hours per feature per dev and we generate billions out of the trading backend so... Maybe Java had the wrong assumption and really should just have made C-string optional first class citizens or something. One thing is sure though, our C++ colleagues in the algo team are so slow to deliver, so prone to crazy bugs, so wild and wizardy, so unable to onboard new joiners, they re getting replaced this year by... a brand new Java algo engine... So not sure what to think.
- FpUser 5y ago
  Initially Java has been created with totally different application scope in mind comparatively to what it is now. I know it may look like a mess but maybe they should've added new optimized string type in addition to already available. As for C++ - I do just that. All my servers are C++ and while I will not claim "bug free" I do not really remember when was the last time I've had memory related bug (alloc / boundary / use after free / etc). Modern C++ along with sanitizers helps greatly in this department. All fixed bugs were logical. I did my share of Java development big while ago and got sick of trying to make it performant enough. In order for me to achieve the goal I've spent way more time caring about memory / resource related issues in Java than I would do in "dangerous languages. Granted it was high performance servers. Doing different type of task in Java could've cost way less aggravation.
- ncmncm 5y ago
  I know exactly what to think: salaries offered for your C++ colleagues are not competitive, so you are collecting competitors' dregs. It can be very hard to come back from that, maybe impossible. In other shops, the notion of conducting trading in Java, GC'd or no, would get you laughed from the room. But Java coders certainly are cheaper.
titzer 5y ago
> The sheer number of String and other lightweight (but existent) instances in a running Java program is astounding, because the language simply doesn't encourage you to be parsimonious with creating them. I agree with this. Java promotes a profligate programming style where allocation (and reclamation) are considered free. (They aren't). This is baked into the JDK deeply. E.g. with standard APIs it is not even possible to parse an integer out of a string without allocating memory. Those are language and library design choices that have a huge confounding effect on trying to measure GC. For example, I wrote the Avrora AVR emulator and sensor network simulator, which is one of the DaCapo benchmarks. It's a CPU emulator for very small devices and uses almost no standard JDK stuff in its core work. As such it allocates very little and has a pretty small heap. It often shows up weird (sometimes obnoxiously small or uninteresting) in papers that study memory management because it really isn't very Java-like in its behavior.
- _old_dude_ 5y ago
  > with standard APIs it is not even possible to parse an integer out of a string without allocating memory Not true anymore :) https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Integer.html#parseInt(java.lang.CharSequence,int,int,int) https://docs.oracle.com/en/java/javase/17/docs/api/java.base...
  - titzer 5y ago
    Wonders never cease :)
initplus 5y ago
I'm curious, is there a reason why GC'd languages don't use some form of escape analysis so that only allocations that can possibly live beyond the current scope need to be heap allocated?
- kragen 5y ago
  Lots of GCed languages do use escape analysis. That's what escape analysis was invented for!

chihuahua 5y ago

C# has the option of background/concurrent GC. Did that not work for your application?

xwolfi 5y ago
It was less cpu intensive to buy RAM, so we did that first. We need all 20 cores for all the other stuff the desktop user uses sadly, and we can only give them 3 computers with 20 cores each. I tried lobbying for threadrippers but... not easy to onboard a new CPU in a giga corporation :(