4 ms·

I worked on Java code at AWS for a few years and nobody tried to optimize allocations. Then I changed jobs and started working on a Java MPP database and my fir

I worked on Java code at AWS for a few years and nobody tried to optimize allocations. Then I changed jobs and started working on a Java MPP database and my first code review was brutal. You were expected to avoid allocations as much as possible (mostly by using the SoA pattern everywhere). At that scale no GC could save you from excessive allocations.

motoboi 3mo ago
I believe the whole string vs stringbuffer that later was made redundant by compiler contributed to that vision. People started dismissing allocation discipline as a thing from the past because "that thing was solved a lot ago and the compiler now is smart enough". Well, for string, yes, but not for arbitrary objects.
- cogman10 3mo ago
  The most surprising allocation pressure I constantly run into is primitive boxing. The JVM does heroics to try and avoid it as much as possible, but when you end up with some primitive boxing in a hotspot the amount of GC pressure that creates can be unreal.
  - drob518 3mo ago
    Yep, and sometimes just a small code change can flip on boxing.
    - cogman10 3mo ago
      One of my least proud (most proud?) hacks when working with very large data sets is something like this Map<Integer, Integer> intCache = new HashMap<>(); while (loading) { Integer feild1 = intCache.computeIfAbsent(getField1(), (i)->i); } This is a terrible thing that shouldn't be as useful as it is to us... but it is really useful. We have a bunch of objects that can optionally have Integer values (hence a null is valid) but those int values are frequently the same. This saves a bunch of memory and ultimately GC pressure as a result. Valhalla can't come soon enough for us.
      - drob518 3mo ago
        I do something similar for Java Time local dates. Financial data in particular has lots of redundant date info and benefits from being memoized. Converting to epoch millis also works.
        actionfromafar 3mo ago
        If I squint, is this a special kind of heap compression?
        cogman10 3mo ago
        Not really heap compression or special, it's just reusing a reference to an object already allocated on the heap. Right now, if I do this LocalDate a = LocalDate.of(2020, 1, 1); LocalDate b = LocalDate.of(2020, 1, 1); A and B reference 2 different object allocations on the heap even though they are the same date. a != b. In Java, that can be pretty expensive even for an object as light as a LocalDate. By running the cache and doing var cache = new HashMap<LocalDate, LocalDate>(); LocalDate a = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i); LocalDate b = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i); Now you have the situation where `a == b` and you immediately end up dropping the object allocation for b on the next GC. The technique works best when you have a lot of repeated objects which are immutable. It is also only really needed because Valhalla isn't here. Once "value types" become a thing, then the representation for `LocalDate` inside the JVM can become just the fields and not a reference. The JVM is also free to do the sort of de-duplication optimization all on it's own for larger objects.
        actionfromafar 3mo ago
        Now I understand what Valhalla is (new JVM vesion!), I thought it was some kind of dark joke at first. Thanks for the explanation!
        drob518 3mo ago
        Right. And this is safe now since Java Time classes are immutable. That’s key. Before, you had to worry about someone modifying the object.
  - spullara 3mo ago
    can't wait for Project Valhalla, going into preview shortly
  - hinkley 3mo ago
    The especially nasty thing about that boxing pressure is that when I still paid attention to the JVM, they had some sort of cache for boxing of numbers under about 128, which meant that if you did tests with toy inputs, identity checks would pass in code that needed an equality check instead, and benchmarks built the same way would see no GC pressure in those call trees but see quite a lot of pressure in production. I remember being really mad, almost to the point of a sense of betrayal, when I learned that. It's kind of a DieselGate but with more plausible deniability. You can't say it was deliberate, but if you already had trust concerns with that person it would move the dial farther into the red.
    - throawayonthe 3mo ago
      isn't that kinda how python preallocates integers -5 to 256? https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong
      - hinkley 3mo ago
        Sounds like it. I wasn't familiar with how python does this but it looks like the identity operator is 'is' so while someone could definitely screw that up, it would stick out more in a code review than == versus ===
- hinkley 3mo ago
  I worked on projects where we indeed needed the manual stringbuffer refactor on hot paths in the code. It's not difficult work but boy is it tedious. Spent a lot of time with headphones on during those days. Only to work on another project a few years later where I paid for my sins by reversing the same change on someone else's codebase. There's a version of Java, I can't recall which but I want to say 4? Maybe 3? Where someone rewrote parts of the Swing backend as native code to speed it up. Then Hotspot got good enough in the next version that the generated code was faster than the native code + FFI overhead. FFI was pretty high at the time. So they reverted the native code migration and went back to the old code.
zaphirplane 3mo ago
How does SoA help ?

motoboi 3mo ago

I believe the whole string vs stringbuffer that later was made redundant by compiler contributed to that vision. People started dismissing allocation discipline as a thing from the past because "that thing was solved a lot ago and the compiler now is smart enough". Well, for string, yes, but not for arbitrary objects.

cogman10 3mo ago
The most surprising allocation pressure I constantly run into is primitive boxing. The JVM does heroics to try and avoid it as much as possible, but when you end up with some primitive boxing in a hotspot the amount of GC pressure that creates can be unreal.
- drob518 3mo ago
  Yep, and sometimes just a small code change can flip on boxing.
  - cogman10 3mo ago
    One of my least proud (most proud?) hacks when working with very large data sets is something like this Map<Integer, Integer> intCache = new HashMap<>(); while (loading) { Integer feild1 = intCache.computeIfAbsent(getField1(), (i)->i); } This is a terrible thing that shouldn't be as useful as it is to us... but it is really useful. We have a bunch of objects that can optionally have Integer values (hence a null is valid) but those int values are frequently the same. This saves a bunch of memory and ultimately GC pressure as a result. Valhalla can't come soon enough for us.
    - drob518 3mo ago
      I do something similar for Java Time local dates. Financial data in particular has lots of redundant date info and benefits from being memoized. Converting to epoch millis also works.
      - actionfromafar 3mo ago
        If I squint, is this a special kind of heap compression?
        cogman10 3mo ago
        Not really heap compression or special, it's just reusing a reference to an object already allocated on the heap. Right now, if I do this LocalDate a = LocalDate.of(2020, 1, 1); LocalDate b = LocalDate.of(2020, 1, 1); A and B reference 2 different object allocations on the heap even though they are the same date. a != b. In Java, that can be pretty expensive even for an object as light as a LocalDate. By running the cache and doing var cache = new HashMap<LocalDate, LocalDate>(); LocalDate a = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i); LocalDate b = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i); Now you have the situation where `a == b` and you immediately end up dropping the object allocation for b on the next GC. The technique works best when you have a lot of repeated objects which are immutable. It is also only really needed because Valhalla isn't here. Once "value types" become a thing, then the representation for `LocalDate` inside the JVM can become just the fields and not a reference. The JVM is also free to do the sort of de-duplication optimization all on it's own for larger objects.
        actionfromafar 3mo ago
        Now I understand what Valhalla is (new JVM vesion!), I thought it was some kind of dark joke at first. Thanks for the explanation!
        drob518 3mo ago
        Right. And this is safe now since Java Time classes are immutable. That’s key. Before, you had to worry about someone modifying the object.
- spullara 3mo ago
  can't wait for Project Valhalla, going into preview shortly
- hinkley 3mo ago
  The especially nasty thing about that boxing pressure is that when I still paid attention to the JVM, they had some sort of cache for boxing of numbers under about 128, which meant that if you did tests with toy inputs, identity checks would pass in code that needed an equality check instead, and benchmarks built the same way would see no GC pressure in those call trees but see quite a lot of pressure in production. I remember being really mad, almost to the point of a sense of betrayal, when I learned that. It's kind of a DieselGate but with more plausible deniability. You can't say it was deliberate, but if you already had trust concerns with that person it would move the dial farther into the red.
  - throawayonthe 3mo ago
    isn't that kinda how python preallocates integers -5 to 256? https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong
    - hinkley 3mo ago
      Sounds like it. I wasn't familiar with how python does this but it looks like the identity operator is 'is' so while someone could definitely screw that up, it would stick out more in a code review than == versus ===
hinkley 3mo ago
I worked on projects where we indeed needed the manual stringbuffer refactor on hot paths in the code. It's not difficult work but boy is it tedious. Spent a lot of time with headphones on during those days. Only to work on another project a few years later where I paid for my sins by reversing the same change on someone else's codebase. There's a version of Java, I can't recall which but I want to say 4? Maybe 3? Where someone rewrote parts of the Swing backend as native code to speed it up. Then Hotspot got good enough in the next version that the generated code was faster than the native code + FFI overhead. FFI was pretty high at the time. So they reverted the native code migration and went back to the old code.

zaphirplane 3mo ago

How does SoA help ?