3 ms·

I think the real issue is that you probably should worry about this overhead only in the context of really tight loops where you are basically wasting a lot of

I think the real issue is that you probably should worry about this overhead only in the context of really tight loops where you are basically wasting a lot of memory allocation and garbage collection for no good reason. But otherwise, Java is actually pretty good dealing with short lived small objects. The garbage collector is pretty good for that. Normal usage of this stuff is not going to cause any more issues than trigger the OCD of people that obsess about this stuff. And in the rare case that you do have an issue, you can do something more optimal indeed.

pkolaczk 4y ago
If you're writing a database engine, and you accidentally wrap each table cell in a Java object, you'll very quickly hit a GC performance wall, particularly when using a low pause GC. So we try to write Java without objects. Guess how easy that is. ;)
- marginalia_nu 4y ago
  Yeah I deal with that a lot with my search engine index too. Honestly it's not that bad once you get used to it. You can get away with referencing the data through (mutable and reusable) pointer objects that reference memory mapped areas yet provide a relatively comfortable higher level interface. This gets rid of object churn while keeping a relatively sane interface.
  - CHY872 4y ago
    Yes, this is the way. Surprising how high level you can make this work - I’ve seen Collection implementations and all sorts.
  - tsss 4y ago
    Can you give an example how to do that? I've only seen IO and parsing libraries that copy around byte arrays, but you can't cast byte arrays to arbitrary objects, which means you usually have to manipulate the bytes directly (unless you're lucky to work with only strings). At that point I imagine it would be far easier to just use C++ or Rust.
    - marginalia_nu 4y ago
      Yeah this sort of work isn't Java's strong suite. A lot of it's sort of like programming old-school C with oven mitts. It'll get a bit better with the Foreign Memory API which is in the JEP pipe. But a very bare bones example might look something like class MdrLayout { static final int FOO_OFFSET = 0; static final int BAR_OFFSET = 4; static final int BAZ_OFFSET = 12; static final int ENTRY_SIZE = 13; } class MyDataRecord { int idx = Integer.MIN_VALUE; ByteBuffer buffer = null; MyDataRecord() { } void movePointer(ByteBuffer buffer, int idx) { this.buffer = buffer; this.idx = idx; } int foo() { return buffer.getInt(ENTRY_SIZE * idx + FOO_OFFSET); } long bar() { return buffer.getLong(ENTRY_SIZE * idx + BAR_OFFSET); } byte baz() { return buffer.get(ENTRY_SIZE * idx + BAZ_OFFSET); } // may have operators like reset(), next() or skip() as well } or class MyDataRecord2 { final ByteBuffer buffer; MyDataRecord2(ByteBuffer buffer) { this.buffer = buffer; } int foo(int idx) { return buffer.getInt(ENTRY_SIZE * idx + FOO_OFFSET); } long bar(int idx) { return buffer.getLong(ENTRY_SIZE * idx + BAR_OFFSET); } byte baz(int idx) { return buffer.get(ENTRY_SIZE * idx + BAZ_OFFSET); } }
- jillesvangurp 4y ago
  If you do stuff like that, use a profiler and identify and fix your real performance bottlenecks. As opposed to applying premature optimization blindly. Same with GC tuning. This has gotten easier over the years. But there are still lots of tradeoffs here. There are plenty of fast performing databases and other middleware written in Java. The JVM is a popular platform for that kind of thing for a good reason. Writing good software of course is a bit of a skill. Benchmarks like this are kind of pointless. Doing an expensive thing in a loop is slow. Well duh. Don't do that.
pdpi 4y ago
GC performance isn't the only thing you need to care about here. Iterating over an ArrayList<T> in Java already makes you chase pointers all over the place due to Java's lack of value types, which wrecks cache locality. ArrayList<Optional<T>> just makes you chase two pointers for each object instead.
- CHY872 4y ago
  Java uses thread local allocation buffers such that objects allocated one after another are typically contiguous in RAM. Most modern Java gcs are also compacting, meaning that the heap ends up approximately breadth first ordered after GC. What this means is that in practice, pointer chasing is less of an issue than you’d expect. Even a linked list will end up with decent cache locality. Obviously this won’t always work, but it generally works a lot better than the same structure in a systems language. https://shipilev.net/jvm/anatomy-quarks/11-moving-gc-locality/ https://shipilev.net/jvm/anatomy-quarks/11-moving-gc-localit...
- vips7L 4y ago
  List<Optional<T>> is pure insanity fwiw. Filter out your empties before making the list.
  - bobbylarrybobby 4y ago
    What if you have an in-memory DataFrame? Not hard to end up with something that looks like List<Optional<T>> (say, NullableSeries<T>) as one of the columns.
  - pdpi 4y ago
    Sure, but Optional<T> is what we were already discussing, and List<T> is the collection type that most obviously requires no indirection. Feel free to replace Optional<T> with any other sort of sum type and the point still stands — you always have one extra object between the list and the "actual value".
    - yakubin 4y ago

pkolaczk 4y ago

If you're writing a database engine, and you accidentally wrap each table cell in a Java object, you'll very quickly hit a GC performance wall, particularly when using a low pause GC. So we try to write Java without objects. Guess how easy that is. ;)

marginalia_nu 4y ago
Yeah I deal with that a lot with my search engine index too. Honestly it's not that bad once you get used to it. You can get away with referencing the data through (mutable and reusable) pointer objects that reference memory mapped areas yet provide a relatively comfortable higher level interface. This gets rid of object churn while keeping a relatively sane interface.
- CHY872 4y ago
  Yes, this is the way. Surprising how high level you can make this work - I’ve seen Collection implementations and all sorts.
- tsss 4y ago
  Can you give an example how to do that? I've only seen IO and parsing libraries that copy around byte arrays, but you can't cast byte arrays to arbitrary objects, which means you usually have to manipulate the bytes directly (unless you're lucky to work with only strings). At that point I imagine it would be far easier to just use C++ or Rust.
  - marginalia_nu 4y ago
    Yeah this sort of work isn't Java's strong suite. A lot of it's sort of like programming old-school C with oven mitts. It'll get a bit better with the Foreign Memory API which is in the JEP pipe. But a very bare bones example might look something like class MdrLayout { static final int FOO_OFFSET = 0; static final int BAR_OFFSET = 4; static final int BAZ_OFFSET = 12; static final int ENTRY_SIZE = 13; } class MyDataRecord { int idx = Integer.MIN_VALUE; ByteBuffer buffer = null; MyDataRecord() { } void movePointer(ByteBuffer buffer, int idx) { this.buffer = buffer; this.idx = idx; } int foo() { return buffer.getInt(ENTRY_SIZE * idx + FOO_OFFSET); } long bar() { return buffer.getLong(ENTRY_SIZE * idx + BAR_OFFSET); } byte baz() { return buffer.get(ENTRY_SIZE * idx + BAZ_OFFSET); } // may have operators like reset(), next() or skip() as well } or class MyDataRecord2 { final ByteBuffer buffer; MyDataRecord2(ByteBuffer buffer) { this.buffer = buffer; } int foo(int idx) { return buffer.getInt(ENTRY_SIZE * idx + FOO_OFFSET); } long bar(int idx) { return buffer.getLong(ENTRY_SIZE * idx + BAR_OFFSET); } byte baz(int idx) { return buffer.get(ENTRY_SIZE * idx + BAZ_OFFSET); } }
jillesvangurp 4y ago
If you do stuff like that, use a profiler and identify and fix your real performance bottlenecks. As opposed to applying premature optimization blindly. Same with GC tuning. This has gotten easier over the years. But there are still lots of tradeoffs here. There are plenty of fast performing databases and other middleware written in Java. The JVM is a popular platform for that kind of thing for a good reason. Writing good software of course is a bit of a skill. Benchmarks like this are kind of pointless. Doing an expensive thing in a loop is slow. Well duh. Don't do that.

pdpi 4y ago

GC performance isn't the only thing you need to care about here. Iterating over an ArrayList<T> in Java already makes you chase pointers all over the place due to Java's lack of value types, which wrecks cache locality. ArrayList<Optional<T>> just makes you chase two pointers for each object instead.

CHY872 4y ago
Java uses thread local allocation buffers such that objects allocated one after another are typically contiguous in RAM. Most modern Java gcs are also compacting, meaning that the heap ends up approximately breadth first ordered after GC. What this means is that in practice, pointer chasing is less of an issue than you’d expect. Even a linked list will end up with decent cache locality. Obviously this won’t always work, but it generally works a lot better than the same structure in a systems language. https://shipilev.net/jvm/anatomy-quarks/11-moving-gc-locality/ https://shipilev.net/jvm/anatomy-quarks/11-moving-gc-localit...
vips7L 4y ago
List<Optional<T>> is pure insanity fwiw. Filter out your empties before making the list.
- bobbylarrybobby 4y ago
  What if you have an in-memory DataFrame? Not hard to end up with something that looks like List<Optional<T>> (say, NullableSeries<T>) as one of the columns.
- pdpi 4y ago
  Sure, but Optional<T> is what we were already discussing, and List<T> is the collection type that most obviously requires no indirection. Feel free to replace Optional<T> with any other sort of sum type and the point still stands — you always have one extra object between the list and the "actual value".
  - yakubin 4y ago