3 ms·

>Using offheap memory allocation in Java is generally a bad idea. Memory mapped files and direct buffers do work well. I'd never consider it a bad idea; techni

by xxs 3y ago

>Using offheap memory allocation in Java is generally a bad idea. Memory mapped files and direct buffers do work well. I'd never consider it a bad idea; technically both would be freed if there are no strong references to them (not much different than allocating massive arrays). Think of it like that the jdk standard library has to use native memory for any IO or even a trivial zip compression. ByteBuffers were introduced in 1.4 (around 20y back) to allow operations outside the managed system, the tools are available to anyone, e.g. implementing zstd with direct buffers is not hard.

marginalia_nu 3y ago
Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff with in the marginalia search index code. Most operations run about 50-100% faster, even discounting the rigmarole needed to deal with mmapping >2 GB files. Haven't dug too deep into why this is so I'm not entirely sure why, may have something to do with being able to declare non-shared memory ranges in the arena allocator, saves a bunch of synchronization maybe? I can't wait for this stuff to leave experimental. It's such a quality of life boon for dealing with off-heap memory, having explicit lifecycle control is.
- xxs 3y ago
  > Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I'd disagree. Heap based bytebuffers - I don't consider them interesting, they are just byte arrays. The direct ones suffer from unmapping (munmap on linux) - it's a slow process that has to flush the TLB. So the only sane way to use them allocate once/reuse. Anything else I'd consider a programmer error. mmap on large files indeed does suck (a bit) w/ ByteBuffers as it'd require an array of them. Personally I have been using direct buffer since 1.4, so I guess I am also quite used to them as well.
  - marginalia_nu 3y ago
    Right, I'm not talking about mapping or allocation costs or anything surrounding the access, just the impact of addressing off heap bytebuffers them seems quite a bit slower than what you get with the new foreign memory API... for whatever reason.
    - xxs 3y ago
      You mean using directBuffers in java - that could be, if the JIT fails to remove the bound checks. It should never be an issue in native code, of course. But yes - it's possible that it happens with Java code. In that case you'd need print assembly and looking at the generated code. Some libraries have 'switched' to straight unsafe use (which has no bound checks, of course), so there is that.
    - bitcharmer 3y ago
      This makes zero sense. I'll believe it when someone proves that (warmed up) direct byte buffer access is slower than Unsafe.
      - marginalia_nu 3y ago
        I'm not comparing against Unsafe, this is JDK21. No unsafe anymore.
- bitcharmer 3y ago
  > I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff That's just a sign of bad benchmark. Off-heap BBs and foreign memory are both native (non-JVM) heap. They are the same thing

by xxs 3y ago

marginalia_nu 3y ago

Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff with in the marginalia search index code. Most operations run about 50-100% faster, even discounting the rigmarole needed to deal with mmapping >2 GB files. Haven't dug too deep into why this is so I'm not entirely sure why, may have something to do with being able to declare non-shared memory ranges in the arena allocator, saves a bunch of synchronization maybe? I can't wait for this stuff to leave experimental. It's such a quality of life boon for dealing with off-heap memory, having explicit lifecycle control is.

xxs 3y ago
> Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I'd disagree. Heap based bytebuffers - I don't consider them interesting, they are just byte arrays. The direct ones suffer from unmapping (munmap on linux) - it's a slow process that has to flush the TLB. So the only sane way to use them allocate once/reuse. Anything else I'd consider a programmer error. mmap on large files indeed does suck (a bit) w/ ByteBuffers as it'd require an array of them. Personally I have been using direct buffer since 1.4, so I guess I am also quite used to them as well.
- marginalia_nu 3y ago
  Right, I'm not talking about mapping or allocation costs or anything surrounding the access, just the impact of addressing off heap bytebuffers them seems quite a bit slower than what you get with the new foreign memory API... for whatever reason.
  - xxs 3y ago
    You mean using directBuffers in java - that could be, if the JIT fails to remove the bound checks. It should never be an issue in native code, of course. But yes - it's possible that it happens with Java code. In that case you'd need print assembly and looking at the generated code. Some libraries have 'switched' to straight unsafe use (which has no bound checks, of course), so there is that.
  - bitcharmer 3y ago
    This makes zero sense. I'll believe it when someone proves that (warmed up) direct byte buffer access is slower than Unsafe.
    - marginalia_nu 3y ago
      I'm not comparing against Unsafe, this is JDK21. No unsafe anymore.
bitcharmer 3y ago
> I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff That's just a sign of bad benchmark. Off-heap BBs and foreign memory are both native (non-JVM) heap. They are the same thing