3 ms·

Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I saw a pretty jawdropping speedup moving from bytebuffers to the new forei

Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff with in the marginalia search index code. Most operations run about 50-100% faster, even discounting the rigmarole needed to deal with mmapping >2 GB files. Haven't dug too deep into why this is so I'm not entirely sure why, may have something to do with being able to declare non-shared memory ranges in the arena allocator, saves a bunch of synchronization maybe? I can't wait for this stuff to leave experimental. It's such a quality of life boon for dealing with off-heap memory, having explicit lifecycle control is.

xxs 3y ago
> Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I'd disagree. Heap based bytebuffers - I don't consider them interesting, they are just byte arrays. The direct ones suffer from unmapping (munmap on linux) - it's a slow process that has to flush the TLB. So the only sane way to use them allocate once/reuse. Anything else I'd consider a programmer error. mmap on large files indeed does suck (a bit) w/ ByteBuffers as it'd require an array of them. Personally I have been using direct buffer since 1.4, so I guess I am also quite used to them as well.
- marginalia_nu 3y ago
  Right, I'm not talking about mapping or allocation costs or anything surrounding the access, just the impact of addressing off heap bytebuffers them seems quite a bit slower than what you get with the new foreign memory API... for whatever reason.
  - xxs 3y ago
    You mean using directBuffers in java - that could be, if the JIT fails to remove the bound checks. It should never be an issue in native code, of course. But yes - it's possible that it happens with Java code. In that case you'd need print assembly and looking at the generated code. Some libraries have 'switched' to straight unsafe use (which has no bound checks, of course), so there is that.
  - bitcharmer 3y ago
    This makes zero sense. I'll believe it when someone proves that (warmed up) direct byte buffer access is slower than Unsafe.
    - marginalia_nu 3y ago
      I'm not comparing against Unsafe, this is JDK21. No unsafe anymore.
bitcharmer 3y ago
> I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff That's just a sign of bad benchmark. Off-heap BBs and foreign memory are both native (non-JVM) heap. They are the same thing

xxs 3y ago

> Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones. I'd disagree. Heap based bytebuffers - I don't consider them interesting, they are just byte arrays. The direct ones suffer from unmapping (munmap on linux) - it's a slow process that has to flush the TLB. So the only sane way to use them allocate once/reuse. Anything else I'd consider a programmer error. mmap on large files indeed does suck (a bit) w/ ByteBuffers as it'd require an array of them. Personally I have been using direct buffer since 1.4, so I guess I am also quite used to them as well.

marginalia_nu 3y ago
Right, I'm not talking about mapping or allocation costs or anything surrounding the access, just the impact of addressing off heap bytebuffers them seems quite a bit slower than what you get with the new foreign memory API... for whatever reason.
- xxs 3y ago
  You mean using directBuffers in java - that could be, if the JIT fails to remove the bound checks. It should never be an issue in native code, of course. But yes - it's possible that it happens with Java code. In that case you'd need print assembly and looking at the generated code. Some libraries have 'switched' to straight unsafe use (which has no bound checks, of course), so there is that.
- bitcharmer 3y ago
  This makes zero sense. I'll believe it when someone proves that (warmed up) direct byte buffer access is slower than Unsafe.
  - marginalia_nu 3y ago
    I'm not comparing against Unsafe, this is JDK21. No unsafe anymore.

bitcharmer 3y ago

> I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff That's just a sign of bad benchmark. Off-heap BBs and foreign memory are both native (non-JVM) heap. They are the same thing