5 ms·

As a rust fanboy, I was very happy to see the 99% and memory numbers. Still, very impressive for Java, I wonder whether this is just trading off the 99% percent

As a rust fanboy, I was very happy to see the 99% and memory numbers. Still, very impressive for Java, I wonder whether this is just trading off the 99% percentile peak vs higher common case throughput? I guess rust actually allocates/deallocates all the memory on each request while java batches the deallocation?

oriolid 5y ago
Looking at the memory column, it looks to me like Java benchmark uses a large heap to avoid deallocations for most of requests. This means less but larger garbage collections and reduces average time while increasing 99% where the GC shows up. A long time ago I had to implement a real time service with Java. The best solution was to use whopping 16 megabytes of heap so that it would do a full GC multiple times per second but each of the GCs lasted less than a millisecond.
- cutemonster 5y ago
  Was there a theoretical upper bound for how long collecting those 16 MB could take?
  - oriolid 5y ago
    No idea. It was much faster than the service really needed to be, so we just set the heap to 128M in production and never heard about any latency problems.
- Tsarbomb 5y ago
  Just curious, did you experiment with different GCs?
  - oriolid 5y ago
    At the time CMS was the only option.
amelius 5y ago
In server applications you really shouldn't use the default allocator, but have one allocator per request which can then allocate and batch-deallocate very quickly. Unless you want to keep objects after the request ends, of course.
- Yoric 5y ago
  Is that possible in Rust these days?
  - Nullabillity 5y ago
    Kind of. You can use a bump allocator like bumpalo[0], but any code that allocates needs to be aware of it (or it won't be able to take advantage of it). It also needs to reinvent all collection types (bumpalo provides strings and vectors atm). The plan is to parametrize all std collections on the allocator eventually, but that's not stable yet. [0]: https://docs.rs/bumpalo/3.6.1/bumpalo/ https://docs.rs/bumpalo/3.6.1/bumpalo/
    - amelius 5y ago
      Hopefully this project could allow allocations inside memory-mapped files (which can then later be loaded at a different base address) in one quick load operation (instead of serializing everything).
    - Eikon 5y ago
      Or you could just use jemallocator with background threads. https://crates.io/crates/jemalloc-sys https://crates.io/crates/jemalloc-sys "background_threads (disabled by default): enables background threads by default at run-time. When set to true, background threads are created on demand (the number of background threads will be no more than the number of CPUs or active arenas). Threads run periodically, and handle purging asynchronously. [...]"
- cutemonster 5y ago
  What about an app server same process in memory cache, if using per request allocators? > keep objects after the request ends Yes. Can that be made in a way compatible with per request allocators?
rapsey 5y ago
With optimized memory handling Rust should easily beat Java. Reusing allocations or at least using jemalloc/mimalloc.

oriolid 5y ago

Looking at the memory column, it looks to me like Java benchmark uses a large heap to avoid deallocations for most of requests. This means less but larger garbage collections and reduces average time while increasing 99% where the GC shows up. A long time ago I had to implement a real time service with Java. The best solution was to use whopping 16 megabytes of heap so that it would do a full GC multiple times per second but each of the GCs lasted less than a millisecond.

cutemonster 5y ago
Was there a theoretical upper bound for how long collecting those 16 MB could take?
- oriolid 5y ago
  No idea. It was much faster than the service really needed to be, so we just set the heap to 128M in production and never heard about any latency problems.
Tsarbomb 5y ago
Just curious, did you experiment with different GCs?
- oriolid 5y ago
  At the time CMS was the only option.

amelius 5y ago

In server applications you really shouldn't use the default allocator, but have one allocator per request which can then allocate and batch-deallocate very quickly. Unless you want to keep objects after the request ends, of course.

Yoric 5y ago
Is that possible in Rust these days?
- Nullabillity 5y ago
  Kind of. You can use a bump allocator like bumpalo[0], but any code that allocates needs to be aware of it (or it won't be able to take advantage of it). It also needs to reinvent all collection types (bumpalo provides strings and vectors atm). The plan is to parametrize all std collections on the allocator eventually, but that's not stable yet. [0]: https://docs.rs/bumpalo/3.6.1/bumpalo/ https://docs.rs/bumpalo/3.6.1/bumpalo/
  - amelius 5y ago
    Hopefully this project could allow allocations inside memory-mapped files (which can then later be loaded at a different base address) in one quick load operation (instead of serializing everything).
  - Eikon 5y ago
    Or you could just use jemallocator with background threads. https://crates.io/crates/jemalloc-sys https://crates.io/crates/jemalloc-sys "background_threads (disabled by default): enables background threads by default at run-time. When set to true, background threads are created on demand (the number of background threads will be no more than the number of CPUs or active arenas). Threads run periodically, and handle purging asynchronously. [...]"
cutemonster 5y ago
What about an app server same process in memory cache, if using per request allocators? > keep objects after the request ends Yes. Can that be made in a way compatible with per request allocators?

rapsey 5y ago

With optimized memory handling Rust should easily beat Java. Reusing allocations or at least using jemalloc/mimalloc.