4 ms·

Sorry if this is obvious to Java experts, but much as parallel GC is fine for batch workloads, is there a case for explicit GC control for web workloads? For ex

Sorry if this is obvious to Java experts, but much as parallel GC is fine for batch workloads, is there a case for explicit GC control for web workloads? For example a single request to a web server will create a bunch of objects, but then when it completes 200ms later they can all be destroyed, so why even run GC during the request thread execution?

jacobn 8mo ago
Most web request cases where you care about performance probably have multiple parallel web requests, so there’s no clean separation possible?
- yunnpp 8mo ago
  His question is still valid for latency. That parallel GC in Java still seems to pause threads from a quick search. https://inside.java/2022/08/01/sip062/ https://inside.java/2022/08/01/sip062/
  - zokier 7mo ago
    That's why we got ZGC and Shenandoah, and their generational variants, which have very low pause times (in the order of 1 ms)
- mattclarkdotnet 7mo ago
  Sure, but each request has its own context. Shared resources like DB connection pools will be longer lived but by definition they aren’t alllcated by the request thread. So why not simply exempt everything allocated by a request thread from GC, and simply destroy it on request completion?
  - NovaX 7mo ago
    Go tried that [1], a failed experiment that was a complex NIH version of the generational hypothesis. They currently use a CMS-stye collector. [1] https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ftk-tcRRJcDFANj2VwCB0 https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft...
  - hinkley 7mo ago
    Generational GC assumes that short lived objects tend to come in groups, which is probably the best you can do in an OO language with shared everything.
noelwelsh 7mo ago
There are a few ways of looking at this: - Purely on the JVM, you probably want ZGC (or Shenandoah) because latency is more important than throughput. - On Erlang / the BEAM VM, each thread gets its own private heap, so GC is a per thread operation. If the request doesn't spill over the heap then GC would never need to run during a request handler and all memory could be reclaimed when the handler finishes. - There can still be cases where a request handler allocates memory that is no solely owned by it. E.g. if it causes a new database connection to be allocated in a connection pool, that connection is not owned by the request handler and should not be deallocated when the handler finishes. - The general idea you're getting at is often called "memory regions": you can point to a scope in the code and say "all the memory can be freed when this scope exits". In this case the scope is the request handler. It's the same idea behind arena or slab memory allocation. There are languages that can encode this, and do safe automatic memory management without GC. Rust is an obvious example, but I don't find it very ergonomic. I think the OxCaml [1] and Scala 3 [2] approaches are better. [1]: https://oxcaml.org/documentation/stack-allocation/reference/ https://oxcaml.org/documentation/stack-allocation/reference/ [2]: https://docs.scala-lang.org/scala3/reference/experimental/cc.html https://docs.scala-lang.org/scala3/reference/experimental/cc...
- mattclarkdotnet 7mo ago
  Thank you, that’s what I came here to learn!
- hinkley 7mo ago
  See also arena allocation, for realtime systems. But those systems typically require that any task have a reasonably tight upper bound on memory usage.

jacobn 8mo ago

Most web request cases where you care about performance probably have multiple parallel web requests, so there’s no clean separation possible?

yunnpp 8mo ago
His question is still valid for latency. That parallel GC in Java still seems to pause threads from a quick search. https://inside.java/2022/08/01/sip062/ https://inside.java/2022/08/01/sip062/
- zokier 7mo ago
  That's why we got ZGC and Shenandoah, and their generational variants, which have very low pause times (in the order of 1 ms)
mattclarkdotnet 7mo ago
Sure, but each request has its own context. Shared resources like DB connection pools will be longer lived but by definition they aren’t alllcated by the request thread. So why not simply exempt everything allocated by a request thread from GC, and simply destroy it on request completion?
- NovaX 7mo ago
  Go tried that [1], a failed experiment that was a complex NIH version of the generational hypothesis. They currently use a CMS-stye collector. [1] https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ftk-tcRRJcDFANj2VwCB0 https://docs.google.com/document/d/1gCsFxXamW8RRvOe5hECz98Ft...
- hinkley 7mo ago
  Generational GC assumes that short lived objects tend to come in groups, which is probably the best you can do in an OO language with shared everything.

noelwelsh 7mo ago

There are a few ways of looking at this: - Purely on the JVM, you probably want ZGC (or Shenandoah) because latency is more important than throughput. - On Erlang / the BEAM VM, each thread gets its own private heap, so GC is a per thread operation. If the request doesn't spill over the heap then GC would never need to run during a request handler and all memory could be reclaimed when the handler finishes. - There can still be cases where a request handler allocates memory that is no solely owned by it. E.g. if it causes a new database connection to be allocated in a connection pool, that connection is not owned by the request handler and should not be deallocated when the handler finishes. - The general idea you're getting at is often called "memory regions": you can point to a scope in the code and say "all the memory can be freed when this scope exits". In this case the scope is the request handler. It's the same idea behind arena or slab memory allocation. There are languages that can encode this, and do safe automatic memory management without GC. Rust is an obvious example, but I don't find it very ergonomic. I think the OxCaml [1] and Scala 3 [2] approaches are better. [1]: https://oxcaml.org/documentation/stack-allocation/reference/ https://oxcaml.org/documentation/stack-allocation/reference/ [2]: https://docs.scala-lang.org/scala3/reference/experimental/cc.html https://docs.scala-lang.org/scala3/reference/experimental/cc...

mattclarkdotnet 7mo ago
Thank you, that’s what I came here to learn!
hinkley 7mo ago
See also arena allocation, for realtime systems. But those systems typically require that any task have a reasonably tight upper bound on memory usage.