6 ms·

> We'll never get there, for the mythical "there" where GC overhead and latency isn't an issue that needs to be tuned in deployment. I think that for the most

by pron 8y ago

> We'll never get there, for the mythical "there" where GC overhead and latency isn't an issue that needs to be tuned in deployment. I think that for the most part we're "there" already. Such improvements in GCs, like Shenandoah, C4 and ZGC, are more than keeping up with most applications' evolving requirements, and yield tremendous benefits for the relatively low cost of added RAM footprint. > it's time to start recognizing that fact instead of trying to make the JVM and .NET do what we get from explicit heaps in C and Rust. To do better than the JVM, especially in concurrent applications, you have to work hard. What we "get" from manual memory management we actually buy for a rather steep price (and remember that for shared data structures, which are extremely important in concurrent applications, Rust also uses a GC, just a particularly primitive one -- reference counting). And even what we buy is not universally "better." I just saw this nice benchmark the other day, that compares allocation costs (even without concurrency). As always, it gives a small part of the picture, but a very real one: https://github.com/rbehrends/btree-alloc https://github.com/rbehrends/btree-alloc. I don't think anyone can reasonably claim that manual memory management is "generally better" than modern GCs, even once the effort is invested. It's better in some respects and worse in others. So it's not even clear which of them has that "last 5%" (and sometimes more) advantage.

ajross 8y ago
See... calling reference counting "primitive GC" is I think symptomatic of the wrong thinking here. Reference counting has, for the small heap blocks typical in apps that tend to run in this space, non-ignorable (not "high", but real) space and CPU overhead relative to traditional GC. But it's 100% deterministic! The outrageous amount of work that's gone into two new GC ideas over the last half century has, almost exclusively, been trying to address the latency excursions inherent in trying to collect a giant heap. And it's a completely manufactured problem, because refcounting (and manual allocation strategies more generally) never head it to begin with! So... I just don't see the point you're trying to make. I mean, yeah, there are a handful of workloads where a good GC will beat naive heap allocation (though the heap solution can always be optimized to beat a GC, by definition). But everything has its sweet spot. My point is that Java puts its bet on the idea that the GC will work better for effectively all useful workloads. And that's not true. It hasn't been true for 20 years (c.f. all the GC work we're discussing), and it won't be true in the next 20 years either. "We won't get there", as it were.
- pron 8y ago
  > for the small heap blocks typical in apps that tend to run in this space What space is that? The big applications that the JVM targets have very large working sets (hundreds of GBs or even a couple TBs). > The outrageous amount of work that's gone into two new GC ideas over the last half century has, almost exclusively, been trying to address the latency excursions inherent in trying to collect a giant heap. But that work has been successful! We're now at 1.5ms max pause with an acceptable cost to throughput. > And it's a completely manufactured problem, because refcounting (and manual allocation strategies more generally) never head it to begin with! This is just false. Ref-counted deallocation of a large data structure also introduces latency. Unless you're also controlling the allocator (which is more work), it's also not entirely deterministic. > there are a handful of workloads Those "handful of workloads" are the ones that interest those companies running on the JDK, like Google, Netflix, Apple, Twitter, Amazon and many, many others. I would agree that some use cases benefit more from manual/refcount-collection and some benefit more from tracing, but the market for tracing GCs is humongous. > It hasn't been true for 20 years (c.f. all the GC work we're discussing), and it won't be true in the next 20 years either. "We won't get there", as it were. Well, I strongly disagree, and the millions of happy JDK users will likely disagree, as well.
- mcguire 8y ago
  You realize the latency excursion needed to deterministically free a large data structure is also problematic, exacerbated by the background data structure needed to manage allocation and deallocation. (Not all data structures can be "small".) Further, manual heaps tend to scatter your small blocks all over your address space, making your heap work harder as well as doing bad things to your caches. (Unless you go to the work of building cache-aware data structures, which most developers are not capable of considering that they have problems with the discipline needed to make manual management work at all.)
  - ajross 8y ago
    Of course, but it's treatable. Large applications deployed on traditional heap implementations do not, as a general rule, suffer from the kind of GC tuning voodoo that plagues the JVM and which inspired ZGC and Shenandoah (and two decades of their ancestors!). There's no free lunch, but if you know heap behavior is a performance issue in a giant C++ app or whatever, you have a big list of tools with which to address it in software. And in practice this works. Large deployments of, say, PostgreSQL don't require that the admins tune the allocation strategy of the process or carefully size the heap memory internal to the process (obviously there is external tuning at the OS level, though).
zzzcpan 8y ago
You can always do better than GCed shared memory VM, especially in concurrent applications. And in general shared data structures always slow down concurrent applications. There is no future for JVM in concurrency at this point. Even a toy VM/language designed for concurrency will be able to beat it.
- pron 8y ago
  No, but you can try if you think you can.
  - zzzcpan 8y ago
    We are not in the dark ages anymore. Denying reality doesn't help anyone here. Shared memory is dead for concurrency, it can't be made neither fast, nor easy to use.
    - pron 8y ago
      I don't know what reality I'm denying. These post-dark-ages days Amazon, Apple, Google, Netflix and Twitter -- among many, many others -- are betting heavily on OpenJDK, with its shared heap GCs, for many/most of their big, concurrent backend applications.
      - zzzcpan 8y ago
        I though we were talking about engineering challenges, research, not bets execs of large companies make for very different reasons, mostly hiring reasons.
        pron 8y ago
        Oh, sorry, I didn't realize you're a Google/Apple/Netflix/Amazon executive who's intimately familiar with their technical decisions. It's certainly true there is a lot of research on non-shared memory, but also a lot of research on shared memory. If non-shared has so decisively won, then not only the executives but also the researchers didn't get that memo. If and when non-shared "wins", the popular industry platforms will adjust as well.
erichocean 8y ago
> Rust also uses a GC, just a particularly primitive one -- reference counting This meme that reference counting and garbage collection are even remotely the same needs to die in a fire. Reference counting is not, and never has been, "primitive garbage collection", and garbage collection is not, and never has been, "super sophisticated reference counting".
- earenndil 8y ago
  Garbage collection means a way to automatically collect garbage. Reference counts automatically collect garbage. Therefore reference counting is a subset of garbage collection.
  - erichocean 8y ago
    > Reference counts automatically collect garbage. Reference counts are used to manage resources, one of which is memory (and I'm assuming you are referring to no-longer-referenced memory as "garbage"). But that's not the only use for reference counts! You can, and should, use reference counts for external resources, such as files, or memory buffers on a GPU, or…. Garbage collection is COMPLETELY USELESS for managing non-memory resources because you need to be able to deterministically release non-memory resources (and oftentimes, memory resources—e.g., on the GPU). And since all a GC can do is examine memory, well, it doesn't work. Go ahead: point me to a 3D game engine using garbage collection for GPU resources. I'll wait. So yeah: "reference counting is a subset of garbage collection" is absolutely not true, and even the most basic knowledge of how reference counting is used in practice should be enough to convince anyone of that. Again: this meme needs to die. Garbage collection is not "super sophisticated reference counting", and in fact, can't even be used in most places where reference counting is used. It's not a substitution, or a superset, of reference counting. GC is a compelling alternative to reference counting for resources just in the main memory heap in some situations where performance (esp. total memory usage) isn't an important concern relative to, say, programmer productivity.
    - erik_seaberg 8y ago
      You can close a non-memory resource when an object that owns it becomes garbage (Java calls this a "finalizer"), it's just not common. Those resources tend to be too expensive to leave unused indefinitely, and explicit closing risks IOExceptions but not disastrous undefined behavior. Refcounting detects new garbage more quickly than periodic tracing, which is why it costs so much more in bus cycles. But both are dynamically detecting live objects and garbage. I've also seen systems where the only reference count values are "one" and "more than one", with the latter going uncollected until tracing.
Jweb_Guru 8y ago
Rust doesn't need to use reference counting for shared data structures, which I'm pretty sure you already know. The use of reference counting is about dealing with unknown lifetimes, not dealing with sharing.
- afiori 8y ago
  If there is no sharing then you just need a box, reference counting is for when there is not a clear owner and many references all still want to share. I am not sure how the thread situation is for GCs and Java, but Arc is indeed reference counting and I believe it is used quite often in rust.
  - Jweb_Guru 8y ago
    You can just use a shared reference to achieve sharing in Rust; you don't need a Box or an Arc. Pretty much every shared core data structure in Rust works fine with references with no Arc in sight, and this makes plenty of sense when you consider that the main way you interact with a type protected by an Arc is by extracting a shared reference from it.
    - afiori 8y ago
      But one of the advantages of Rc is to allow for dynamic GC-like lifetimes, where values have no clear owners.
      - Jweb_Guru 8y ago
        Yes, we are in agreement about that. But they are certainly not necessary for shared data structures (even concurrent ones) and that is what the post to which I was responding said. This also isn't a purely pedantic point; scoped threads and libraries like Rayon are explicitly designed to allow concurrent sharing of values where the sharing has a bounded lifetime, and they do not require reference counting in order to support this (indeed, some usecases of these libraries, such as splitting up arrays into disjoint slices, don't really work with reference counting).
p_l 8y ago
Rust's region management also falls under automatic memory management, just "compile-time elided"

by pron 8y ago

ajross 8y ago

See... calling reference counting "primitive GC" is I think symptomatic of the wrong thinking here. Reference counting has, for the small heap blocks typical in apps that tend to run in this space, non-ignorable (not "high", but real) space and CPU overhead relative to traditional GC. But it's 100% deterministic! The outrageous amount of work that's gone into two new GC ideas over the last half century has, almost exclusively, been trying to address the latency excursions inherent in trying to collect a giant heap. And it's a completely manufactured problem, because refcounting (and manual allocation strategies more generally) never head it to begin with! So... I just don't see the point you're trying to make. I mean, yeah, there are a handful of workloads where a good GC will beat naive heap allocation (though the heap solution can always be optimized to beat a GC, by definition). But everything has its sweet spot. My point is that Java puts its bet on the idea that the GC will work better for effectively all useful workloads. And that's not true. It hasn't been true for 20 years (c.f. all the GC work we're discussing), and it won't be true in the next 20 years either. "We won't get there", as it were.

pron 8y ago
> for the small heap blocks typical in apps that tend to run in this space What space is that? The big applications that the JVM targets have very large working sets (hundreds of GBs or even a couple TBs). > The outrageous amount of work that's gone into two new GC ideas over the last half century has, almost exclusively, been trying to address the latency excursions inherent in trying to collect a giant heap. But that work has been successful! We're now at 1.5ms max pause with an acceptable cost to throughput. > And it's a completely manufactured problem, because refcounting (and manual allocation strategies more generally) never head it to begin with! This is just false. Ref-counted deallocation of a large data structure also introduces latency. Unless you're also controlling the allocator (which is more work), it's also not entirely deterministic. > there are a handful of workloads Those "handful of workloads" are the ones that interest those companies running on the JDK, like Google, Netflix, Apple, Twitter, Amazon and many, many others. I would agree that some use cases benefit more from manual/refcount-collection and some benefit more from tracing, but the market for tracing GCs is humongous. > It hasn't been true for 20 years (c.f. all the GC work we're discussing), and it won't be true in the next 20 years either. "We won't get there", as it were. Well, I strongly disagree, and the millions of happy JDK users will likely disagree, as well.
mcguire 8y ago
You realize the latency excursion needed to deterministically free a large data structure is also problematic, exacerbated by the background data structure needed to manage allocation and deallocation. (Not all data structures can be "small".) Further, manual heaps tend to scatter your small blocks all over your address space, making your heap work harder as well as doing bad things to your caches. (Unless you go to the work of building cache-aware data structures, which most developers are not capable of considering that they have problems with the discipline needed to make manual management work at all.)
- ajross 8y ago
  Of course, but it's treatable. Large applications deployed on traditional heap implementations do not, as a general rule, suffer from the kind of GC tuning voodoo that plagues the JVM and which inspired ZGC and Shenandoah (and two decades of their ancestors!). There's no free lunch, but if you know heap behavior is a performance issue in a giant C++ app or whatever, you have a big list of tools with which to address it in software. And in practice this works. Large deployments of, say, PostgreSQL don't require that the admins tune the allocation strategy of the process or carefully size the heap memory internal to the process (obviously there is external tuning at the OS level, though).

zzzcpan 8y ago

You can always do better than GCed shared memory VM, especially in concurrent applications. And in general shared data structures always slow down concurrent applications. There is no future for JVM in concurrency at this point. Even a toy VM/language designed for concurrency will be able to beat it.

pron 8y ago
No, but you can try if you think you can.
- zzzcpan 8y ago
  We are not in the dark ages anymore. Denying reality doesn't help anyone here. Shared memory is dead for concurrency, it can't be made neither fast, nor easy to use.
  - pron 8y ago
    I don't know what reality I'm denying. These post-dark-ages days Amazon, Apple, Google, Netflix and Twitter -- among many, many others -- are betting heavily on OpenJDK, with its shared heap GCs, for many/most of their big, concurrent backend applications.
    - zzzcpan 8y ago
      I though we were talking about engineering challenges, research, not bets execs of large companies make for very different reasons, mostly hiring reasons.
      - pron 8y ago
        Oh, sorry, I didn't realize you're a Google/Apple/Netflix/Amazon executive who's intimately familiar with their technical decisions. It's certainly true there is a lot of research on non-shared memory, but also a lot of research on shared memory. If non-shared has so decisively won, then not only the executives but also the researchers didn't get that memo. If and when non-shared "wins", the popular industry platforms will adjust as well.

erichocean 8y ago

> Rust also uses a GC, just a particularly primitive one -- reference counting This meme that reference counting and garbage collection are even remotely the same needs to die in a fire. Reference counting is not, and never has been, "primitive garbage collection", and garbage collection is not, and never has been, "super sophisticated reference counting".

earenndil 8y ago
Garbage collection means a way to automatically collect garbage. Reference counts automatically collect garbage. Therefore reference counting is a subset of garbage collection.
- erichocean 8y ago
  > Reference counts automatically collect garbage. Reference counts are used to manage resources, one of which is memory (and I'm assuming you are referring to no-longer-referenced memory as "garbage"). But that's not the only use for reference counts! You can, and should, use reference counts for external resources, such as files, or memory buffers on a GPU, or…. Garbage collection is COMPLETELY USELESS for managing non-memory resources because you need to be able to deterministically release non-memory resources (and oftentimes, memory resources—e.g., on the GPU). And since all a GC can do is examine memory, well, it doesn't work. Go ahead: point me to a 3D game engine using garbage collection for GPU resources. I'll wait. So yeah: "reference counting is a subset of garbage collection" is absolutely not true, and even the most basic knowledge of how reference counting is used in practice should be enough to convince anyone of that. Again: this meme needs to die. Garbage collection is not "super sophisticated reference counting", and in fact, can't even be used in most places where reference counting is used. It's not a substitution, or a superset, of reference counting. GC is a compelling alternative to reference counting for resources just in the main memory heap in some situations where performance (esp. total memory usage) isn't an important concern relative to, say, programmer productivity.
  - erik_seaberg 8y ago
    You can close a non-memory resource when an object that owns it becomes garbage (Java calls this a "finalizer"), it's just not common. Those resources tend to be too expensive to leave unused indefinitely, and explicit closing risks IOExceptions but not disastrous undefined behavior. Refcounting detects new garbage more quickly than periodic tracing, which is why it costs so much more in bus cycles. But both are dynamically detecting live objects and garbage. I've also seen systems where the only reference count values are "one" and "more than one", with the latter going uncollected until tracing.

Jweb_Guru 8y ago

Rust doesn't need to use reference counting for shared data structures, which I'm pretty sure you already know. The use of reference counting is about dealing with unknown lifetimes, not dealing with sharing.

afiori 8y ago
If there is no sharing then you just need a box, reference counting is for when there is not a clear owner and many references all still want to share. I am not sure how the thread situation is for GCs and Java, but Arc is indeed reference counting and I believe it is used quite often in rust.
- Jweb_Guru 8y ago
  You can just use a shared reference to achieve sharing in Rust; you don't need a Box or an Arc. Pretty much every shared core data structure in Rust works fine with references with no Arc in sight, and this makes plenty of sense when you consider that the main way you interact with a type protected by an Arc is by extracting a shared reference from it.
  - afiori 8y ago
    But one of the advantages of Rc is to allow for dynamic GC-like lifetimes, where values have no clear owners.
    - Jweb_Guru 8y ago
      Yes, we are in agreement about that. But they are certainly not necessary for shared data structures (even concurrent ones) and that is what the post to which I was responding said. This also isn't a purely pedantic point; scoped threads and libraries like Rayon are explicitly designed to allow concurrent sharing of values where the sharing has a bounded lifetime, and they do not require reference counting in order to support this (indeed, some usecases of these libraries, such as splitting up arrays into disjoint slices, don't really work with reference counting).

p_l 8y ago

Rust's region management also falls under automatic memory management, just "compile-time elided"