3 ms·

The idea of having a language that can work with or without a garbage collector in different part of a code seems completely impossible to manage to me. How can

by bsaul 6y ago

The idea of having a language that can work with or without a garbage collector in different part of a code seems completely impossible to manage to me. How can this even work in practice ? Do you have twice the stdlib ? Twice the libraries ? I'm pretty sure the D guys figured it out, but i'm really curious to see what's the end result in the real world, once it reaches the dirty hands of end-product developers :))

WalterBright 6y ago
Most of my programs tend to be a mix of GC allocations and malloc allocations. They work together rather smoothly.
- Thiez 6y ago
  How does that work? As I understand it the D GC scans the stack and the GC heap memory for pointers (or values looking like pointers) to determine which objects are reachable. But can the GC scan memory allocated by malloc? How does it know whether a value allocated with malloc has since been freed?
  - david2ndaccount 6y ago
    You call a function to register and deregister a region of memory as potentially holding pointers to Gc-allocated memory.
    - Thiez 6y ago
      Right, but if you do that, is there any advantage over just allocating that memory directly from the GC instead of calling malloc and then registering the memory?
  - mhh__ 6y ago
    The GC is precise where it matters but has no interest in memory you allocate yourself (exceptions to this rule exist but they're fairly obvious). You don't need to worry about it nearly always.
    - Thiez 6y ago
      So you don't want to store pointers to GC objects in memory allocated yourself, or it may get freed if there are no other pointers? Seems like quite the footgun.
      - mhh__ 6y ago
        Basically in this situation you have to inform the GC but realistically this extremely rare in practice. Of the issues the GC has, this isn't one of 'em.
      - destructionator 6y ago
        You do have to be legitimately careful with that. I've tripped myself before while sending objects through a self-pipe. The D GC only runs upon a function call, so with a bit of care you can defer runs, but here I allocated a new object while another one was still in the kernel's pipe buffer... and the GC assumed it was dead since it can't see that buffer and reused its memory for the new object. Took me a while to realize what was going on. But on the other hand, I write a lot of D code and often go back to that same story because it isn't terribly common anyway and easy to manage when you know how. Just yeah if you are mixing techniques without knowing the potential issue you can have trouble.
zozbot234 6y ago
> The idea of having a language that can work with or without a garbage collector in different part of a code ... How can this even work in practice ? "Pluggable" GC-based arenas could make it work quite well. People routinely use ECS (entity-component systems) frameworks in GC-less languages like Rust that could mesh quite well with GC. You do need some extra glue to harmonize the GC-based and GC-less paradigms (such as adding new "GC roots" for any GC'd objects that are being kept "alive" via links from non-GC ones, and demoting them when the references are dropped) but it's quite doable. Rust will probably get facilities for this once "local allocators" are added to the language in a reasonably stable form.
mhh__ 6y ago
The principle is a combination of pragmatism, pain, and then realisation that all memory allocation is slow (allocate less memory). Lots of D code is fundamentally lazy.
- WalterBright 6y ago
  The fastest memory allocator is no-allocation allocation. D has some cool methods for doing that, like the chain template and voldemort types. For example, chain can be used to "concatenate" two arrays without allocating a byte. https://dlang.org/phobos/std_range.html#chain https://dlang.org/phobos/std_range.html#chain https://wiki.dlang.org/Voldemort_types https://wiki.dlang.org/Voldemort_types
p0nce 6y ago
In extreme cases, this can become uneasy and then you need a mental model of what is traceable and what GC "roots" to add. If it's not GC-reachable it gets collected. But in most cases, most people will likely just need custom allocators, or just malloc. You can generate arbitrarily less garbage in D progressively. Mixed allocation strategies is still nice because whatever is in GC memory doesn't need an owner (it has a global owner). Nowadays, the D GC is really unlikely to be a problem whatever your performance requirements. It isn't a problem for corporate users, as you can see on the parent link. Besides the GC performance increased in the last few years.

by bsaul 6y ago

WalterBright 6y ago

Most of my programs tend to be a mix of GC allocations and malloc allocations. They work together rather smoothly.

Thiez 6y ago
How does that work? As I understand it the D GC scans the stack and the GC heap memory for pointers (or values looking like pointers) to determine which objects are reachable. But can the GC scan memory allocated by malloc? How does it know whether a value allocated with malloc has since been freed?
- david2ndaccount 6y ago
  You call a function to register and deregister a region of memory as potentially holding pointers to Gc-allocated memory.
  - Thiez 6y ago
    Right, but if you do that, is there any advantage over just allocating that memory directly from the GC instead of calling malloc and then registering the memory?
- mhh__ 6y ago
  The GC is precise where it matters but has no interest in memory you allocate yourself (exceptions to this rule exist but they're fairly obvious). You don't need to worry about it nearly always.
  - Thiez 6y ago
    So you don't want to store pointers to GC objects in memory allocated yourself, or it may get freed if there are no other pointers? Seems like quite the footgun.
    - mhh__ 6y ago
      Basically in this situation you have to inform the GC but realistically this extremely rare in practice. Of the issues the GC has, this isn't one of 'em.
    - destructionator 6y ago
      You do have to be legitimately careful with that. I've tripped myself before while sending objects through a self-pipe. The D GC only runs upon a function call, so with a bit of care you can defer runs, but here I allocated a new object while another one was still in the kernel's pipe buffer... and the GC assumed it was dead since it can't see that buffer and reused its memory for the new object. Took me a while to realize what was going on. But on the other hand, I write a lot of D code and often go back to that same story because it isn't terribly common anyway and easy to manage when you know how. Just yeah if you are mixing techniques without knowing the potential issue you can have trouble.

zozbot234 6y ago

> The idea of having a language that can work with or without a garbage collector in different part of a code ... How can this even work in practice ? "Pluggable" GC-based arenas could make it work quite well. People routinely use ECS (entity-component systems) frameworks in GC-less languages like Rust that could mesh quite well with GC. You do need some extra glue to harmonize the GC-based and GC-less paradigms (such as adding new "GC roots" for any GC'd objects that are being kept "alive" via links from non-GC ones, and demoting them when the references are dropped) but it's quite doable. Rust will probably get facilities for this once "local allocators" are added to the language in a reasonably stable form.

mhh__ 6y ago

The principle is a combination of pragmatism, pain, and then realisation that all memory allocation is slow (allocate less memory). Lots of D code is fundamentally lazy.

WalterBright 6y ago
The fastest memory allocator is no-allocation allocation. D has some cool methods for doing that, like the chain template and voldemort types. For example, chain can be used to "concatenate" two arrays without allocating a byte. https://dlang.org/phobos/std_range.html#chain https://dlang.org/phobos/std_range.html#chain https://wiki.dlang.org/Voldemort_types https://wiki.dlang.org/Voldemort_types

p0nce 6y ago

In extreme cases, this can become uneasy and then you need a mental model of what is traceable and what GC "roots" to add. If it's not GC-reachable it gets collected. But in most cases, most people will likely just need custom allocators, or just malloc. You can generate arbitrarily less garbage in D progressively. Mixed allocation strategies is still nice because whatever is in GC memory doesn't need an owner (it has a global owner). Nowadays, the D GC is really unlikely to be a problem whatever your performance requirements. It isn't a problem for corporate users, as you can see on the parent link. Besides the GC performance increased in the last few years.