5 ms·

When will Go have a completely precise garbage collector? In 2013 it seems like poor form to have automatic memory management that isn't asymptotically safe-for

When will Go have a completely precise garbage collector? In 2013 it seems like poor form to have automatic memory management that isn't asymptotically safe-for-space.

Groxx 13y ago
I assume there are tradeoffs? Or are precise garbage collectors simply better everywhere (except maybe complexity)?
- cwzwarich 13y ago
  They're better everywhere except complexity. The two main things that add to the complexity in Go's case are: 1) Existing compilers. Both GCC and LLVM don't precisely track object pointers throughout code generation. 2) Existing FFIs. If you expose an FFI that allows object pointers leak to C code without anything to anchor them as a root then you lock yourself into a conservative GC.
  - deleted 13y ago
    [deleted]
  - cmccabe 13y ago
    I'm pretty sure that the existing conservative GC in cgo doesn't even scan the C part of the heap, so if you try to use unanchored pointers to Go objects, you will crash. More details here, in the comments: https://code.google.com/p/go/source/browse/misc/cgo/gmp/gmp.go?r=release https://code.google.com/p/go/source/browse/misc/cgo/gmp/gmp.... I'm not sure how gccgo does it. There may be some limitations there based on the gcc internals, but I don't see why that would prevent cgo from developing a precise collector.
    - cwzwarich 13y ago
      It wasn't obvious to me looking at the source of te actual GC implementatiob whether it avoids the C stack frames between Go frames. If it does then they don't have much to worry about.
      - cmccabe 13y ago
        I'm not sure about stack frames, but I do remember a lot of discussion about golang using a single contiguous area of virtual memory for its heap. Since C wouldn't be using that same address range, there would be no potential issues where someone would put a Go object on the C heap for a long time and have that result in implicit GC pinning. C stack frames may be an issue (I haven't checked either) but only for upcalls from C, which seem rare.
songgao 13y ago
Check this out: https://github.com/songgao/bufferManager.go https://github.com/songgao/bufferManager.go Not a precise GC, but will be really helpful in some scenarios.
- vanderZwan 13y ago
  Have you updated the benchmarks with Go 1.1? How does it compare to other buffer packages? http://godoc.org/github.com/cznic/bufs http://godoc.org/github.com/cznic/bufs http://godoc.org/github.com/kotokoko/chihaya/bufferpool http://godoc.org/github.com/kotokoko/chihaya/bufferpool
  - songgao 13y ago
    This one is not exactly "buffer" as shown in the two links. It uses a custom structure data type instead of []byte, to avoid conversion between actual data and []byte (marshaling) or type assertion (conversion from/to interface{}). Check https://github.com/songgao/bufferManager.go/blob/master/buffered/data.go https://github.com/songgao/bufferManager.go/blob/master/buff... , inside Data there can be anything. The internal implementation is closer to `bufferpool` (2nd one) though, since they both use a channel as the pool. As to performance in GO 1.1, I did try it with this implementation but have not updated result to Github. The speedup is similar. You could simply clone the repo and run `run_test.sh` yourself :-)
    - vanderZwan 13y ago
      Very clear answer! Apologies for being lazy and just thinking "ooh, buffer, like the other packages with the name buffer" without digging into the sources, and thanks for answering anyway :)

When will Go have a completely precise garbage collector? In 2013 it seems like poor form to have automatic memory management that isn't asymptotically safe-for-space.

Groxx 13y ago

I assume there are tradeoffs? Or are precise garbage collectors simply better everywhere (except maybe complexity)?

cwzwarich 13y ago
They're better everywhere except complexity. The two main things that add to the complexity in Go's case are: 1) Existing compilers. Both GCC and LLVM don't precisely track object pointers throughout code generation. 2) Existing FFIs. If you expose an FFI that allows object pointers leak to C code without anything to anchor them as a root then you lock yourself into a conservative GC.
- deleted 13y ago
  [deleted]
- cmccabe 13y ago
  I'm pretty sure that the existing conservative GC in cgo doesn't even scan the C part of the heap, so if you try to use unanchored pointers to Go objects, you will crash. More details here, in the comments: https://code.google.com/p/go/source/browse/misc/cgo/gmp/gmp.go?r=release https://code.google.com/p/go/source/browse/misc/cgo/gmp/gmp.... I'm not sure how gccgo does it. There may be some limitations there based on the gcc internals, but I don't see why that would prevent cgo from developing a precise collector.
  - cwzwarich 13y ago
    It wasn't obvious to me looking at the source of te actual GC implementatiob whether it avoids the C stack frames between Go frames. If it does then they don't have much to worry about.
    - cmccabe 13y ago
      I'm not sure about stack frames, but I do remember a lot of discussion about golang using a single contiguous area of virtual memory for its heap. Since C wouldn't be using that same address range, there would be no potential issues where someone would put a Go object on the C heap for a long time and have that result in implicit GC pinning. C stack frames may be an issue (I haven't checked either) but only for upcalls from C, which seem rare.

songgao 13y ago

Check this out: https://github.com/songgao/bufferManager.go https://github.com/songgao/bufferManager.go Not a precise GC, but will be really helpful in some scenarios.

vanderZwan 13y ago
Have you updated the benchmarks with Go 1.1? How does it compare to other buffer packages? http://godoc.org/github.com/cznic/bufs http://godoc.org/github.com/cznic/bufs http://godoc.org/github.com/kotokoko/chihaya/bufferpool http://godoc.org/github.com/kotokoko/chihaya/bufferpool
- songgao 13y ago
  This one is not exactly "buffer" as shown in the two links. It uses a custom structure data type instead of []byte, to avoid conversion between actual data and []byte (marshaling) or type assertion (conversion from/to interface{}). Check https://github.com/songgao/bufferManager.go/blob/master/buffered/data.go https://github.com/songgao/bufferManager.go/blob/master/buff... , inside Data there can be anything. The internal implementation is closer to `bufferpool` (2nd one) though, since they both use a channel as the pool. As to performance in GO 1.1, I did try it with this implementation but have not updated result to Github. The speedup is similar. You could simply clone the repo and run `run_test.sh` yourself :-)
  - vanderZwan 13y ago
    Very clear answer! Apologies for being lazy and just thinking "ooh, buffer, like the other packages with the name buffer" without digging into the sources, and thanks for answering anyway :)