5 ms·

> Arenas are, however, unfeasible to implement in Go because it is a garbage collected language. If you are willing to use cgo, google already implemented one

> Arenas are, however, unfeasible to implement in Go because it is a garbage collected language. If you are willing to use cgo, google already implemented one for gapid. https://github.com/google/gapid/tree/master/core/memory/arena https://github.com/google/gapid/tree/master/core/memory/aren...

pjmlp 5y ago
Not only that, there are other garbage collected languages like D, Nim and C# that offer the language features to do arenas without having to touch any C code. There is still so much education to do.
- p_l 5y ago
  Aren't arenas old news in GC languages in general? Most of the time, their non-presence is due to general pools being just as good most of the time, or people simply not needing them that much with modern GC
  - pjmlp 5y ago
    Yes, so I really did not got how come such assertion was made. Probably lack of experience with machine friendly code.
- dimitrios1 5y ago
  I can't believe we've managed to have this lengthy of a discussion about GC languages and speed without anyone mentioning rust. Has HN turned a corner?
  - shoefindortz 5y ago
    Rust has an arena allocator too[1], but it is implemented with 165(!!!) usages of unsafe. :) [1] https://github.com/fitzgen/bumpalo https://github.com/fitzgen/bumpalo
    - coder543 5y ago
      This is far from the only arena allocator written in Rust. From the same author, a zero-unsafe arena allocator: https://github.com/fitzgen/generational-arena https://github.com/fitzgen/generational-arena There are many, many arena implementations available with varying characteristics. It's disingenuous to act like Rust requires the author of an arena library to write "unsafe" everywhere.
  - pjmlp 5y ago
    Maybe, don't know. In what concerns me, although I like Rust, I only see it for scenarios where any kind of memory allocation is very precious, Ada/SPARK and MISRA-C style. I have been using GC languages with C++ like features, or polyglot codebases, for almost 20 years to think otherwise. Most of the time developers learn about new and miss out on the low level language features. It is a matter of balance, either trying to do everything in a single language, or eventually write a couple of functions in a lower level language that are then used as building blocks for the rest of the application. No need to throw away the ecosystem and developer tooling just to rewrite a data structure.
    - azth 5y ago
      Would you consider codecs or heavy numerical simulations to fall under those memory allocation scenarios that you'd use Rust for as well?
      - pjmlp 5y ago
        That is a good scenario, however you can still use languages like D, Nim, Swift, C#, F#, Go, among others for such scenarios. For example, you can do codecs in C# on WinRT with .NET Native, https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/audio-graphs https://docs.microsoft.com/en-us/windows/uwp/audio-video-cam... In the context of protobuf, https://devblogs.microsoft.com/aspnet/grpc-performance-improvements-in-net-5/ https://devblogs.microsoft.com/aspnet/grpc-performance-impro... Back to Rust, yes it is a good option, I just wouldn't write the whole application on it, just specialized libraries. Hence why I am looking forward to Rust/Windows efforts.
- throwaway894345 5y ago
  Do I misunderstand what arenas are? I thought it was just "allocate this big array as a single allocation rather than N little allocations"? If so, how is that not supported in Go? (e.g., `arena := make([]Foo, 1000000000)`)
  - slimsag 5y ago
    An arena allocator allows you to store many allocations _of different types_ in the same single chunk of memory, and then free all of them at one point in time.
    - throwaway894345 5y ago
      Why can't you do this in Go? I'm 99% sure we can allocate a massive array of bytes using safe Go and use unsafe to cast a chunk of bytes to an instance of a type. This isn't type safe, but neither would the equivalent C code.
      - slimsag 5y ago
        That's what this whole thread is about: you can literally do just that.
        throwaway894345 5y ago
        > That's what this whole thread is about: you can literally do just that I don't know how you get that from the thread: > Arenas are, however, unfeasible to implement in Go because it is a garbage collected language. > If you are willing to use cgo, google already implemented one for gapid. > there are other garbage collected languages like D, Nim and C# that offer the language features to do arenas without having to touch any C code. It seems like the above statements implicitly or explicitly claim that this isn't feasible in Go without C.
        pjmlp 5y ago
        You are misunderstanding the thread, I just mentioned some of the languages I like (still waiting for Go's generics), and the comment I was replying to made an assert about an implementation that uses cgo. Both of us are dismissing the assertion that "Arenas are, however, unfeasible to implement in Go because it is a garbage collected language." You can do manually memory allocation via a syscall into the host OS, use unsafe to cast memory blocks to the types that you want and then clean it all up with defer, assuming the arena is only usable inside a lexical region, otherwise extra care is needed to avoid leaks.

pjmlp 5y ago

Not only that, there are other garbage collected languages like D, Nim and C# that offer the language features to do arenas without having to touch any C code. There is still so much education to do.

p_l 5y ago
Aren't arenas old news in GC languages in general? Most of the time, their non-presence is due to general pools being just as good most of the time, or people simply not needing them that much with modern GC
- pjmlp 5y ago
  Yes, so I really did not got how come such assertion was made. Probably lack of experience with machine friendly code.
dimitrios1 5y ago
I can't believe we've managed to have this lengthy of a discussion about GC languages and speed without anyone mentioning rust. Has HN turned a corner?
- shoefindortz 5y ago
  Rust has an arena allocator too[1], but it is implemented with 165(!!!) usages of unsafe. :) [1] https://github.com/fitzgen/bumpalo https://github.com/fitzgen/bumpalo
  - coder543 5y ago
    This is far from the only arena allocator written in Rust. From the same author, a zero-unsafe arena allocator: https://github.com/fitzgen/generational-arena https://github.com/fitzgen/generational-arena There are many, many arena implementations available with varying characteristics. It's disingenuous to act like Rust requires the author of an arena library to write "unsafe" everywhere.
- pjmlp 5y ago
  Maybe, don't know. In what concerns me, although I like Rust, I only see it for scenarios where any kind of memory allocation is very precious, Ada/SPARK and MISRA-C style. I have been using GC languages with C++ like features, or polyglot codebases, for almost 20 years to think otherwise. Most of the time developers learn about new and miss out on the low level language features. It is a matter of balance, either trying to do everything in a single language, or eventually write a couple of functions in a lower level language that are then used as building blocks for the rest of the application. No need to throw away the ecosystem and developer tooling just to rewrite a data structure.
  - azth 5y ago
    Would you consider codecs or heavy numerical simulations to fall under those memory allocation scenarios that you'd use Rust for as well?
    - pjmlp 5y ago
      That is a good scenario, however you can still use languages like D, Nim, Swift, C#, F#, Go, among others for such scenarios. For example, you can do codecs in C# on WinRT with .NET Native, https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/audio-graphs https://docs.microsoft.com/en-us/windows/uwp/audio-video-cam... In the context of protobuf, https://devblogs.microsoft.com/aspnet/grpc-performance-improvements-in-net-5/ https://devblogs.microsoft.com/aspnet/grpc-performance-impro... Back to Rust, yes it is a good option, I just wouldn't write the whole application on it, just specialized libraries. Hence why I am looking forward to Rust/Windows efforts.
throwaway894345 5y ago
Do I misunderstand what arenas are? I thought it was just "allocate this big array as a single allocation rather than N little allocations"? If so, how is that not supported in Go? (e.g., `arena := make([]Foo, 1000000000)`)
- slimsag 5y ago
  An arena allocator allows you to store many allocations _of different types_ in the same single chunk of memory, and then free all of them at one point in time.
  - throwaway894345 5y ago
    Why can't you do this in Go? I'm 99% sure we can allocate a massive array of bytes using safe Go and use unsafe to cast a chunk of bytes to an instance of a type. This isn't type safe, but neither would the equivalent C code.
    - slimsag 5y ago
      That's what this whole thread is about: you can literally do just that.
      - throwaway894345 5y ago
        > That's what this whole thread is about: you can literally do just that I don't know how you get that from the thread: > Arenas are, however, unfeasible to implement in Go because it is a garbage collected language. > If you are willing to use cgo, google already implemented one for gapid. > there are other garbage collected languages like D, Nim and C# that offer the language features to do arenas without having to touch any C code. It seems like the above statements implicitly or explicitly claim that this isn't feasible in Go without C.
        pjmlp 5y ago
        You are misunderstanding the thread, I just mentioned some of the languages I like (still waiting for Go's generics), and the comment I was replying to made an assert about an implementation that uses cgo. Both of us are dismissing the assertion that "Arenas are, however, unfeasible to implement in Go because it is a garbage collected language." You can do manually memory allocation via a syscall into the host OS, use unsafe to cast memory blocks to the types that you want and then clean it all up with defer, assuming the arena is only usable inside a lexical region, otherwise extra care is needed to avoid leaks.