5 ms·

Reposting my comment from when this was on Reddit: I'm confused about why you need jemalloc instead of just grabbing an N-gigabyte []Node and then manually mak

Reposting my comment from when this was on Reddit: I'm confused about why you need jemalloc instead of just grabbing an N-gigabyte []Node and then manually making them "live" or "dead" by taking pointers. Or take a giant []byte and then cut that up into nodes with unsafe. Seems like those would be equivalent to jemalloc. Maybe throw in some runtime.KeepAlive to keep the GC from scanning it?

BobbyJo 6y ago
Because then you have to actually implement the allocation logic.
- identity0 6y ago
  Allocation logic is much simpler than you think. A solution to a specific problem will always be simpler and faster than a general solution like jemalloc.
  - eloff 6y ago
    Yes? Allocation in a multithreaded program is simple? Balancing allocator space overhead, fragmentation, and allocator performance? What about getting statistics and insight into allocations? Detecting buffer overruns? You couldn't be more wrong. Behold the source code to jemalloc and despair. Allocation can be simple in very specific cases where you can use a single threaded arena.
  - ayende 6y ago
    Allocation and deallocation in a perfomant manner, without introducing fragmentation, while being thread safe isn't easy by any means. Adds in the relevant taxes, such as detecting use after free, etc and that takes a LOT of work
    - earthboundkid 6y ago
      The solution in TFA doesn’t do anything about use after free, does it?
  - WJW 6y ago
    That is a logical fallacy. A solution for a specific _can_ be simpler and faster than a general solution, given enough time. However, jemalloc has had an absolutely huge amount of people-hours invested into optimizing it so it is not unlikely that it'll still be faster for specific problems unless the specific solution also has significant time invested into it.
    - identity0 6y ago
      I have never seen a well-made but general solution beat a well-made and specific solution for one problem, in complexity or run time, ever. This is very true with allocators. A lot of the time people will just use 'malloc' without any thought into what they're actually allocating. For example, if you only allocate/deallocate from one thread, jemalloc is already way overblown in complexity.
      - WJW 6y ago
        That's not what I meant. If you can muster the time and budget for a well-made specific solution, great. What I was getting at is that due to time and/or budget constraints, most custom solution will not actually be well-made and the implementer would have been better off just picking the battle-tested off the shelf solution.
        earthboundkid 6y ago
        But TFA isn’t about just adapting some off the shelf quickie solution. It explains all the hoops necessary to cross the CGo barrier and use jemalloc instead of the normal Go garbage collector. ISTM once you put in that LOE, you’re in the space where a specific solution can beat a general one.
    - scottlamb 6y ago
      Their Allocator library is really an arena, a special-purpose allocator that was discussed on HN recently. [1] I think it's fair to say that when not using GC, it's worth looking for a suitable scope for arenas: short-lived, bounded but significant number of allocations. In many servers, an arena per request is appropriate. You can totally beat directly using the global allocator by layering arenas on top, whether the global allocator is jemalloc or anything else. Batching the frees makes a huge difference, both because there are fewer calls to free and because you have to do less pointer-chasing (with potential cache misses) to decide what calls to make to free. Maybe the arenas reduce the allocations enough (and makes them of reasonably uniform size) such that a simple buddy or slab allocator underneath would beat jemalloc. These simple allocators would have an "unfair" advantage of not having to go through Cgo. Or maybe just each Allocator (arena) using the Go allocator for its backing allocations would be okay. It'd still be garbage-collected which they've been trying to avoid, but the allocator would no longer looking through each individual allocation, so maybe it'd zippier about freeing stuff. Note that (as in my other comment) I still think Rust is a better language for this program. In fairness to them, there are plenty of practical reasons they might have ended up here: * Rust may not have existed or been a reasonable choice when they started coding. In my experience, porting to Rust really is fairly labor-intensive. * There may be significant other parts of the program where Go/GC is more appropriate. * Maybe the whole rest of their company's codebase is in Go and they want to use that expertise. [1] https://news.ycombinator.com/item?id=24762840 https://news.ycombinator.com/item?id=24762840
- deleted 6y ago
  [deleted]
chewxy 6y ago
I think the idea is that so you have memory that is invisible to the Go GC. That way you are free (and at risk of memory leaks) to do whatever you want with that slab.
- mish15 6y ago
  You can do this in pure Go though, you just need to manage the allocator yourself. That is a significant technical undertaking, but once done the advantages are huge. No GC overhead, no cgo. Done properly you can even have multiple readers and writers working concurrently in the same allocation space.
  - chewxy 6y ago
    OK you piqued my curiosity. How?
    - yencabulator 6y ago
      malloc isn't magic, just mmap some memory to dole out.
    - mish15 6y ago
      You just mmap memory as a slice. You can grow and shrink it, flush to disk if backed by a file, etc. In code it’s just a slice. Tough part is you need to allocate, control bounds, binary encode/decode, struct align, defrag, etc. It’s hard work, but the performance is 100% worth it if performance is your main goal.
      - chewxy 6y ago
        TIL mmap'd memory is invisible to Go's GC

BobbyJo 6y ago

Because then you have to actually implement the allocation logic.

identity0 6y ago
Allocation logic is much simpler than you think. A solution to a specific problem will always be simpler and faster than a general solution like jemalloc.
- eloff 6y ago
  Yes? Allocation in a multithreaded program is simple? Balancing allocator space overhead, fragmentation, and allocator performance? What about getting statistics and insight into allocations? Detecting buffer overruns? You couldn't be more wrong. Behold the source code to jemalloc and despair. Allocation can be simple in very specific cases where you can use a single threaded arena.
- ayende 6y ago
  Allocation and deallocation in a perfomant manner, without introducing fragmentation, while being thread safe isn't easy by any means. Adds in the relevant taxes, such as detecting use after free, etc and that takes a LOT of work
  - earthboundkid 6y ago
    The solution in TFA doesn’t do anything about use after free, does it?
- WJW 6y ago
  That is a logical fallacy. A solution for a specific _can_ be simpler and faster than a general solution, given enough time. However, jemalloc has had an absolutely huge amount of people-hours invested into optimizing it so it is not unlikely that it'll still be faster for specific problems unless the specific solution also has significant time invested into it.
  - identity0 6y ago
    I have never seen a well-made but general solution beat a well-made and specific solution for one problem, in complexity or run time, ever. This is very true with allocators. A lot of the time people will just use 'malloc' without any thought into what they're actually allocating. For example, if you only allocate/deallocate from one thread, jemalloc is already way overblown in complexity.
    - WJW 6y ago
      That's not what I meant. If you can muster the time and budget for a well-made specific solution, great. What I was getting at is that due to time and/or budget constraints, most custom solution will not actually be well-made and the implementer would have been better off just picking the battle-tested off the shelf solution.
      - earthboundkid 6y ago
        But TFA isn’t about just adapting some off the shelf quickie solution. It explains all the hoops necessary to cross the CGo barrier and use jemalloc instead of the normal Go garbage collector. ISTM once you put in that LOE, you’re in the space where a specific solution can beat a general one.
  - scottlamb 6y ago
    Their Allocator library is really an arena, a special-purpose allocator that was discussed on HN recently. [1] I think it's fair to say that when not using GC, it's worth looking for a suitable scope for arenas: short-lived, bounded but significant number of allocations. In many servers, an arena per request is appropriate. You can totally beat directly using the global allocator by layering arenas on top, whether the global allocator is jemalloc or anything else. Batching the frees makes a huge difference, both because there are fewer calls to free and because you have to do less pointer-chasing (with potential cache misses) to decide what calls to make to free. Maybe the arenas reduce the allocations enough (and makes them of reasonably uniform size) such that a simple buddy or slab allocator underneath would beat jemalloc. These simple allocators would have an "unfair" advantage of not having to go through Cgo. Or maybe just each Allocator (arena) using the Go allocator for its backing allocations would be okay. It'd still be garbage-collected which they've been trying to avoid, but the allocator would no longer looking through each individual allocation, so maybe it'd zippier about freeing stuff. Note that (as in my other comment) I still think Rust is a better language for this program. In fairness to them, there are plenty of practical reasons they might have ended up here: * Rust may not have existed or been a reasonable choice when they started coding. In my experience, porting to Rust really is fairly labor-intensive. * There may be significant other parts of the program where Go/GC is more appropriate. * Maybe the whole rest of their company's codebase is in Go and they want to use that expertise. [1] https://news.ycombinator.com/item?id=24762840 https://news.ycombinator.com/item?id=24762840
deleted 6y ago
[deleted]

chewxy 6y ago

I think the idea is that so you have memory that is invisible to the Go GC. That way you are free (and at risk of memory leaks) to do whatever you want with that slab.

mish15 6y ago
You can do this in pure Go though, you just need to manage the allocator yourself. That is a significant technical undertaking, but once done the advantages are huge. No GC overhead, no cgo. Done properly you can even have multiple readers and writers working concurrently in the same allocation space.
- chewxy 6y ago
  OK you piqued my curiosity. How?
  - yencabulator 6y ago
    malloc isn't magic, just mmap some memory to dole out.
  - mish15 6y ago
    You just mmap memory as a slice. You can grow and shrink it, flush to disk if backed by a file, etc. In code it’s just a slice. Tough part is you need to allocate, control bounds, binary encode/decode, struct align, defrag, etc. It’s hard work, but the performance is 100% worth it if performance is your main goal.
    - chewxy 6y ago
      TIL mmap'd memory is invisible to Go's GC