4 ms·
Looks like they are missing jemalloc in the benchmarks, which would be good to see. I think the simplicity of a 1300 line allocator is very alluring. At that l
by CyberDildonics 10y ago
Looks like they are missing jemalloc in the benchmarks, which would be good to see.
I think the simplicity of a 1300 line allocator is very alluring. At that level it should avoid bloating small programs and is hopefully easy enough to understand if digging in to it.
Also it looks like it is split up into 3 files. consolidating it to a single header file C library would be very nice.
One oddity is the 16 bytes alignment. 64 bytes is the false sharing boundary in x64 so I'm surprised to see this in a concurrent allocator, but maybe I've misunderstood exactly what that means here.
- TwoBit 10y agojemalloc doesn't support non-Unix platforms (eg Windows). You may or may not be able to find a Windows port of any particular jemalloc version, but the developers ignore Windows because they don't use it in their server software. We gave up using jemalloc on Windows because it was such a crap shoot.
- kbenson 10y agoThat sounds like a good basis for a "portable" version project. Other very targeted software (such as OpenSSH[1]) does this. I'm not sure what it takes to make jemalloc windows compatible, but even if the jemalloc developers don't want to support windows, it's possibly they might be more amenable to working with a project dedicated to portability in general. Also, my understanding is that Rust uses jemalloc, and Rust runs on windows, so what does Rust do? Just use a known good ported jemalloc version? 1: https://www.openssh.com/portable.html https://www.openssh.com/portable.html
- steveklabnik 10y agoWe don't use jemalloc on Windows, we use the system allocator, aka HeapAlloc.
- CyberDildonics 10y agoThere are already versions out there on github that have windows support built in, I've used it on windows. I also had problems with it however.
- jasone 10y agojemalloc does support Windows, and stability has improved dramatically in recent releases, primarily because we now have continuous integration coverage for several Windows configurations.
- krenoten 10y agoIndeed. jemalloc uses a lock-free radix tree at its core.
- maniccoder 10y agoI'm going to add jemalloc, lockless and scalloc in the benchmark comparisons soon(ish). Regarding the files, only rpmalloc.[h|c] is needed. The malloc.c file is just for overriding the malloc family of entry points, and provide automatic init/fini glue. The 16 byte alignment is just a note that all returned blocks are naturally aligned to 16 byte boundaries due to the fact that all size buckets are a multiple of 16 bytes internally. It is needed for SSE instructions, which is why I mention it.
- olliej 10y agoAny plan to add the allocators used in modern browsers? bmalloc (webkit), which ever tcmalloc variant chrome currently uses, the current firefox allocator, I have no idea if edge has an allocator that is not the default windows one?
- maniccoder 10y agoI will look into it
- rurban 10y agoFair would be a comparison single-threaded to ptmalloc2 (glibc). I assume the slowdown will be ~5%. And the size granularity is split into 3 (small, medium, large) which is fine, but other's have finer granularities. So benchmarks with object systems would be nice, not just random data.
- maniccoder 10y agoThere already is a benchmark against glibc single threaded, compile the rpmalloc-benchmark repository linked in the readme and run the runall script. Compare rpmalloc 1 thread case against crt 1 thread (with crt being the C runtime, i.e glibc in this case). Graphs and analysis for a Linux system will be added soon, but a spoiler: rpmalloc 22.7m memory ops/CPU second, 26MiB bytes peak, 15% overhead jemalloc 21.3m memory ops/CPU second, 30MiB bytes peak, 33% overhead tcmalloc 20.4m memory ops/CPU second, 30MiB bytes peak, 32% overhead crt 13.5m memory ops/CPU second, 24MiB bytes peak, 7% overhead rpmalloc is faster and has less overhead than both jemalloc and tcmalloc, and a lot faster than glibc at the cost of extra memory overhead. The reasoning behind the granularities is that 16 bytes is a reasonable granularity for smaller objects (below 2KiB in size). Medium sized requests (between 2KiB and 32KiB) have a 512 byte granularity, while large requests (up to 2MiB) have a 62KiB granularity. If you have a suggestion for additional benchmark models I would be happy to add an implementation for it.
- ahoka 10y ago16 bytes is the required aligment for long double, which is the largest (at least on x86) required alignment. If you need more use memalign or something similar.