5 ms·

This may not be universally true. The Rust docs[1] point out that as of today, compiled binaries use either jemalloc[2] or the system allocator: > Binaries gen

This may not be universally true. The Rust docs[1] point out that as of today, compiled binaries use either jemalloc[2] or the system allocator: > Binaries generated by the compiler will use `alloc_jemalloc` by default (where available). [...] Dynamic and static libraries, however, will use `alloc_system` by default. [1]: https://doc.rust-lang.org/nightly/book/custom-allocators.html https://doc.rust-lang.org/nightly/book/custom-allocators.htm... [2]: http://www.canonware.com/jemalloc/ http://www.canonware.com/jemalloc/

mbrubeck 11y ago
Rust has used jemalloc for a long time (since Rust 0.8, I think). The new feature is the ability to configure it to use the system allocator instead. But whichever malloc implementation it uses, it still relies on the kernel. For example, jemalloc uses the operating system to map memory, by calling library functions like VirtualAlloc (win32) or mmap (POSIX). https://github.com/jemalloc/jemalloc/blob/3a92319ddc5610b755f755cbbbd12791ca9d0c3d/src/pages.c https://github.com/jemalloc/jemalloc/blob/3a92319ddc5610b755...
frozenport 11y ago
That sounds like a deployment nightmare! Does that mean if you return a pointer from a dynamic library you can't free it in a static library, because they were actually using two allocation systems?
- Manishearth 11y ago
  At the moment Rust doesn't have a stable ABI (aside from things you mark as C-compatible), so dynamic libaries that stick around on your system to do things aren't used much. I bet there will be a solution for this once we stabilize the ABI, though I suspect it may already be solved (I think you can tell rust to dynamically link to the allocator too, or something).
- tatterdemalion 11y ago
  Rust binaries essentially never dynamically link Rust libraries, mostly for the reason Manishearth said in the sibling. The decision to have different default allocators is because there were problems linking to Rust from C libraries which used the system allocator, because Rust previously was exclusively using jemalloc. Because Rust libraries being used in Rust projects will be statically linked currently, they will be built using jemalloc. Deploying a pure Rust project is as simple as pushing out a binary (std links to libc, so very few Rust projects are technically pure Rust, but its as good as because the system probably has libc). Also, as a rule, you don't "free" in Rust at all, and code which defines how the memory is deallocated will be defined in the same library it was allocated in. I don't know the details of what it means when two libraries using two different allocators are interacting, but I don't think its as total a conflict as if you were to try to free a pointer with the wrong allocator's free function.
- Gankro 11y ago
  Rust automatically chooses the "best" strategy based on how you're building the source, per https://github.com/rust-lang/rfcs/blob/master/text/1183-swap-out-jemalloc.md https://github.com/rust-lang/rfcs/blob/master/text/1183-swap... If you're building a dynamic or static library, we use the system allocator because Rust is "subordinate". If you're building a binary, we use jemalloc because Rust is in control and jemalloc is basically the best allocator in town. These behaviors can be manually overridden, and you can also provide your own custom global allocator (see linked RFC for details). A Rust project using Rust libraries will generally link them in as rlibs, which is basically the Rust equivalent of an object file. rlibs just inherit whatever allocator from what they're linked into.
- pjmlp 11y ago
  It it nothing new. There are OS which have an heap manager by dynamic library, Windows being one of them. It is never a good idea to try to release memory in a module that didn't allocate it in first place, specially if one doesn't know how it was allocated.
- anon4 11y ago
  That's generally the rule in C/C++-land. If you have two shared objects/dlls linked into your program that are linked with different versions of the C++ runtime, it's not safe for an object allocated by one to be freed by the other.

mbrubeck 11y ago

Rust has used jemalloc for a long time (since Rust 0.8, I think). The new feature is the ability to configure it to use the system allocator instead. But whichever malloc implementation it uses, it still relies on the kernel. For example, jemalloc uses the operating system to map memory, by calling library functions like VirtualAlloc (win32) or mmap (POSIX). https://github.com/jemalloc/jemalloc/blob/3a92319ddc5610b755f755cbbbd12791ca9d0c3d/src/pages.c https://github.com/jemalloc/jemalloc/blob/3a92319ddc5610b755...

frozenport 11y ago

That sounds like a deployment nightmare! Does that mean if you return a pointer from a dynamic library you can't free it in a static library, because they were actually using two allocation systems?

Manishearth 11y ago
At the moment Rust doesn't have a stable ABI (aside from things you mark as C-compatible), so dynamic libaries that stick around on your system to do things aren't used much. I bet there will be a solution for this once we stabilize the ABI, though I suspect it may already be solved (I think you can tell rust to dynamically link to the allocator too, or something).
tatterdemalion 11y ago
Rust binaries essentially never dynamically link Rust libraries, mostly for the reason Manishearth said in the sibling. The decision to have different default allocators is because there were problems linking to Rust from C libraries which used the system allocator, because Rust previously was exclusively using jemalloc. Because Rust libraries being used in Rust projects will be statically linked currently, they will be built using jemalloc. Deploying a pure Rust project is as simple as pushing out a binary (std links to libc, so very few Rust projects are technically pure Rust, but its as good as because the system probably has libc). Also, as a rule, you don't "free" in Rust at all, and code which defines how the memory is deallocated will be defined in the same library it was allocated in. I don't know the details of what it means when two libraries using two different allocators are interacting, but I don't think its as total a conflict as if you were to try to free a pointer with the wrong allocator's free function.
Gankro 11y ago
Rust automatically chooses the "best" strategy based on how you're building the source, per https://github.com/rust-lang/rfcs/blob/master/text/1183-swap-out-jemalloc.md https://github.com/rust-lang/rfcs/blob/master/text/1183-swap... If you're building a dynamic or static library, we use the system allocator because Rust is "subordinate". If you're building a binary, we use jemalloc because Rust is in control and jemalloc is basically the best allocator in town. These behaviors can be manually overridden, and you can also provide your own custom global allocator (see linked RFC for details). A Rust project using Rust libraries will generally link them in as rlibs, which is basically the Rust equivalent of an object file. rlibs just inherit whatever allocator from what they're linked into.
pjmlp 11y ago
It it nothing new. There are OS which have an heap manager by dynamic library, Windows being one of them. It is never a good idea to try to release memory in a module that didn't allocate it in first place, specially if one doesn't know how it was allocated.
anon4 11y ago
That's generally the rule in C/C++-land. If you have two shared objects/dlls linked into your program that are linked with different versions of the C++ runtime, it's not safe for an object allocated by one to be freed by the other.