3 ms·

Why don’t things like malloc get fixed? I’ve known about these issues for years. Does the core team not know?

by api 6y ago

dragontamer 6y ago
Because there's no such thing as a truly general purpose memory allocator. Memory allocation is built to some purpose, and the programmer who uses it should be aware of its problems. There's probably a technical reason why the original glibc malloc does what it does (probably better optimized for low-thread counts)
- api 6y ago
  What you say is technically true but can be used to just hand wave away any issue. Things can almost always be improved. Allocators like TCMalloc are not significantly worse in single threaded mode. The glibc allocator just isn’t very good. I have personally fought with the glibc allocator and seen issues with it that don’t exist on other OSes.
  - trasz 6y ago
    In FreeBSD, the malloc libc uses is jemalloc. Works fine for a general-purpose allocator.
  - klodolph 6y ago
    Historically the glibc allocator had a number of hooks as well as support for unexec(). I think unexec() has been killed (thankfully) but the glibc allocator API is still not exactly equal to other allocator APIs.
- kevingadd 6y ago
  I'm not aware of any production use cases where tcmalloc would be worse than glibc malloc in a way that end users would actually feel. tcmalloc and jemalloc are used all over the place in all sorts of different applications. If glibc malloc is so bad at something like this I find it hard to believe it would be great at anything important if jemalloc/tcmalloc are not.
alexhutcheson 6y ago
The FSF requires copyright assignment for code added to glibc, which makes it difficult to add code from other projects.
- api 6y ago
  What if you asked those other projects? I find it tough to believe nobody would contribute. It’s not like the other mallocs are for profit products.
  - alexhutcheson 6y ago
    Unless those projects have required copyright assignments (neither does, to my knowledge), then you’d need to get agreement from the copyright holder for every contribution that still has code in the project. If there are any holdouts (or even any people you just couldn’t get in touch with) then you’d need to remove the parts they contributed to proceed. More practically, anyone who was asked could reasonably respond “Why do you need to own the copyright? It’s already permissively licensed (in jemalloc’s case, with an LGPLv2.1-compatible license) - just use it!” FSF policy is the only obstacle, and they could make an exception if they wanted to.

Why don’t things like malloc get fixed? I’ve known about these issues for years. Does the core team not know?

by api 6y ago

Why don’t things like malloc get fixed? I’ve known about these issues for years. Does the core team not know?

Because there's no such thing as a truly general purpose memory allocator. Memory allocation is built to some purpose, and the programmer who uses it should be aware of its problems. There's probably a technical reason why the original glibc malloc does what it does (probably better optimized for low-thread counts)

api 6y ago
What you say is technically true but can be used to just hand wave away any issue. Things can almost always be improved. Allocators like TCMalloc are not significantly worse in single threaded mode. The glibc allocator just isn’t very good. I have personally fought with the glibc allocator and seen issues with it that don’t exist on other OSes.
- trasz 6y ago
  In FreeBSD, the malloc libc uses is jemalloc. Works fine for a general-purpose allocator.
- klodolph 6y ago
  Historically the glibc allocator had a number of hooks as well as support for unexec(). I think unexec() has been killed (thankfully) but the glibc allocator API is still not exactly equal to other allocator APIs.
kevingadd 6y ago
I'm not aware of any production use cases where tcmalloc would be worse than glibc malloc in a way that end users would actually feel. tcmalloc and jemalloc are used all over the place in all sorts of different applications. If glibc malloc is so bad at something like this I find it hard to believe it would be great at anything important if jemalloc/tcmalloc are not.

alexhutcheson 6y ago

The FSF requires copyright assignment for code added to glibc, which makes it difficult to add code from other projects.

api 6y ago
What if you asked those other projects? I find it tough to believe nobody would contribute. It’s not like the other mallocs are for profit products.
- alexhutcheson 6y ago
  Unless those projects have required copyright assignments (neither does, to my knowledge), then you’d need to get agreement from the copyright holder for every contribution that still has code in the project. If there are any holdouts (or even any people you just couldn’t get in touch with) then you’d need to remove the parts they contributed to proceed. More practically, anyone who was asked could reasonably respond “Why do you need to own the copyright? It’s already permissively licensed (in jemalloc’s case, with an LGPLv2.1-compatible license) - just use it!” FSF policy is the only obstacle, and they could make an exception if they wanted to.