3 ms·

That's why languages like Zig come with several specialized allocators implemented in the stdlib which can be plugged into any other code in the stdlib that nee

That's why languages like Zig come with several specialized allocators implemented in the stdlib which can be plugged into any other code in the stdlib that needs to allocate memory. You can still implement your own allocator of course, but often there's one already in the stdlib that fits your need (especially also because allocators are stackable/composable).

jeroenhd 2y ago
Rust also has a bunch of tested allocators ready to use, though usually in the form of crates (jemalloc, mimalloc, slab allocators, what have you). The problems caused by people implementing custom allocators are no different in either Rust or Zig.
- flohofwoe 2y ago
  Thinking about it, with proper library design the 'custom allocator' should never be baked into a dependency, but instead injected by the dependency user all the way to the top (and the dependency user can then just inject a different more robust allocator). It's a design philosophy and convention though, so a bit hard to enforce by the language.
  - littlestymaar 2y ago
    Not the language but the standard library can, requiring the allocator to be supplied to the containers at initialization. (Which zig does, or at least did when I looked at it a few years back). The problem with that approach is that you end up with a “function coloring problem” akin to async/await (or `Result` for that matter). Function that allocate becomes “red” functions that can only be called by a function that itself allocates. Like Result and async/await it has the benefit of making things more explicit in the code, but on the flip side it has this contaminating effect that forces you to refactor your code more than otherwise, and also cause combinatorial explosion in helper functions (iterator combinators for instance) if the number of such effect is too high, so there's a balance between explicitness and the burden of adding too many effects like these (or you need to go full “algebraic effect” in your language design but then the complexity budget of your language takes a big step, it's unsuitable for either for Rust or Zig which already have their own share of alien-ness (borrowck and comptime, respectively)).
    - flohofwoe 2y ago
      Odin solves that with an implicit context pointer (which among others carries an allocator) that's passed down the call chain (but that also means that all code is 'colored' with the more expensive color of an implicitly passed extra pointer parameter). A global allocator is IMHO worse than either alternative though. Also, from my experience with Zig, it's not such a big problem. It's actually a good thing to know which code allocates and which doesn't, actually improves trust when working with other people's code
      - littlestymaar 2y ago
        I tend to like the more explicit approach in general (I like async/await and Result more than exceprion typically) but at the same time I acknowledge that there's a trade-off and that a language cannot make everything explicit without becoming unusable, so each language must make their own choice in terms of which effect is going to be explicitly written down in code and which will not. And it's going to depend a lot on the expected use case for the language. With Zig aiming at the kind of code you can write in C it doesn't surprise me that it works pretty well. (Also I'm a bit sceptical about the actual future of the language, which IMHO came a good decade too late, if not two: I feel that Zig could have succeeded where D couldn't, but I don't see it achieving anything nowadays, as the value proposition is too low IMHO. Except as a toolchain for C cross compilation actually, but not as a language).
        galangalalgol 2y ago
        Austral could use its linear types to deny any dependency the ability to allocate memory, just like it can deny io, giving those dependencies access through a provided type. They did it for supply chain safety, but it also lets you specify how your entire dependency tree handles all the system resources.
      - SleepyMyroslav 2y ago
        >A global allocator is IMHO worse than either alternative though. Just fyi, console games settled on global allocator. Because everyone allocates and game needs to run at say 7 out 8Gb used consistently. It makes folks passing around pointers to allocators completely wasting their time and space. There are small parts of code with explicit pooling and allocator pointers but they are like 5% of total code. It is funny when C++17 standard got PMR allocators that make a dream of explicit passing allocators around come true then folks noticed that 8 bytes in every string object are not that cheap. There are very small islands of usage of PMR allocators in the library ecosystem. It does not make global allocators universal truth. It just shows that tradeoffs are different.
- Chabsff 2y ago
  You make the distinction between something being in the standard library vs an arbitrary external package sound like a minor detail. It's not. It makes a world of difference. Having a hard guarantee that you are not dragging any transient dependency is often a big deal. Not to mention the maintenance guarantees.
  - remram 2y ago
    Why do you think that?

jeroenhd 2y ago

Rust also has a bunch of tested allocators ready to use, though usually in the form of crates (jemalloc, mimalloc, slab allocators, what have you). The problems caused by people implementing custom allocators are no different in either Rust or Zig.

flohofwoe 2y ago
Thinking about it, with proper library design the 'custom allocator' should never be baked into a dependency, but instead injected by the dependency user all the way to the top (and the dependency user can then just inject a different more robust allocator). It's a design philosophy and convention though, so a bit hard to enforce by the language.
- littlestymaar 2y ago
  Not the language but the standard library can, requiring the allocator to be supplied to the containers at initialization. (Which zig does, or at least did when I looked at it a few years back). The problem with that approach is that you end up with a “function coloring problem” akin to async/await (or `Result` for that matter). Function that allocate becomes “red” functions that can only be called by a function that itself allocates. Like Result and async/await it has the benefit of making things more explicit in the code, but on the flip side it has this contaminating effect that forces you to refactor your code more than otherwise, and also cause combinatorial explosion in helper functions (iterator combinators for instance) if the number of such effect is too high, so there's a balance between explicitness and the burden of adding too many effects like these (or you need to go full “algebraic effect” in your language design but then the complexity budget of your language takes a big step, it's unsuitable for either for Rust or Zig which already have their own share of alien-ness (borrowck and comptime, respectively)).
  - flohofwoe 2y ago
    Odin solves that with an implicit context pointer (which among others carries an allocator) that's passed down the call chain (but that also means that all code is 'colored' with the more expensive color of an implicitly passed extra pointer parameter). A global allocator is IMHO worse than either alternative though. Also, from my experience with Zig, it's not such a big problem. It's actually a good thing to know which code allocates and which doesn't, actually improves trust when working with other people's code
    - littlestymaar 2y ago
      I tend to like the more explicit approach in general (I like async/await and Result more than exceprion typically) but at the same time I acknowledge that there's a trade-off and that a language cannot make everything explicit without becoming unusable, so each language must make their own choice in terms of which effect is going to be explicitly written down in code and which will not. And it's going to depend a lot on the expected use case for the language. With Zig aiming at the kind of code you can write in C it doesn't surprise me that it works pretty well. (Also I'm a bit sceptical about the actual future of the language, which IMHO came a good decade too late, if not two: I feel that Zig could have succeeded where D couldn't, but I don't see it achieving anything nowadays, as the value proposition is too low IMHO. Except as a toolchain for C cross compilation actually, but not as a language).
      - galangalalgol 2y ago
        Austral could use its linear types to deny any dependency the ability to allocate memory, just like it can deny io, giving those dependencies access through a provided type. They did it for supply chain safety, but it also lets you specify how your entire dependency tree handles all the system resources.
    - SleepyMyroslav 2y ago
      >A global allocator is IMHO worse than either alternative though. Just fyi, console games settled on global allocator. Because everyone allocates and game needs to run at say 7 out 8Gb used consistently. It makes folks passing around pointers to allocators completely wasting their time and space. There are small parts of code with explicit pooling and allocator pointers but they are like 5% of total code. It is funny when C++17 standard got PMR allocators that make a dream of explicit passing allocators around come true then folks noticed that 8 bytes in every string object are not that cheap. There are very small islands of usage of PMR allocators in the library ecosystem. It does not make global allocators universal truth. It just shows that tradeoffs are different.
Chabsff 2y ago
You make the distinction between something being in the standard library vs an arbitrary external package sound like a minor detail. It's not. It makes a world of difference. Having a hard guarantee that you are not dragging any transient dependency is often a big deal. Not to mention the maintenance guarantees.
- remram 2y ago
  Why do you think that?