3 ms·

I can't help but feel this is reintroducing some of the problems it claims to solve in working around the difficulties. * The fact that there is an `Oom` error

I can't help but feel this is reintroducing some of the problems it claims to solve in working around the difficulties. * The fact that there is an `Oom` error that can be thrown means that there is no increase in reliability: you still don't know how much memory you're going to need, but now you have the added problem of asking the user for how much memory you're going to be using — which they are going to be guessing blindly on! * This is because the memory usage is not much more predictable than it would be in easy mode Rust. (Also that "mem.lem()/2" scratch space is kind of crappy; if you're going to do this, do it well. Perhaps in allocating the correct amount of scratch space, you end up with a dynamic allocator at the end of your memory. Does that sound like stack space at the start of memory and heap space at the end of memory? Yes, your programming language does that for you already, but built-in instead of bolted-on.) * Furthermore, the "easy mode" code uses lots of Box, but if you want you can get the benefits of RAII without all the boxes by allocating owned vectors scrupulously. Then you get the benefit of an ownership tracking system in the language's typesystem without having to `unsafe` your way to a half reimplementation of the same. You can get your performance without most of the mess. * Spaghetti can be avoided (if you so desire) in the same way as the previous point. What you do achieve is that at least you can test that Oom condition. Perhaps what you actually want is an allocator that allows for simulating a particular max heap size.

Animats 2y ago
> I can't help but feel this is reintroducing some of the problems it claims to solve in working around the difficulties. When I read "So we do need to write our own allocator", I winced. Almost every time I've had to hunt down a hard bug in a Rust crate, one that requires a debugger, it turns out to be someone who wrote their own unsafe allocator and botched the job.
- flohofwoe 2y ago
  That's why languages like Zig come with several specialized allocators implemented in the stdlib which can be plugged into any other code in the stdlib that needs to allocate memory. You can still implement your own allocator of course, but often there's one already in the stdlib that fits your need (especially also because allocators are stackable/composable).
  - jeroenhd 2y ago
    Rust also has a bunch of tested allocators ready to use, though usually in the form of crates (jemalloc, mimalloc, slab allocators, what have you). The problems caused by people implementing custom allocators are no different in either Rust or Zig.
    - flohofwoe 2y ago
      Thinking about it, with proper library design the 'custom allocator' should never be baked into a dependency, but instead injected by the dependency user all the way to the top (and the dependency user can then just inject a different more robust allocator). It's a design philosophy and convention though, so a bit hard to enforce by the language.
      - littlestymaar 2y ago
        Not the language but the standard library can, requiring the allocator to be supplied to the containers at initialization. (Which zig does, or at least did when I looked at it a few years back). The problem with that approach is that you end up with a “function coloring problem” akin to async/await (or `Result` for that matter). Function that allocate becomes “red” functions that can only be called by a function that itself allocates. Like Result and async/await it has the benefit of making things more explicit in the code, but on the flip side it has this contaminating effect that forces you to refactor your code more than otherwise, and also cause combinatorial explosion in helper functions (iterator combinators for instance) if the number of such effect is too high, so there's a balance between explicitness and the burden of adding too many effects like these (or you need to go full “algebraic effect” in your language design but then the complexity budget of your language takes a big step, it's unsuitable for either for Rust or Zig which already have their own share of alien-ness (borrowck and comptime, respectively)).
        flohofwoe 2y ago
        Odin solves that with an implicit context pointer (which among others carries an allocator) that's passed down the call chain (but that also means that all code is 'colored' with the more expensive color of an implicitly passed extra pointer parameter). A global allocator is IMHO worse than either alternative though. Also, from my experience with Zig, it's not such a big problem. It's actually a good thing to know which code allocates and which doesn't, actually improves trust when working with other people's code

Animats 2y ago

> I can't help but feel this is reintroducing some of the problems it claims to solve in working around the difficulties. When I read "So we do need to write our own allocator", I winced. Almost every time I've had to hunt down a hard bug in a Rust crate, one that requires a debugger, it turns out to be someone who wrote their own unsafe allocator and botched the job.

flohofwoe 2y ago
That's why languages like Zig come with several specialized allocators implemented in the stdlib which can be plugged into any other code in the stdlib that needs to allocate memory. You can still implement your own allocator of course, but often there's one already in the stdlib that fits your need (especially also because allocators are stackable/composable).
- jeroenhd 2y ago
  Rust also has a bunch of tested allocators ready to use, though usually in the form of crates (jemalloc, mimalloc, slab allocators, what have you). The problems caused by people implementing custom allocators are no different in either Rust or Zig.
  - flohofwoe 2y ago
    Thinking about it, with proper library design the 'custom allocator' should never be baked into a dependency, but instead injected by the dependency user all the way to the top (and the dependency user can then just inject a different more robust allocator). It's a design philosophy and convention though, so a bit hard to enforce by the language.
    - littlestymaar 2y ago
      Not the language but the standard library can, requiring the allocator to be supplied to the containers at initialization. (Which zig does, or at least did when I looked at it a few years back). The problem with that approach is that you end up with a “function coloring problem” akin to async/await (or `Result` for that matter). Function that allocate becomes “red” functions that can only be called by a function that itself allocates. Like Result and async/await it has the benefit of making things more explicit in the code, but on the flip side it has this contaminating effect that forces you to refactor your code more than otherwise, and also cause combinatorial explosion in helper functions (iterator combinators for instance) if the number of such effect is too high, so there's a balance between explicitness and the burden of adding too many effects like these (or you need to go full “algebraic effect” in your language design but then the complexity budget of your language takes a big step, it's unsuitable for either for Rust or Zig which already have their own share of alien-ness (borrowck and comptime, respectively)).
      - flohofwoe 2y ago
        Odin solves that with an implicit context pointer (which among others carries an allocator) that's passed down the call chain (but that also means that all code is 'colored' with the more expensive color of an implicitly passed extra pointer parameter). A global allocator is IMHO worse than either alternative though. Also, from my experience with Zig, it's not such a big problem. It's actually a good thing to know which code allocates and which doesn't, actually improves trust when working with other people's code