6 ms·

I don't know Zig, but it sounds like Zig allows to use arbitrary allocators for anything. The abstraction Rust is getting will only work for things that do acco

I don't know Zig, but it sounds like Zig allows to use arbitrary allocators for anything. The abstraction Rust is getting will only work for things that do account for using arbitrary allocators. Anything that doesn't will end up using the global allocator. That's a significant difference.

steveklabnik 6y ago
That is being backfilled in; Vec already implements it on nightly, IIRC. And really, what you're talking about here is "the standard library data structures," which aren't super likely to be used in firmware anyway. It's a lot easier for ecosystem data structures to add support, after all, they already would choose to call the global allocator, so now they can do either. And it is much easier for them to cut backwards-incompatible changes, if they have to.
- glandium 6y ago
  I mean, in terms of Rust, it sounds like Zig allows to use any allocator for anything in any crate. Not only structs in std or other crates that explicitly allow a custom allocator. In Rust, and only talking about std, you'd need to change a lot of things to allow e.g. BufWrite, etc. to use a custom allocator. And every crate that uses types that allocate stuff under the hood. But maybe I'm misunderstanding what Zig allows.
  - steveklabnik 6y ago
    You are not misunderstanding what Zig allows, but Rust can do the same thing. https://doc.rust-lang.org/stable/core/alloc/trait.Allocator.html https://doc.rust-lang.org/stable/core/alloc/trait.Allocator.... just isn't stable yet. And it's conventional for it to take this as an argument for everything that needs it in Zig. BufWrite would do it the same as any data structure would, an additional parameter, all the same.
    - glandium 6y ago
      I actually did misunderstand. I thought it allowed callers to give an allocator and the callees didn't have to know for it to be used.
      - yellowapple 6y ago
        I mean, I'd say you were mostly right, in the sense that the callee doesn't know the implementation details of the passed allocator; it's only aware of the interface (i.e. the struct of function pointers that defines that interface).
- yellowapple 6y ago
  > which aren't super likely to be used in firmware anyway. Why not? Zig's std library is specifically designed to be usable for freestanding/baremetal targets (e.g. firmware), and the compiler is smart enough to only include the parts of a library (including std) that are actually used. If you do need to reimplement a part of std, you can just... reimplement that part, and import your own implementation instead of the one from std. Unless you're talking purely about Rust?
  - steveklabnik 6y ago
    I am talking purely about Rust, yes. Firmware tends to use libcore, and if it does happen to dynamically allocate memory, liballoc. libstd assumes you have an OS, so...
yellowapple 6y ago
To clarify, it's up to the function being called; the convention set by Zig's standard library is that if a function needs to allocate memory, then the allocator (specifically, a struct of function pointers to an allocator's implementations of realloc and shrink) should be one of the function's arguments. There is of course nothing stopping a function from ignoring this and using the C global allocator if need be (or, as I've done in some experiments, using a C library's custom allocator - in my case that of SQLite). (EDIT: from what I understand, there's technically nothing stopping C from using this sort of strategy, either; a struct of function pointers ain't exactly exotic. It's just a matter of libraries being written with that convention in mind, which doesn't seem to be very common.)
- glandium 6y ago
  Oh so it has, in fact, the same caveat as Rust's scheme. Edit: I guess the difference is that Zig doesn't have years of not supporting it, making the ecosystem more prone to support it.
- roca 6y ago
  In a large complex application you are going to want to use the same allocator everywhere, or close to everywhere, and almost all functions may allocate memory directly or indirectly, in which case this Zig convention will require most every function to have a useless parameter. That sounds enraging.
  - tjalfi 6y ago
    It's common to use multiple allocators in some domains; game developers often use a bump allocator[0] which is reset at the end of every frame. [0] https://twitter.com/SebAaltonen/status/1080235671883841541 https://twitter.com/SebAaltonen/status/1080235671883841541
    - roca 6y ago
      Thanks, that's a good counterexample.
  - 6y ago

steveklabnik 6y ago

That is being backfilled in; Vec already implements it on nightly, IIRC. And really, what you're talking about here is "the standard library data structures," which aren't super likely to be used in firmware anyway. It's a lot easier for ecosystem data structures to add support, after all, they already would choose to call the global allocator, so now they can do either. And it is much easier for them to cut backwards-incompatible changes, if they have to.

glandium 6y ago
I mean, in terms of Rust, it sounds like Zig allows to use any allocator for anything in any crate. Not only structs in std or other crates that explicitly allow a custom allocator. In Rust, and only talking about std, you'd need to change a lot of things to allow e.g. BufWrite, etc. to use a custom allocator. And every crate that uses types that allocate stuff under the hood. But maybe I'm misunderstanding what Zig allows.
- steveklabnik 6y ago
  You are not misunderstanding what Zig allows, but Rust can do the same thing. https://doc.rust-lang.org/stable/core/alloc/trait.Allocator.html https://doc.rust-lang.org/stable/core/alloc/trait.Allocator.... just isn't stable yet. And it's conventional for it to take this as an argument for everything that needs it in Zig. BufWrite would do it the same as any data structure would, an additional parameter, all the same.
  - glandium 6y ago
    I actually did misunderstand. I thought it allowed callers to give an allocator and the callees didn't have to know for it to be used.
    - yellowapple 6y ago
      I mean, I'd say you were mostly right, in the sense that the callee doesn't know the implementation details of the passed allocator; it's only aware of the interface (i.e. the struct of function pointers that defines that interface).
yellowapple 6y ago
> which aren't super likely to be used in firmware anyway. Why not? Zig's std library is specifically designed to be usable for freestanding/baremetal targets (e.g. firmware), and the compiler is smart enough to only include the parts of a library (including std) that are actually used. If you do need to reimplement a part of std, you can just... reimplement that part, and import your own implementation instead of the one from std. Unless you're talking purely about Rust?
- steveklabnik 6y ago
  I am talking purely about Rust, yes. Firmware tends to use libcore, and if it does happen to dynamically allocate memory, liballoc. libstd assumes you have an OS, so...

yellowapple 6y ago

To clarify, it's up to the function being called; the convention set by Zig's standard library is that if a function needs to allocate memory, then the allocator (specifically, a struct of function pointers to an allocator's implementations of realloc and shrink) should be one of the function's arguments. There is of course nothing stopping a function from ignoring this and using the C global allocator if need be (or, as I've done in some experiments, using a C library's custom allocator - in my case that of SQLite). (EDIT: from what I understand, there's technically nothing stopping C from using this sort of strategy, either; a struct of function pointers ain't exactly exotic. It's just a matter of libraries being written with that convention in mind, which doesn't seem to be very common.)

glandium 6y ago
Oh so it has, in fact, the same caveat as Rust's scheme. Edit: I guess the difference is that Zig doesn't have years of not supporting it, making the ecosystem more prone to support it.
roca 6y ago
In a large complex application you are going to want to use the same allocator everywhere, or close to everywhere, and almost all functions may allocate memory directly or indirectly, in which case this Zig convention will require most every function to have a useless parameter. That sounds enraging.
- tjalfi 6y ago
  It's common to use multiple allocators in some domains; game developers often use a bump allocator[0] which is reset at the end of every frame. [0] https://twitter.com/SebAaltonen/status/1080235671883841541 https://twitter.com/SebAaltonen/status/1080235671883841541
  - roca 6y ago
    Thanks, that's a good counterexample.
- 6y ago