3 ms·

Psuedo-Rust code here, showing off both styles. Parameterized by an allocator: struct Foo<T, A: Allocator> { // ... } impl<T, A> Foo<T, A> {

Psuedo-Rust code here, showing off both styles. Parameterized by an allocator: struct Foo<T, A: Allocator> { // ... } impl<T, A> Foo<T, A> { fn new() -> Foo<T, A> { // do something to make a new foo, calling functions via A } } contrast with "an allocator is passed in explicitly to every method which actually needs to allocate." struct Foo<T> { // ... } impl<T> Foo<T> { fn new<A: Allocator>(a: A) -> Foo<T> { // do something to make a new foo, calling functions via A } } (EDIT: "new" was a bad choice for me to pick here, see the child comment that uses 'put' instead; I was not trying to suggest that the allocator only parameterizes a constructor, but any call that would need to allocate.)

MrBuddyCasino 4y ago
Thanks for the example. I wasn't aware this is something that was planned for Rust, seems like a pretty large breaking change. Interesting!
- steveklabnik 4y ago
  Nothing is breaking! The change isn't a move to a zig-like style, but instead, that in today's Rust, you have struct Foo<T> { // ... } and not struct Foo<T, A: Allocator> { // ... } This isn't breaking because there is a default type provided for A. The handwave is over "Today's Rust", as this machinery has already landed, in a sense, it's just not really possible to use with the standard library because the allocator API isn't fully stable. Here's an actual example: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html https://doc.rust-lang.org/stable/std/vec/struct.Vec.html See that "A = Global"?
  - MrBuddyCasino 4y ago
    I see. I'm interested in the motivation and design goals behind it, is there something that spells those out? The only thing I could find quickly is this [0]. [0] https://internals.rust-lang.org/t/why-bring-your-own-allocator-apis-are-not-more-popular-in-rust/14494/16 https://internals.rust-lang.org/t/why-bring-your-own-allocat...
    - steveklabnik 4y ago
      https://rust-lang.github.io/rfcs/1974-global-allocators.html https://rust-lang.github.io/rfcs/1974-global-allocators.html was the original RFC. My vague understanding is that there's a working group https://github.com/rust-lang/wg-allocators https://github.com/rust-lang/wg-allocators The further I get from working on Rust day to day, the less I know about these things, so that's all I've got for you.
  - throwawaymaths 4y ago
    what if you want there to be individual allocators (of the same type) that you want to point differently based on <X condition>. For example in a M:N greenthreaded system perhaps you want each greenthread to have its own arena allocator. But you can't use something like threadlocal, because at any time a greenthread might move to any given OS thread? Is that possible in rust?
    - steveklabnik 4y ago
      1. You can do anything you want in Rust, you can make your own collections do whatever you want at any time. 2. These changes are about two things, 2a. the first of which is a trait that represents the concept of an allocator, in case users' code would like to be generic over ones that exist in the ecosystem. You can of course still paper over this yourself but the whole point of a vocabulary trait is so that you don't have to do all the work yourself. 2b. the second of which is how the collections in the standard library are customized by allocators. So, in your own code, absolutely you could do that if you wanted to. With this proposal, and trying to do that with a standard library provided data structure? I'm not an expert on the API that it gives, so I can't really speak to its viability here. I would imagine it's not super simple, given what I do know.
      - throwawaymaths 4y ago
        sorry, I should have been clear: I meant with a standard lib provided data structure. Thanks!
        monocasa 4y ago
        Yes, the data structures in std are parameterized over the allocator, with Global as the default allocator if not specified. https://doc.rust-lang.org/std/vec/struct.Vec.html https://doc.rust-lang.org/std/vec/struct.Vec.html
        throwawaymaths 4y ago
        The point is, they are parametrized over a type, not an instance. By contrast, types in zig that are variadic by allocator are (typically) parametrized over instance.
- deleted 4y ago
  [deleted]
fweimer 4y ago
I understood the comment to be more about some way to parameterize allocators on a per-type basis for standard library types, and not a specific mechanism. C++ nowadays has stateful allocators (e.g., allocate in this specific pool please). With that, even the variant with a type parameter on the type benefits from a constructor argument that specifies an allocator object explicitly. And I don't think the type-parameter-on-functions-only approach can be made safe for immutable data structures because the allocators used for allocation and deallocation must match. And the appropriate allocator needs to be known at Drop time anyway.
epage 4y ago
Having dealt with the brittleness of mixed-allocator code in C/C++, I much prefer library designs that ensure I'm using the right allocator in the right place.
tel 4y ago
I suppose the distinction you're suggesting here is that in the Zig-like example, two `Foo<T>` can have the same type and different allocators. Author mostly notes that this is more flexible. I can see it being possible in each case to have the system be parametric in the allocator, though it'll be a lot more annoying in Rust give the need to pipe the types around. Something between that noisiness and the global default does bias Rust toward being uncreative with its choice of allocator. I wonder, too: in this example, the author notes that because their init function takes an allocator and their event loop doesn't, therefore the event loop does no allocation. But, as long as a global allocator could be accessed from somewhere besides your entrypoint, you could still be calling it. Does Zig offer a capabilities model like that? All in all, I think this difference is more subtle than Matklad is making it out to be, but I in no way doubt that he's correct. Especially in today's Rust. Edit: I did some research on Zig and also noticed HashMapUnmanaged which might be more what Matklad was referencing. In pseudo-Rust, it has impl<K, V> HashMapUnmanaged<K, V> { fn put<A: Allocator>(&mut self, key: K, value: V, allocator: A); } This justifies the statement "Rather, an allocator is passed in explicitly to every method which actually needs to allocate." and makes it much more clear where Zig has gone here.
- steveklabnik 4y ago
  Ah you are right, choosing "new" was a poor choice for my example. I didn't even realize that implied something slightly different until you made this comment, I'm going to update mine slightly to point this out, thank you.

MrBuddyCasino 4y ago

Thanks for the example. I wasn't aware this is something that was planned for Rust, seems like a pretty large breaking change. Interesting!

steveklabnik 4y ago
Nothing is breaking! The change isn't a move to a zig-like style, but instead, that in today's Rust, you have struct Foo<T> { // ... } and not struct Foo<T, A: Allocator> { // ... } This isn't breaking because there is a default type provided for A. The handwave is over "Today's Rust", as this machinery has already landed, in a sense, it's just not really possible to use with the standard library because the allocator API isn't fully stable. Here's an actual example: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html https://doc.rust-lang.org/stable/std/vec/struct.Vec.html See that "A = Global"?
- MrBuddyCasino 4y ago
  I see. I'm interested in the motivation and design goals behind it, is there something that spells those out? The only thing I could find quickly is this [0]. [0] https://internals.rust-lang.org/t/why-bring-your-own-allocator-apis-are-not-more-popular-in-rust/14494/16 https://internals.rust-lang.org/t/why-bring-your-own-allocat...
  - steveklabnik 4y ago
    https://rust-lang.github.io/rfcs/1974-global-allocators.html https://rust-lang.github.io/rfcs/1974-global-allocators.html was the original RFC. My vague understanding is that there's a working group https://github.com/rust-lang/wg-allocators https://github.com/rust-lang/wg-allocators The further I get from working on Rust day to day, the less I know about these things, so that's all I've got for you.
- throwawaymaths 4y ago
  what if you want there to be individual allocators (of the same type) that you want to point differently based on <X condition>. For example in a M:N greenthreaded system perhaps you want each greenthread to have its own arena allocator. But you can't use something like threadlocal, because at any time a greenthread might move to any given OS thread? Is that possible in rust?
  - steveklabnik 4y ago
    1. You can do anything you want in Rust, you can make your own collections do whatever you want at any time. 2. These changes are about two things, 2a. the first of which is a trait that represents the concept of an allocator, in case users' code would like to be generic over ones that exist in the ecosystem. You can of course still paper over this yourself but the whole point of a vocabulary trait is so that you don't have to do all the work yourself. 2b. the second of which is how the collections in the standard library are customized by allocators. So, in your own code, absolutely you could do that if you wanted to. With this proposal, and trying to do that with a standard library provided data structure? I'm not an expert on the API that it gives, so I can't really speak to its viability here. I would imagine it's not super simple, given what I do know.
    - throwawaymaths 4y ago
      sorry, I should have been clear: I meant with a standard lib provided data structure. Thanks!
      - monocasa 4y ago
        Yes, the data structures in std are parameterized over the allocator, with Global as the default allocator if not specified. https://doc.rust-lang.org/std/vec/struct.Vec.html https://doc.rust-lang.org/std/vec/struct.Vec.html
        throwawaymaths 4y ago
        The point is, they are parametrized over a type, not an instance. By contrast, types in zig that are variadic by allocator are (typically) parametrized over instance.
deleted 4y ago
[deleted]

fweimer 4y ago

I understood the comment to be more about some way to parameterize allocators on a per-type basis for standard library types, and not a specific mechanism. C++ nowadays has stateful allocators (e.g., allocate in this specific pool please). With that, even the variant with a type parameter on the type benefits from a constructor argument that specifies an allocator object explicitly. And I don't think the type-parameter-on-functions-only approach can be made safe for immutable data structures because the allocators used for allocation and deallocation must match. And the appropriate allocator needs to be known at Drop time anyway.

epage 4y ago

Having dealt with the brittleness of mixed-allocator code in C/C++, I much prefer library designs that ensure I'm using the right allocator in the right place.

tel 4y ago

I suppose the distinction you're suggesting here is that in the Zig-like example, two `Foo<T>` can have the same type and different allocators. Author mostly notes that this is more flexible. I can see it being possible in each case to have the system be parametric in the allocator, though it'll be a lot more annoying in Rust give the need to pipe the types around. Something between that noisiness and the global default does bias Rust toward being uncreative with its choice of allocator. I wonder, too: in this example, the author notes that because their init function takes an allocator and their event loop doesn't, therefore the event loop does no allocation. But, as long as a global allocator could be accessed from somewhere besides your entrypoint, you could still be calling it. Does Zig offer a capabilities model like that? All in all, I think this difference is more subtle than Matklad is making it out to be, but I in no way doubt that he's correct. Especially in today's Rust. Edit: I did some research on Zig and also noticed HashMapUnmanaged which might be more what Matklad was referencing. In pseudo-Rust, it has impl<K, V> HashMapUnmanaged<K, V> { fn put<A: Allocator>(&mut self, key: K, value: V, allocator: A); } This justifies the statement "Rather, an allocator is passed in explicitly to every method which actually needs to allocate." and makes it much more clear where Zig has gone here.

steveklabnik 4y ago
Ah you are right, choosing "new" was a poor choice for my example. I didn't even realize that implied something slightly different until you made this comment, I'm going to update mine slightly to point this out, thank you.