5 ms·

I think I've hit the same kind of problem once. What I learned is that true smart pointers are types that (at least in Rust) aren't really supposed to have meth

I think I've hit the same kind of problem once. What I learned is that true smart pointers are types that (at least in Rust) aren't really supposed to have methods on their own, to avoid this type of ambiguity during method resolution. For example, Box<T> implements Deref so that you can conveniently use T's methods. If you look at the documentation of Box, the things you can do with the Box itself, like Box::leak, are non-method functions. This means that you always have to qualify them, like Box::leak(my_box), and thus you can't get this type of conflict. Using Deref for types like String or Vec doesn't really fall into this category. They implement something more analogous to an "is-a relationship" in OO. Even outside of those examples, I've seen people manually implement inheritance-like behavior, using Deref to delegate to a field that stores the "base class" instance. If you do that, you're likely to have a name conflict between the two types sooner rather than later. That's fine as long as you're aware that Rust doesn't really do overload resolution -- AFAICT, the "outer" methods always wins regardless of the arguments at the call site.

planede 3y ago
This feels analogous to hiding in C++. In C++ by default member functions in a base class don't overload with member functions of the same name in a derived class. They get hidden.
- quietbritishjim 3y ago
  Not disagreeing with you but what's interesting is that C++ doesn't hide in this particular situation. Smart pointers implement operator-> and operator* but then ptr.foo() is still always the foo() method of the pointer type (and won't compile if it doesn't have it) - you need ptr->foo() or (*ptr).foo() to get the method of the pointed-to type. It's one of the few places that C++ is more explicit than Rust. I wonder if there's a reason Rust went a different way? I'm mainly a C++ programmer so I'm genuinely curious.
  - OskarS 3y ago
    I've always liked that feature of C and C++. There's a lot of people that really hate it and in new languages they unify it to be just a dot. Zig does this as well, for instance. I always liked that the distinction was very explicit: use a.b() if you're calling b() on the object itself, use a->b() if you're de-referencing a first. There are lots of things wrong with C and C++ that Zig and Rust fixes, but I never thought of this one being problematic in any way. It's as easy to type, the difference is obvious, and it's easy to see from a glance what's going on.
    - DixieDev 3y ago
      I'm mostly indifferent to it because it doesn't really harm readability, and it's not hard to know when to use it, but I can seem why it might be more strongly disliked. You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement. It's also annoying if you find that you can refactor `func(T*)` to `func(T&)` and now you have to replace all `->`s with `.`s.
      - codeflo 3y ago
        > You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement. To the extent that this is a problem, it's a problem with references as a language feature in general, not with the arrow syntax.
      - quietbritishjim 3y ago
        > You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement. That's true but I think the bigger motivation is avoiding ambiguity than seeing when an indirection is happening. (You also can't see whether a method is virtual, i.e. indirecting via the vtable, from the call site.) In C++ references don't have any standalone methods or operators so there's no ambiguity from using methods via a reference just using a dot.
    - codeflo 3y ago
      I'm among the people who would have liked to eliminate even this special case and have a convenient postfix derefence for this. So instead of a->b() you'd have a*.b(), and if you need to dereference again, write a**.b() and so on. But as you said, lots of people really seem to hate this kind of thing. Zig is almost there with its dereference syntax, but still kept the magic dereferencing dot.
  - gpderetta 3y ago
    People have been trying to get 'operator.' overloading in C++ for the last 20 years. Unfortunately this confusion between pointer operations and pointee operations has always been an obstacle :(.
  - mostlylurks 3y ago
    One of the downsides of the C++ approach is that it makes it more cumbersome to make changes in your code. If you for instance make a trivial change where you change a variable from a stack-allocated object accessed directly via .foo() into a heap-allocated one accessed via ->foo(), you have to either manually change all .foo()s into ->foo()s or do some sort of weird work-around, such as making a separate variable that's a reference to the heap allocated object so that you can keep using those .foo()s. This is all relatively pointless busywork, which also pollutes the commit history with relatively meaningless changes.
    - lionkor 3y ago
      those changes in access are semantically different, so i prefer being forced to change code when semantics change
      - Guvante 3y ago
        There is no sematic difference between passing in a reference and passing in a copy of a smart pointers though. Yes there is a difference at the call site between taking a reference to an owned thing and taking potential ownership of the thing. However inside the function block it is noise. Nothing inside the code is semantically different between the two forms, both do work on preexisting data. Certainly the second version could have a change where it does something with respect to the fact it is a smart pointers, changing what used to be a grabbed pointer to a weak pointer to be more explicit about lifetimes for instance. However that code change would be just as explicit as long as there aren't any methods on the smart pointer in the auto deref world.
qalmakka 3y ago
Deref for Vec<T> and String is fine because they are owning containers that decay to their "view" types (slices). Strings and vectors own memory, while views borrow it and can be used to operate on them without reallocating or moving the container itself, treating it as a contiguous list of values. A similar relationship exists between C++'s `std::string` and `std::string_view`, and `std::vector` and `std::span`. IMHO there the logic behind that is very clear and makes a lot of sense. I think it's natural that view-related traits (the ones who operate on the elements of a contiguous area of memory) should be associated with slices, while memory-related ones (the ones that operate on an owning container) should be implemented on the owning types. If you mix them up, you are bound to mess up first and foremost because _it is not logically sound_. You don't need to own a string in order to check if all of its ASCII characters are all caps, for instance, because it's an operation that's also valid on any array of bytes, without the whole concept of "owning memory".
- edbaskerville 3y ago
  Thank you article, parent, GP. You've saved me just in time from never having read the Deref documentation properly. I have some code to delete before I make this mistake again.
- tialaramex 3y ago
  > You don't need to own a string in order to check if all of its ASCII characters are all caps, for instance, because it's an operation that's also valid on any array of bytes, without the whole concept of "owning memory". Interestingly you can ask whether a string is ASCII, "this".is_ascii() is a predicate which does exactly that, but for what I think is your question, "are they all ASCII caps?" you'd need to go via an iterator and a lambda: "THIS".bytes().all(|b| b.is_ascii_uppercase()) The iterator is because there is no provision of each such predicate over the whole string, which seems reasonable (the character class predicates are provided on u8 (ASCII only), and on char (all of them)), but the lambda is due to Rust's 1.0 compatibility commitment plus an unfortunate historical choice. If not for compatibility we'd fix it so you could just write "THIS".bytes().all(u8::is_ascii_uppercase)
  - 3y ago
marcosdumay 3y ago
> AFAICT, the "outer" methods always wins regardless of the arguments at the call site I imagine this is explicitly state in some portion that I have never read of the Deref documentation. The thing is, every introductory text ignores this, and there is no itemized rule in a more global document so that things like Rc and Cell can point directly to it. The Rust docs have a very general discoverability problem, and this is one of its facets. Since things have a very generic and reusable definition, the docs have no good place to put information like that.
- saghm 3y ago
  It definitely would make sens to include this in the docs for `Deref`. I suspect the reason people hadn't thought to do that yet is because there's nothing specific to `Deref` about this behavior; it works the same whenever methods are present on a type due to trait definition[0] (although I imagine it's potentially less confusing for other traits given that the methods they provide are explicitly listed rather than implicit via the associated type). It's also worth noting that if the ambiguity is between two trait impls rather than the concrete type and one of its trait impls, the compiler does require explicit disambiguation (either by calling the method as a top-level function on the trait and passing in the `self` parameter or by casting before calling the method)[1]. [0]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c859cc6877005b245dc5825ae04f62bb https://play.rust-lang.org/?version=stable&mode=debug&editio... [1]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=f7ca6f5224c6511cfb271d3d67d512b4 https://play.rust-lang.org/?version=stable&mode=debug&editio...

planede 3y ago

This feels analogous to hiding in C++. In C++ by default member functions in a base class don't overload with member functions of the same name in a derived class. They get hidden.

quietbritishjim 3y ago
Not disagreeing with you but what's interesting is that C++ doesn't hide in this particular situation. Smart pointers implement operator-> and operator* but then ptr.foo() is still always the foo() method of the pointer type (and won't compile if it doesn't have it) - you need ptr->foo() or (*ptr).foo() to get the method of the pointed-to type. It's one of the few places that C++ is more explicit than Rust. I wonder if there's a reason Rust went a different way? I'm mainly a C++ programmer so I'm genuinely curious.
- OskarS 3y ago
  I've always liked that feature of C and C++. There's a lot of people that really hate it and in new languages they unify it to be just a dot. Zig does this as well, for instance. I always liked that the distinction was very explicit: use a.b() if you're calling b() on the object itself, use a->b() if you're de-referencing a first. There are lots of things wrong with C and C++ that Zig and Rust fixes, but I never thought of this one being problematic in any way. It's as easy to type, the difference is obvious, and it's easy to see from a glance what's going on.
  - DixieDev 3y ago
    I'm mostly indifferent to it because it doesn't really harm readability, and it's not hard to know when to use it, but I can seem why it might be more strongly disliked. You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement. It's also annoying if you find that you can refactor `func(T*)` to `func(T&)` and now you have to replace all `->`s with `.`s.
    - codeflo 3y ago
      > You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement. To the extent that this is a problem, it's a problem with references as a language feature in general, not with the arrow syntax.
    - quietbritishjim 3y ago
      > You can't entirely rely on it to know a dereference is happening because references (e.g. `int&`) aren't subject to the `->` requirement. That's true but I think the bigger motivation is avoiding ambiguity than seeing when an indirection is happening. (You also can't see whether a method is virtual, i.e. indirecting via the vtable, from the call site.) In C++ references don't have any standalone methods or operators so there's no ambiguity from using methods via a reference just using a dot.
  - codeflo 3y ago
    I'm among the people who would have liked to eliminate even this special case and have a convenient postfix derefence for this. So instead of a->b() you'd have a*.b(), and if you need to dereference again, write a**.b() and so on. But as you said, lots of people really seem to hate this kind of thing. Zig is almost there with its dereference syntax, but still kept the magic dereferencing dot.
- gpderetta 3y ago
  People have been trying to get 'operator.' overloading in C++ for the last 20 years. Unfortunately this confusion between pointer operations and pointee operations has always been an obstacle :(.
- mostlylurks 3y ago
  One of the downsides of the C++ approach is that it makes it more cumbersome to make changes in your code. If you for instance make a trivial change where you change a variable from a stack-allocated object accessed directly via .foo() into a heap-allocated one accessed via ->foo(), you have to either manually change all .foo()s into ->foo()s or do some sort of weird work-around, such as making a separate variable that's a reference to the heap allocated object so that you can keep using those .foo()s. This is all relatively pointless busywork, which also pollutes the commit history with relatively meaningless changes.
  - lionkor 3y ago
    those changes in access are semantically different, so i prefer being forced to change code when semantics change
    - Guvante 3y ago
      There is no sematic difference between passing in a reference and passing in a copy of a smart pointers though. Yes there is a difference at the call site between taking a reference to an owned thing and taking potential ownership of the thing. However inside the function block it is noise. Nothing inside the code is semantically different between the two forms, both do work on preexisting data. Certainly the second version could have a change where it does something with respect to the fact it is a smart pointers, changing what used to be a grabbed pointer to a weak pointer to be more explicit about lifetimes for instance. However that code change would be just as explicit as long as there aren't any methods on the smart pointer in the auto deref world.

qalmakka 3y ago

Deref for Vec<T> and String is fine because they are owning containers that decay to their "view" types (slices). Strings and vectors own memory, while views borrow it and can be used to operate on them without reallocating or moving the container itself, treating it as a contiguous list of values. A similar relationship exists between C++'s `std::string` and `std::string_view`, and `std::vector` and `std::span`. IMHO there the logic behind that is very clear and makes a lot of sense. I think it's natural that view-related traits (the ones who operate on the elements of a contiguous area of memory) should be associated with slices, while memory-related ones (the ones that operate on an owning container) should be implemented on the owning types. If you mix them up, you are bound to mess up first and foremost because _it is not logically sound_. You don't need to own a string in order to check if all of its ASCII characters are all caps, for instance, because it's an operation that's also valid on any array of bytes, without the whole concept of "owning memory".

edbaskerville 3y ago
Thank you article, parent, GP. You've saved me just in time from never having read the Deref documentation properly. I have some code to delete before I make this mistake again.
tialaramex 3y ago
> You don't need to own a string in order to check if all of its ASCII characters are all caps, for instance, because it's an operation that's also valid on any array of bytes, without the whole concept of "owning memory". Interestingly you can ask whether a string is ASCII, "this".is_ascii() is a predicate which does exactly that, but for what I think is your question, "are they all ASCII caps?" you'd need to go via an iterator and a lambda: "THIS".bytes().all(|b| b.is_ascii_uppercase()) The iterator is because there is no provision of each such predicate over the whole string, which seems reasonable (the character class predicates are provided on u8 (ASCII only), and on char (all of them)), but the lambda is due to Rust's 1.0 compatibility commitment plus an unfortunate historical choice. If not for compatibility we'd fix it so you could just write "THIS".bytes().all(u8::is_ascii_uppercase)
- 3y ago

marcosdumay 3y ago

> AFAICT, the "outer" methods always wins regardless of the arguments at the call site I imagine this is explicitly state in some portion that I have never read of the Deref documentation. The thing is, every introductory text ignores this, and there is no itemized rule in a more global document so that things like Rc and Cell can point directly to it. The Rust docs have a very general discoverability problem, and this is one of its facets. Since things have a very generic and reusable definition, the docs have no good place to put information like that.

saghm 3y ago
It definitely would make sens to include this in the docs for `Deref`. I suspect the reason people hadn't thought to do that yet is because there's nothing specific to `Deref` about this behavior; it works the same whenever methods are present on a type due to trait definition[0] (although I imagine it's potentially less confusing for other traits given that the methods they provide are explicitly listed rather than implicit via the associated type). It's also worth noting that if the ambiguity is between two trait impls rather than the concrete type and one of its trait impls, the compiler does require explicit disambiguation (either by calling the method as a top-level function on the trait and passing in the `self` parameter or by casting before calling the method)[1]. [0]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c859cc6877005b245dc5825ae04f62bb https://play.rust-lang.org/?version=stable&mode=debug&editio... [1]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=f7ca6f5224c6511cfb271d3d67d512b4 https://play.rust-lang.org/?version=stable&mode=debug&editio...