14 ms·
Mistakes with Rust smart pointers: when Deref goes wrong
- agalunar 3y agoIn case the author happens to see this thread – there seems to be a missing less-than sign: str::from_utf8( as Deref>::deref(&self.0)) Thanks for the article! ^_^
- fuzzypixelz 3y agoYou're welcome :) I finally found out what was going wrong here. All angle brackets and other characters were being escaped in other parts of the document, even in <code></code> blocks. Except when it came to code blocks inside <pre></pre> blocks. For context, I use Zola and Cloudflare pages. It turns out there is a bug[^0] in Zola 1.40.0 which reads: > Fix code blocks content not being escaped when not using syntax highlighting I presume that this is the culprit, as I use an external syntax highlighting library. Cloudflare Pages has been famously sluggish with updating the Zola version. For reference, 1.14.0 was released in July 2021. This explains why everything was working fine on my machine™. Thanks for pointing this out, though I unsure how to work around it without changing my setup. [^0]: https://github.com/getzola/zola/blob/master/CHANGELOG.md#0141-2021-08-24 https://github.com/getzola/zola/blob/master/CHANGELOG.md#014...
- qsantos 3y agoFor the record, I have noticed two other places where the angle brackets are missing: struct AsciiString(Vec); and impl Index> for AsciiString {
- singhrac 3y agoI'm using Zola and Cloudflare Pages, and it is currently possibly to use ZOLA_VERSION=0.17.1. I thought I had opted into the Pages v2 beta (I remember joining a Discord for this purpose a while ago), but on the actual page the setting is at v1. You can change this under Settings > Builds & deployments > Build system version. Cheers!
- zetaposter 3y agoThanks! Apparently you can now ZOLA_VERSION to _any_ version you desire in v1 and the build system will install it on demand. I didn't realize that part had changed. On the other hand, v2 doesn't even have Zola; my builds failed with a "failed to find zola command". I guess we're now set for life, and will never have to wait for Cloudflare to update the image in order to bump the version. Less headaches for everyone. Source: https://developers.cloudflare.com/pages/platform/language-support-and-tools/ https://developers.cloudflare.com/pages/platform/language-su...
- deleted 3y ago[deleted]
- n8henrie 3y agoTypo: `impl Index> for AsciiString`
- veber-alex 3y agoThere are a bunch of typos and missing generics all over the place, looks like something there doesn't like angle brackets.
- masklinn 3y agoPossibly an HTML-transparent toolchain, something like the original markdown use case would likely have this sort of effect (as HTML transparency was a feature to Gruber).
- codeflo 3y agoI think I've hit the same kind of problem once. What I learned is that true smart pointers are types that (at least in Rust) aren't really supposed to have methods on their own, to avoid this type of ambiguity during method resolution. For example, Box<T> implements Deref so that you can conveniently use T's methods. If you look at the documentation of Box, the things you can do with the Box itself, like Box::leak, are non-method functions. This means that you always have to qualify them, like Box::leak(my_box), and thus you can't get this type of conflict. Using Deref for types like String or Vec doesn't really fall into this category. They implement something more analogous to an "is-a relationship" in OO. Even outside of those examples, I've seen people manually implement inheritance-like behavior, using Deref to delegate to a field that stores the "base class" instance. If you do that, you're likely to have a name conflict between the two types sooner rather than later. That's fine as long as you're aware that Rust doesn't really do overload resolution -- AFAICT, the "outer" methods always wins regardless of the arguments at the call site.
- planede 3y agoThis feels analogous to hiding in C++. In C++ by default member functions in a base class don't overload with member functions of the same name in a derived class. They get hidden.
- quietbritishjim 3y agoNot disagreeing with you but what's interesting is that C++ doesn't hide in this particular situation. Smart pointers implement operator-> and operator* but then ptr.foo() is still always the foo() method of the pointer type (and won't compile if it doesn't have it) - you need ptr->foo() or (*ptr).foo() to get the method of the pointed-to type. It's one of the few places that C++ is more explicit than Rust. I wonder if there's a reason Rust went a different way? I'm mainly a C++ programmer so I'm genuinely curious.
- OskarS 3y agoI've always liked that feature of C and C++. There's a lot of people that really hate it and in new languages they unify it to be just a dot. Zig does this as well, for instance. I always liked that the distinction was very explicit: use a.b() if you're calling b() on the object itself, use a->b() if you're de-referencing a first. There are lots of things wrong with C and C++ that Zig and Rust fixes, but I never thought of this one being problematic in any way. It's as easy to type, the difference is obvious, and it's easy to see from a glance what's going on.
- Animats 3y ago"The issue, it turns out, is that method search does not check method parameter types against argument types." Right. Rust's doesn't allow much overloading of function names. Partly because C++ did. The C++ overload resolution rules got very complex, especially since they interact with implicit conversions. To keep those rules from introducing errors, there's a rule in C++ that the overload chosen must be at least one step "better" than any matching alternative. Something that was supposed to make thing simpler thus became very complicated. So Rust makes the programmer write that stuff out, which is more verbose but less confusing.
- IshKebab 3y agoYes, I've had equally confusing compile errors from C++ due to its lookup rules. For example removing an unused parameter from a function can cause that function not to be found anymore! I'd argue that is more confusing than this Rust issue.
- ninepoints 3y agoWhat does this comment mean? Function parameters used or not are part of the function signature, so obviously would participate in any form of argument dependent lookup.
- codeflo 3y agoI mean, you can do as small a change as adding a const somewhere, and cause an (almost) independent file somewhere else in your codebase to hit the wrong overload.
- foldr 3y agoPerhaps a case where after removing the unused parameter and updating all the call sites, none of the calls resolve to the original function anymore.
- ninepoints 3y agoAnd we would expect this behavior why?
- spacesuitman2 3y ago>Rather, it only checks for method names and where-clauses such as T: Trait. Types are only resolved in the confirm phase, at which point rustc would have already picked its method. Not following here, the compiler iterates over the deref-chain and acquires a list of method names, but it also "checks for ... where-clauses", what does that mean? A quick look at the linked probe code indicates what we filter on Self traits and return value traits.
- thatxliner 3y agoThat was a good read
- w0ne 3y agoThis is very misleading: str::from_utf8( as Deref>::deref(&self.0))
- glandium 3y ago> Moral of the story: don't implement Deref kids! One of the reasons for Deref (ab)use is that Rust doesn't have a good story for delegation for newtypes yet...
- rascul 3y agoDeref polymorphism is a known anti pattern. https://rust-unofficial.github.io/patterns/anti_patterns/deref.html https://rust-unofficial.github.io/patterns/anti_patterns/der...
- an_ko 3y agoYep. The docs for Deref also dedicate a whole paragraph, with a conclusion in bold, near the start of the page, to why it's a bad idea. https://doc.rust-lang.org/std/ops/trait.Deref.html https://doc.rust-lang.org/std/ops/trait.Deref.html
- adamc 3y agoCouldn't one equally see that as a footgun, given the lack of great solutions for something like this example?
- namjh 3y agoWow, if I were you, I would never think of analyzing the rustc source code. Great job!
- LanternLight83 3y agoGreat post, and great links; thanks!
- kccqzy 3y agoExplicit is better than implicit. And I totally agree with the conclusion of the article: don't implement Deref yourself.
- dathinab 3y agoAs a side note, rust does not have inheritance, at not point did `AsciiString` inherited the `Index` implementation through `Deref`. What happened is that in many usages the compiler would insert an implicit deref which would lead to an `Index` implementation being available. But this does not work in all cases. Not just the case mentioned in the article but many others. The simplest is that it doesn't implement the `Index` trait and hence any generic bound requiring `Index` will fail if you try to use it with a `AsciiString`. So it important to _not think of `Deref` as a form of inheritance ever_ nor use this as a oversimplifications when teaching rust or similar, it will only lead to a lot of confusion. Additionally smart pointer in rust are "pointer like" which means you don't expect them to have methods by themself or anything like that. Hence why special methods on smart pointers are implemented as static function on the type, instead of methods. Through `Deref` can also idiomatically be implemented for types with a owned/borrowed duality (e.g. `String`/`&str`) so `AsciiString/AsciiStr` would work but `AsciiString/str` won't be ideomatic. Then if you idiomatically implement traits on this types you won't run into the issue described in the post, as you implement all you `Index` types and similar on the `&AsciiStr`/`&mut AsciiStr` etc. Anyway there is a way to get a reference to a different kind of type, e.g. getting a `&str` from a `AsciiString`. It's `AsRef`. But that is not automatically applied like `Deref` for the reasons many reasons including stuff like mentioned in the Article. Lastly a better way to think about why it doesn't work is that the type resolver looks if the type is implemented on your concrete type in some form and only if it isn't it will potentially dereference. And due to technical details of how type resolving reliable in multiple direction can be tricky `Index<T>` does count here as "the type is implemented" even if the `T` doesn't match. The articles conclusion of not implementing `Deref` (in most cases) is something I can fully agree with.
- deleted 3y ago[deleted]
- bobbylarrybobby 3y agoGreat article, and pretty unusual for Rust to silently start doing the unexpected thing. From the start, I wished that Rust had used different punctuation for method calls on a `T` and a `SmartPtr<T>`. For instance, `my_t.f()` but `my_boxed_t->f()` (or get rid of auto Deref and just require `(my_boxed_t).f()`). For subscripting it could be something like `my_vec->[index]` (or just require `(my_vec)[index]`. Just something there to remind you that you are in fact using a smart pointer. This would also let you distinguish between `box.leak()` and `box->leak()` and not require `Box::leak(box)`.
- classified 3y ago> the idiomatic course of action would be to use a newtype to wrap a Vec<u8> But isn't that wrong already? The element type `u8` is not an ASCII char. We need a subtype of `u8`, like `u7` if that existed.