4 ms·

I only skimmed this article, but is it fair to say if this makes it into Rust proper that we would gain self referential structs?

orlp 3y ago
No, self-referential structs are disallowed by the nature of Rust having move semantics by default, without a concept of a 'move constructor'.
- slashdev 3y ago
  It’s not clear to me that the compiler couldn’t detect when you try to move a self referential string and prevent it. It basically does if you use pin.
  - orlp 3y ago
    I'm not saying a theoretical language couldn't be made that allows this, but in Rust it would be breaking backwards compatibility. For example [T]::sort (obviously) needs to be able to move the elements in the passed array, but there is no Move bound on T or something similar. > It basically does if you use pin. And we have Pin today, but I assumed the OP meant 'self-referential struct' as in, without using Pin.
    - nu11ptr 3y ago
      > And we have Pin today, but I assumed the OP meant 'self-referential struct' as in, without using Pin. I did, but with Pin would be fine as well
    - devit 3y ago
      It would have to be introduced by making it default, so all existing generic items would have Move bound on all generic parameters. Only in a new edition the default can then be swapped.
  - comex 3y ago
    Right. Pin is the canonical way to have objects that can't be moved, and it was invented in order to support objects with self-references (namely, futures). However, Pin's design is pretty hacky, being implemented purely 'in userspace' without any language support. There's widespread desire to add some form of language support for pinning someday, if only to make the ergonomics a bit nicer (e.g. not needing a method call to reborrow mutable pinned references). This would probably also be needed in order for the compiler to support self-referential structs. From I've seen there are two quite different proposals for how to add language support for pinning. One is to add sugar to `Pin` while keeping the design mostly the same. For example, `Pin<&mut T>` could turn into `&pin mut T` or something. The other is to essentially throw away `Pin` and replace it with a `Move` auto trait: instead of `Pin<&mut T>` you would just have `&mut T` where `T: !Move`. A `Move` trait would be quite disruptive, but would also have many benefits and ultimately simplify the language. For example, with `Pin`, every type has two possible kinds of mutable references to it: `Pin<&mut T>` and `&mut T`. In the case of self-referential structs, how would a non-pinned reference to one work? And how would that interact with `Drop`, which always takes `&mut self`? You could answer these questions one way or another, just like they're answered for futures (although futures have it easier because they always start in a non-self-referential state). But it all becomes simpler if you take the `Move` approach, where `&mut T` is pinned if `T` is `!Move`, and there is no second reference type to worry about.
    - armchairhacker 3y ago
      Rust already has the `Unpin` trait, which is identical to `Move` except that it doesn’t pin `&mut T`. I’ve never understood why Rust doesn’t require the `Unpin` trait in `mem::swap` and derivatives instead of having `Pin`. I also don’t really know of any other methods / cases besides those using `mem::swap` where `Pin` and `Unpin` are relevant. It works for `swap` because if you have a self-referential data structure, there’s never a good reason you want to tangle the references. Lastly, normally a mutable reference guarantees there aren’t other references, but a mutable reference to a self-referential data-structure doesn’t have this guarantee because it references itself. Is this why `Pin<&mut T>` is necessary, because a real mutable reference to a set-referential data-structure violates Rust’s not-fully-defined “borrowing rules”? And do we want to keep that specific rule (since we’re mutable borrowing the entire region including the self-referential borrow, and self-referential structures are already unsafe, I doubt it would cause any issues)? Maybe `&pin mut T` should be added, but as its own kind of reference instead of syntax sugar…
- Arnavion 3y ago
  Self-referential structs work fine in Rust and always have. https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=4ce733c6c52318c2105d20400575fe2f https://play.rust-lang.org/?version=stable&mode=debug&editio... The compiler will correctly prevent you from moving the value. (Try adding `drop(foo);` at the end.) The other way to have a struct that requires mutation to move in and out of self-referential-ness (as `async {}` needs, for example) can be achieved with `unsafe` and `Pin::new_unchecked`.
  - orlp 3y ago
    > Self-referential structs work fine in Rust and always have. They 'work fine' in that the Rust compiler will let you create one and then never let you move it or access the whole object mutably again while it is self-referential. They don't 'work fine' as in being first-class objects in the language that work as any other object. For what it's worth, I think Rust made the right choice in not allowing custom move constructors that would enable first-class self-referential types.
    - trws 3y ago
      I agree they made the right decision not to add custom move constructors, potentially throwing move constructors and unmovable types make life very, very difficult for certain things in c++. That said, an annotation on a field that it either always references part of the parent or may reference part of the parent (or better a defined range or expression from the parent) could provide enough information for an automatically generated move constructor to correctly update the parent referencing pointer or reference. It’s entirely machinable, we had to do a version of this for the OpenMP offload memory model. It’s not the simplest thing, but updating references based on known offsets from known bases isn’t itself all that hard. Getting the syntax for the user to provide enough information to do it to be ergonomic however, that’s hard.
comex 3y ago
Yes. From the post: > In follow-up posts I’ll dig into how we can use this to support interior references and other advanced borrowing patterns. [..] > But also [current] Rust is not able to express some important patterns, most notably interior references, where one field of a struct refers to data owned by another field.

orlp 3y ago

No, self-referential structs are disallowed by the nature of Rust having move semantics by default, without a concept of a 'move constructor'.

slashdev 3y ago
It’s not clear to me that the compiler couldn’t detect when you try to move a self referential string and prevent it. It basically does if you use pin.
- orlp 3y ago
  I'm not saying a theoretical language couldn't be made that allows this, but in Rust it would be breaking backwards compatibility. For example [T]::sort (obviously) needs to be able to move the elements in the passed array, but there is no Move bound on T or something similar. > It basically does if you use pin. And we have Pin today, but I assumed the OP meant 'self-referential struct' as in, without using Pin.
  - nu11ptr 3y ago
    > And we have Pin today, but I assumed the OP meant 'self-referential struct' as in, without using Pin. I did, but with Pin would be fine as well
  - devit 3y ago
    It would have to be introduced by making it default, so all existing generic items would have Move bound on all generic parameters. Only in a new edition the default can then be swapped.
- comex 3y ago
  Right. Pin is the canonical way to have objects that can't be moved, and it was invented in order to support objects with self-references (namely, futures). However, Pin's design is pretty hacky, being implemented purely 'in userspace' without any language support. There's widespread desire to add some form of language support for pinning someday, if only to make the ergonomics a bit nicer (e.g. not needing a method call to reborrow mutable pinned references). This would probably also be needed in order for the compiler to support self-referential structs. From I've seen there are two quite different proposals for how to add language support for pinning. One is to add sugar to `Pin` while keeping the design mostly the same. For example, `Pin<&mut T>` could turn into `&pin mut T` or something. The other is to essentially throw away `Pin` and replace it with a `Move` auto trait: instead of `Pin<&mut T>` you would just have `&mut T` where `T: !Move`. A `Move` trait would be quite disruptive, but would also have many benefits and ultimately simplify the language. For example, with `Pin`, every type has two possible kinds of mutable references to it: `Pin<&mut T>` and `&mut T`. In the case of self-referential structs, how would a non-pinned reference to one work? And how would that interact with `Drop`, which always takes `&mut self`? You could answer these questions one way or another, just like they're answered for futures (although futures have it easier because they always start in a non-self-referential state). But it all becomes simpler if you take the `Move` approach, where `&mut T` is pinned if `T` is `!Move`, and there is no second reference type to worry about.
  - armchairhacker 3y ago
    Rust already has the `Unpin` trait, which is identical to `Move` except that it doesn’t pin `&mut T`. I’ve never understood why Rust doesn’t require the `Unpin` trait in `mem::swap` and derivatives instead of having `Pin`. I also don’t really know of any other methods / cases besides those using `mem::swap` where `Pin` and `Unpin` are relevant. It works for `swap` because if you have a self-referential data structure, there’s never a good reason you want to tangle the references. Lastly, normally a mutable reference guarantees there aren’t other references, but a mutable reference to a self-referential data-structure doesn’t have this guarantee because it references itself. Is this why `Pin<&mut T>` is necessary, because a real mutable reference to a set-referential data-structure violates Rust’s not-fully-defined “borrowing rules”? And do we want to keep that specific rule (since we’re mutable borrowing the entire region including the self-referential borrow, and self-referential structures are already unsafe, I doubt it would cause any issues)? Maybe `&pin mut T` should be added, but as its own kind of reference instead of syntax sugar…
Arnavion 3y ago
Self-referential structs work fine in Rust and always have. https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=4ce733c6c52318c2105d20400575fe2f https://play.rust-lang.org/?version=stable&mode=debug&editio... The compiler will correctly prevent you from moving the value. (Try adding `drop(foo);` at the end.) The other way to have a struct that requires mutation to move in and out of self-referential-ness (as `async {}` needs, for example) can be achieved with `unsafe` and `Pin::new_unchecked`.
- orlp 3y ago
  > Self-referential structs work fine in Rust and always have. They 'work fine' in that the Rust compiler will let you create one and then never let you move it or access the whole object mutably again while it is self-referential. They don't 'work fine' as in being first-class objects in the language that work as any other object. For what it's worth, I think Rust made the right choice in not allowing custom move constructors that would enable first-class self-referential types.
  - trws 3y ago
    I agree they made the right decision not to add custom move constructors, potentially throwing move constructors and unmovable types make life very, very difficult for certain things in c++. That said, an annotation on a field that it either always references part of the parent or may reference part of the parent (or better a defined range or expression from the parent) could provide enough information for an automatically generated move constructor to correctly update the parent referencing pointer or reference. It’s entirely machinable, we had to do a version of this for the OpenMP offload memory model. It’s not the simplest thing, but updating references based on known offsets from known bases isn’t itself all that hard. Getting the syntax for the user to provide enough information to do it to be ergonomic however, that’s hard.

comex 3y ago

Yes. From the post: > In follow-up posts I’ll dig into how we can use this to support interior references and other advanced borrowing patterns. [..] > But also [current] Rust is not able to express some important patterns, most notably interior references, where one field of a struct refers to data owned by another field.