10 ms·

Rust Lifetimes

102 points by luu 11y ago

shadowmint 11y ago
cool post. what about traits and 'static? Anyhow; Its definitely worth making two points about lifetimes very clear: 1) They don't exist at runtime. Lifetimes are purely a compile time thing the compiler uses for formal safety. 2) 'a is not a actual lifetime, it is the minimum possible life time valid for that item; ie. a bound, not a concrete value. &'a means; this arg/trait/ref/whatever must live at least as long as 'a. That's why different objects with different lifetimes can coexist in a struct of 'a, without needing 'a, 'b, 'c etc for each reference in the struct.
- dbaupp 11y ago
  It's not clear from context in your comment if you're saying this, but, for `x: &'a T`, 'a is a bound in two ways: - it is the maximum possible lifetime of the x variable itself, - it is the minimum lifetime of the `T` instance x points to. These are 100% related of course (x can't point to something that is invalid), but it is worth being careful about which end of a reference/pointer you're talking about when talking about validity.
  - sixbrx 11y ago
    Thanks for this comment, that clarifies things a lot for me. I think it should be in the docs (with equal brevity) if not already.
    - steveklabnik 11y ago
      I've made a note to check it out, thanks.
jhasse 11y ago
struct Sheep<'c> { age: &'c i32, } I don't get why this is needed at all and isn't implicit? What are other possible values for a lifetime in a struct?
- Ygg2 11y ago
  In theory, Rust definitely could use extra lifetime elision. Question is how often is such elision useful. You could have struct Example<'a, 'b>{ ref1: &'a i32, ref2: &'b i32}
  - groovy2shoes 11y ago
    In this case, how is the lifetime 'a different from the lifetime 'b?
    - Manishearth 11y ago
      More accurately, it allows the lifetimes to _be_ different. Think of it like generics. What's the difference between: struct Foo<T> {x: T, y: T} and struct Bar<T,U> {x: T, y: U} "How's T different from U"? That's not the right question to ask -- over here Bar lets T be different from U, whereas in Foo both x and y must have the same type. With the example above struct Example<'a, 'b>{ ref1: &'a i32, ref2: &'b i32} this means that there are two references of independent lifetimes. It could also be: struct Example<'a>{ ref1: &'a i32, ref2: &'a i32} which means that the lifetimes must match, or even struct Example<'a, 'b: 'a>{ ref1: &'a i32, ref2: &'b i32} which means that 'b is a subset of 'a. (This becomes useful if a struct stores a borrow to a thing and then a borrow of something extracted from that thing) While the first case is the most common (and theoretically could be elided), it's still often a tossup and it's clearer when you know that a struct is borrowing things.
      - groovy2shoes 11y ago
        Thank you. The article made it sound to me like struct Foo<'c>, 'c is the lifetime of the struct. Am I correct in thinking that, in reality, 'c must live as long or outlive the struct?
        dbaupp 11y ago
        Yeah, the lifetimes on a struct are the lifetimes of things that the struct points to, not an instance of the struct itself. For an instance to be valid, everything it points to must be valid, so if `foo: Foo<'c>`, `foo` cannot be stored for longer than `'c`. E.g. this code is invalid, because the `Foo` instance would live longer than the thing it points to: let foo = { let age = 0; Foo { age: &age } }; The `age` variable goes out of scope at the } (i.e. the maximum possible lifetime for &age ends there), and hence letting the `Foo` escape from that scope would be a bad idea. (NB. this applies to the "core" lifetime'd types like `&'c T` too: that is really just a nicer way to write a parameterised type like `Ref<'c, T>`. So the rule "cannot return a reference to a local variable" is the same rule that stops a `Foo<'c>` living too long.)
Manishearth 11y ago
[This comment is outdated, I discussed this with the author and he improved the post :)] I feel this post confuses two different notions of lifetime. And also mixes concrete lifetimes with lifetime parameters. There's the CS notion of liveness, and then there's the lifetime notation in Rust. > To know when to deallocate objects This is handled by the CS notion of liveness. Well, an approximation to it. "Things are live until they are used, or until they can no longer be used". The deallocation comes from Rust's affine type system. In fact, Box<T> and other self-deallocating things don't have an associated "lifetime" in the Rust sense of the word, at least not one that dictates their deallocation. The lifetime syntax is basically a form of generics which lets one put constraints on the allowed liveness of &-pointers and their ilk. > example_function<'a> names the lifetime defined by this function, called 'a. This isn't true. There does exist an anonymous inner lifetime for the scope of the function. It's not `'a` What's happening here is that we're saying, "For any lifetime `'a`, we can have a version of example_function which returns a pointer which can live that long.". This doesn't compile, because the compiler knows that this function can only return pointers which live as long as its scope or less, not "any" lifetime. (Which is impossible to return from anyway). This becomes clearer when you have a function like `fn foo<'a> (x: &'a [u8]) -> &'a u8 { &x[0] }` (FWIW if you omit the lifetimes Rust will elide them and internally produce the same function). Here, what we're saying is, "For any lifetime 'a, foo can take a slice that lives as long as 'a and return a pointer that lives as long as 'a". This compiles, because the compiler can see that a reference to x will live as long as x itself. Similarly, with structs: > Sheep<'c> instead of Sheep > > This is not doing anything concrete. It's just giving a name to the lifetime that Sheep structs can last for. This is wrong, too. It's giving a _parameter_ to Sheep, saying "Every Sheep has an associated lifetime 'c, which is the lifetime of its inner reference". In fact, it is possible to define the body of sheep in such a way that Sheep is guaranteed to live _longer_ than `'c` (instead of shorter, as in the example above). This is explained in the Rustonomicon (http://doc.rust-lang.org/stable/nomicon/subtyping.html http://doc.rust-lang.org/stable/nomicon/subtyping.html). But it's an advanced topic, doesn't crop up often in actual Rust programming.
- dbaupp 11y ago
  > Here, what we're saying is, "For any lifetime 'a, foo can take a slice that lives as long as 'a and return a pointer that lives as long as 'a". Another way to phrase it is: the return value of the function is guaranteed to be valid for at least as long as the argument (but may borrow from it, and hence mutations of the argument need to be outlawed as long as the return value is held).
dave_ops 11y ago
This has been my problems with Rust for the last 10-12 months. Lifetime elision seems thoroughly broken or extremely limited in functionality, and now when I'm writing (sometimes seemingly trivial) solutions in Rust I'm spending at least as much time and mental energy explicitly annotating lifetimes as I would be if I was just managing malloc and free in C. As a result the pain vs. benefit curve doesn't bend nearly as far toward Rust as it theoretically should.
- Manishearth 11y ago
  I'm interested in seeing your use cases where lifetime elision isn't helping. I've found that it's helpful in the vast majority of cases, and when it isn't working (for functions, at least) there's a decision to be made. It also could be a programming style thing; I've seen newcomers from C++ often trying to program in the C++ style and having lifetime troubles because the C++ style isn't really amenable to Rust's model. (I feel this might be the issue since you mentioned "sometimes seemingly trivial" -- almost every time someone has said something like that about Rust it's a matter of a programming pattern not translating directly) Also, really, lifetimes don't do away with the effort required in manual memory management, they just confer it to compile time, so at least you can be sure that your code works and will not break in the future. They're not really a new concept, you think about them anyway in C++, just in a different way.
  - dave_ops 11y ago
    I'm a functional programmer (Lisp, OCaml, Erlang), and I only touch C++ when I have to wrap it in something to interface with a higher-level language. Most of the cases where I run into this problem and end up feeling like I didn't gain much, if anything, from doing a straight C implementation are situations with deeply nested data structures. I get that the thinking is that the benefit is that once I've made the compiler stop complaining, my memory management model should at least be sound and safe, and that is a win. Though immediately following that I start to have dream-like fantasies where this entire static analysis stage is simply bolted onto C instead of being a whole new language. Afterall you can do some pretty impressive stuff with nothing but a pile of preprocessor macros (see Objective C).
aabshar 11y ago
test 2
fooyc 11y ago
Note that the go exemple about returning a pointer to a stack variable is actually valid in Go: variables that escape their scope are allocated on the heap.
- steveklabnik 11y ago
  I _think_ the author was just trying to convey that it's no longer stack allocated in this case. Escape analysis is really nice, but when you need performance, it can get in the way. I've talked to people who ended up having to leave a ton of comments with the equivalent of "dont do this, it messes up escape analysis and performance goes south" all over their codebase.
  - Manishearth 11y ago
    I'm actually considering adding escape analysis to _Rust_ (as a part of clippy, that is) so that we can advise users to convert heap allocated Vec/Boxes to stack allocated thingies.
    - steveklabnik 11y ago
      That would be super cool!
orbitur 11y ago
Can someone explain what happened in the C example?
- steveklabnik 11y ago
  Well, it's not guaranteed to print 3, it can do anything it wants. Since you have a dangling pointer, it's not pointing to valid memory. So it depends.
seivadmas 11y ago
Rust is a unique language in that it deallocates memory on the heap without requiring the writer to call free, while at the same time having no need for a garbage collector. Someone correct me if I'm wrong but surely this is not unique - don't Objective C and Swift have automatic reference counting which accomplishes much the same thing?
- Manishearth 11y ago
  Lots of people consider refcounting as "garbage collection". Not everyone, but many people.
- Nyetan 11y ago
  Intersection ARC, Lifetimes: * Not having to worry about trivial memory/resource allocations ARC - Lifetimes: * Can blow up hilariously with refcount cycles * Incurs a runtime overhead Lifetimes - ARC: * For the subset of allocation patterns it handles, it does so with zero runtime overhead (i.e. invalid programs do not compile, invalid = leaks memory) * Use extends beyond just memory allocation/deallocation (see the concurrency docs [1] for example) (note that Mozilla wrote Rust to help write Servo [2], their prototype highly-parallel browser engine, so this use case was a target from the getgo) * Cannot encode all data structures a GC can (hence why reference counting is part of the stdlib [3]) [1]: https://doc.rust-lang.org/book/concurrency.html https://doc.rust-lang.org/book/concurrency.html [2]: https://github.com/servo/servo https://github.com/servo/servo [3]: https://doc.rust-lang.org/alloc/rc/ https://doc.rust-lang.org/alloc/rc/
  - Rusky 11y ago
    > invalid = leaks memory Not quite true- invalid use of lifetimes typically means use-after-free rather than a memory leak (and if you include the standard library, then you can leak memory in safe Rust).
    - tatterdemalion 11y ago
      Its interesting to me how everyone goes to "memory leaks" as their example of memory unsafety, and then if we're talking about Rust we need to clarify that Rust allows leaks (but it makes them hard to do by accident). I guess mnemonically "memory leak" is easier to bring to mind than "dangling pointer" when talking about memory.

102 points by luu 11y ago

cool post. what about traits and 'static? Anyhow; Its definitely worth making two points about lifetimes very clear: 1) They don't exist at runtime. Lifetimes are purely a compile time thing the compiler uses for formal safety. 2) 'a is not a actual lifetime, it is the minimum possible life time valid for that item; ie. a bound, not a concrete value. &'a means; this arg/trait/ref/whatever must live at least as long as 'a. That's why different objects with different lifetimes can coexist in a struct of 'a, without needing 'a, 'b, 'c etc for each reference in the struct.

dbaupp 11y ago
It's not clear from context in your comment if you're saying this, but, for `x: &'a T`, 'a is a bound in two ways: - it is the maximum possible lifetime of the x variable itself, - it is the minimum lifetime of the `T` instance x points to. These are 100% related of course (x can't point to something that is invalid), but it is worth being careful about which end of a reference/pointer you're talking about when talking about validity.
- sixbrx 11y ago
  Thanks for this comment, that clarifies things a lot for me. I think it should be in the docs (with equal brevity) if not already.
  - steveklabnik 11y ago
    I've made a note to check it out, thanks.

jhasse 11y ago

struct Sheep<'c> { age: &'c i32, } I don't get why this is needed at all and isn't implicit? What are other possible values for a lifetime in a struct?

Ygg2 11y ago
In theory, Rust definitely could use extra lifetime elision. Question is how often is such elision useful. You could have struct Example<'a, 'b>{ ref1: &'a i32, ref2: &'b i32}
- groovy2shoes 11y ago
  In this case, how is the lifetime 'a different from the lifetime 'b?
  - Manishearth 11y ago
    More accurately, it allows the lifetimes to _be_ different. Think of it like generics. What's the difference between: struct Foo<T> {x: T, y: T} and struct Bar<T,U> {x: T, y: U} "How's T different from U"? That's not the right question to ask -- over here Bar lets T be different from U, whereas in Foo both x and y must have the same type. With the example above struct Example<'a, 'b>{ ref1: &'a i32, ref2: &'b i32} this means that there are two references of independent lifetimes. It could also be: struct Example<'a>{ ref1: &'a i32, ref2: &'a i32} which means that the lifetimes must match, or even struct Example<'a, 'b: 'a>{ ref1: &'a i32, ref2: &'b i32} which means that 'b is a subset of 'a. (This becomes useful if a struct stores a borrow to a thing and then a borrow of something extracted from that thing) While the first case is the most common (and theoretically could be elided), it's still often a tossup and it's clearer when you know that a struct is borrowing things.
    - groovy2shoes 11y ago
      Thank you. The article made it sound to me like struct Foo<'c>, 'c is the lifetime of the struct. Am I correct in thinking that, in reality, 'c must live as long or outlive the struct?
      - dbaupp 11y ago
        Yeah, the lifetimes on a struct are the lifetimes of things that the struct points to, not an instance of the struct itself. For an instance to be valid, everything it points to must be valid, so if `foo: Foo<'c>`, `foo` cannot be stored for longer than `'c`. E.g. this code is invalid, because the `Foo` instance would live longer than the thing it points to: let foo = { let age = 0; Foo { age: &age } }; The `age` variable goes out of scope at the } (i.e. the maximum possible lifetime for &age ends there), and hence letting the `Foo` escape from that scope would be a bad idea. (NB. this applies to the "core" lifetime'd types like `&'c T` too: that is really just a nicer way to write a parameterised type like `Ref<'c, T>`. So the rule "cannot return a reference to a local variable" is the same rule that stops a `Foo<'c>` living too long.)

Manishearth 11y ago

[This comment is outdated, I discussed this with the author and he improved the post :)] I feel this post confuses two different notions of lifetime. And also mixes concrete lifetimes with lifetime parameters. There's the CS notion of liveness, and then there's the lifetime notation in Rust. > To know when to deallocate objects This is handled by the CS notion of liveness. Well, an approximation to it. "Things are live until they are used, or until they can no longer be used". The deallocation comes from Rust's affine type system. In fact, Box<T> and other self-deallocating things don't have an associated "lifetime" in the Rust sense of the word, at least not one that dictates their deallocation. The lifetime syntax is basically a form of generics which lets one put constraints on the allowed liveness of &-pointers and their ilk. > example_function<'a> names the lifetime defined by this function, called 'a. This isn't true. There does exist an anonymous inner lifetime for the scope of the function. It's not `'a` What's happening here is that we're saying, "For any lifetime `'a`, we can have a version of example_function which returns a pointer which can live that long.". This doesn't compile, because the compiler knows that this function can only return pointers which live as long as its scope or less, not "any" lifetime. (Which is impossible to return from anyway). This becomes clearer when you have a function like `fn foo<'a> (x: &'a [u8]) -> &'a u8 { &x[0] }` (FWIW if you omit the lifetimes Rust will elide them and internally produce the same function). Here, what we're saying is, "For any lifetime 'a, foo can take a slice that lives as long as 'a and return a pointer that lives as long as 'a". This compiles, because the compiler can see that a reference to x will live as long as x itself. Similarly, with structs: > Sheep<'c> instead of Sheep > > This is not doing anything concrete. It's just giving a name to the lifetime that Sheep structs can last for. This is wrong, too. It's giving a _parameter_ to Sheep, saying "Every Sheep has an associated lifetime 'c, which is the lifetime of its inner reference". In fact, it is possible to define the body of sheep in such a way that Sheep is guaranteed to live _longer_ than `'c` (instead of shorter, as in the example above). This is explained in the Rustonomicon (http://doc.rust-lang.org/stable/nomicon/subtyping.html http://doc.rust-lang.org/stable/nomicon/subtyping.html). But it's an advanced topic, doesn't crop up often in actual Rust programming.

dbaupp 11y ago
> Here, what we're saying is, "For any lifetime 'a, foo can take a slice that lives as long as 'a and return a pointer that lives as long as 'a". Another way to phrase it is: the return value of the function is guaranteed to be valid for at least as long as the argument (but may borrow from it, and hence mutations of the argument need to be outlawed as long as the return value is held).

dave_ops 11y ago

This has been my problems with Rust for the last 10-12 months. Lifetime elision seems thoroughly broken or extremely limited in functionality, and now when I'm writing (sometimes seemingly trivial) solutions in Rust I'm spending at least as much time and mental energy explicitly annotating lifetimes as I would be if I was just managing malloc and free in C. As a result the pain vs. benefit curve doesn't bend nearly as far toward Rust as it theoretically should.

Manishearth 11y ago
I'm interested in seeing your use cases where lifetime elision isn't helping. I've found that it's helpful in the vast majority of cases, and when it isn't working (for functions, at least) there's a decision to be made. It also could be a programming style thing; I've seen newcomers from C++ often trying to program in the C++ style and having lifetime troubles because the C++ style isn't really amenable to Rust's model. (I feel this might be the issue since you mentioned "sometimes seemingly trivial" -- almost every time someone has said something like that about Rust it's a matter of a programming pattern not translating directly) Also, really, lifetimes don't do away with the effort required in manual memory management, they just confer it to compile time, so at least you can be sure that your code works and will not break in the future. They're not really a new concept, you think about them anyway in C++, just in a different way.
- dave_ops 11y ago
  I'm a functional programmer (Lisp, OCaml, Erlang), and I only touch C++ when I have to wrap it in something to interface with a higher-level language. Most of the cases where I run into this problem and end up feeling like I didn't gain much, if anything, from doing a straight C implementation are situations with deeply nested data structures. I get that the thinking is that the benefit is that once I've made the compiler stop complaining, my memory management model should at least be sound and safe, and that is a win. Though immediately following that I start to have dream-like fantasies where this entire static analysis stage is simply bolted onto C instead of being a whole new language. Afterall you can do some pretty impressive stuff with nothing but a pile of preprocessor macros (see Objective C).

aabshar 11y ago

test 2

fooyc 11y ago

Note that the go exemple about returning a pointer to a stack variable is actually valid in Go: variables that escape their scope are allocated on the heap.

steveklabnik 11y ago
I _think_ the author was just trying to convey that it's no longer stack allocated in this case. Escape analysis is really nice, but when you need performance, it can get in the way. I've talked to people who ended up having to leave a ton of comments with the equivalent of "dont do this, it messes up escape analysis and performance goes south" all over their codebase.
- Manishearth 11y ago
  I'm actually considering adding escape analysis to _Rust_ (as a part of clippy, that is) so that we can advise users to convert heap allocated Vec/Boxes to stack allocated thingies.
  - steveklabnik 11y ago
    That would be super cool!

orbitur 11y ago

Can someone explain what happened in the C example?

steveklabnik 11y ago
Well, it's not guaranteed to print 3, it can do anything it wants. Since you have a dangling pointer, it's not pointing to valid memory. So it depends.

seivadmas 11y ago

Rust is a unique language in that it deallocates memory on the heap without requiring the writer to call free, while at the same time having no need for a garbage collector. Someone correct me if I'm wrong but surely this is not unique - don't Objective C and Swift have automatic reference counting which accomplishes much the same thing?

Manishearth 11y ago
Lots of people consider refcounting as "garbage collection". Not everyone, but many people.
Nyetan 11y ago
Intersection ARC, Lifetimes: * Not having to worry about trivial memory/resource allocations ARC - Lifetimes: * Can blow up hilariously with refcount cycles * Incurs a runtime overhead Lifetimes - ARC: * For the subset of allocation patterns it handles, it does so with zero runtime overhead (i.e. invalid programs do not compile, invalid = leaks memory) * Use extends beyond just memory allocation/deallocation (see the concurrency docs [1] for example) (note that Mozilla wrote Rust to help write Servo [2], their prototype highly-parallel browser engine, so this use case was a target from the getgo) * Cannot encode all data structures a GC can (hence why reference counting is part of the stdlib [3]) [1]: https://doc.rust-lang.org/book/concurrency.html https://doc.rust-lang.org/book/concurrency.html [2]: https://github.com/servo/servo https://github.com/servo/servo [3]: https://doc.rust-lang.org/alloc/rc/ https://doc.rust-lang.org/alloc/rc/
- Rusky 11y ago
  > invalid = leaks memory Not quite true- invalid use of lifetimes typically means use-after-free rather than a memory leak (and if you include the standard library, then you can leak memory in safe Rust).
  - tatterdemalion 11y ago
    Its interesting to me how everyone goes to "memory leaks" as their example of memory unsafety, and then if we're talking about Rust we need to clarify that Rust allows leaks (but it makes them hard to do by accident). I guess mnemonically "memory leak" is easier to bring to mind than "dangling pointer" when talking about memory.