6 ms·
Non-Lexical Lifetimes in Rust
- kibwen 11y agoShort background: in today's Rust, the lifetime of a reference is bound to a lexical scope. Sometimes this forces one to create temporary variables solely to change the lexical scope of a reference, which is an annoyance. Take the following example: let mut nums = Vec::new(); nums.push(nums.len()); The above code would compile if we had non-lexical lifetimes, but today it doesn't. This is because method calls are effectively desugared to something like this: let mut nums = Vec::new(); Vec::push(&mut nums, nums.len()); The arguments are evaluated left-to-right, and the mutable borrow prevents the immutable borrow later. Once non-lexical lifetimes arrives this will be possible thanks to control-flow analysis, but today we have to make a temporary: let mut nums = Vec::new(); let length = nums.len(); nums.push(length); It doesn't involve any new user-facing features to the language and is fully backward-compatible, so it's basically just a straight quality-of-life improvement for Rust programmers.
- untothebreach 11y agoMy understanding was that implementing this is dependent on MIR fully landing, is that correct?
- steveklabnik 11y agoIn an extremely strict sense, no, but in reality yes. MIR makes it so much easier to implement a feature like this that we will wait for it before adding this.
- TylerE 11y agoSo is it likely that this will be a Rust 2.x kind of thing?
- steveklabnik 11y agoNot at all. MIR is not user-facing, it's a refactoring of compiler internals, and it's getting close to being finished. The reason Niko wrote this blog post was to start thinking about the eventual RFC to specify exactly how they should work. There are no current plans for a Rust 2.0.
- kibwen 11y agoNope, nowhere near that far off. MIR will be landing this year, and while we don't have a timeframe for NLL yet it's definitely a priority and it's clear that the devs are giving thought to it, as per this post.
- GolDDranks 11y agoNot at all – 2.0 means a breaking change, but currently the borrow checker is just too restrictive. Loosening it a bit won't be a breaking change, as it allows strictly more code, not less. The landing of MIR is an implementation detail (a quite large at that, but nevertheless), and it shouldn't have any observable effects. (Except for maybe some bugs getting fixed.)
- eridius 11y agoI'm not quite sure how non-lexical lifetimes is supposed to fix this issue. Control-flow analysis doesn't change the fact that the arguments are evaluated in left-to-right order, and it doesn't change the fact that the &mut reference is created prior to the method arguments being evaluated and lives until the method itself is invoked. Control flow analysis will simply confirm the fact that the &mut borrow exists while the arguments are evaluated. What might fix this is a special-case rule where an implicit¹ & or &mut reference as a method receiver is temporarily ignored while the arguments are being evaluated (with the borrow checker double-checking after the arguments are evaluated that the & or &mut reference is still legal, i.e. that the referenced value wasn't moved and, in the case of a &mut reference, that no other borrow exists as a result of evaluating the arguments). This rule, being a special case, doesn't require non-lexical lifetimes. ¹I say implicit because it would be potentially confusing for an explicit reference to be ignored while the arguments are evaluated.
- webkike 11y agoMy guess is that if a mutable reference dominates the lifetimes of all its borrows, then an exception to the rule can be made.
- kibwen 11y agoYou're correct in that I was painting with too broad a stroke by attributing all of NLL to control-flow analysis, I was just trying to be brief. :) Having knowledge of control flow is just one of the approaches that will be used to achieve non-lexical lifetimes, though it is particularly applicable when also talking about MIR given that MIR is structured as a control-flow graph.
- cwzwarich 11y agoThe assumed fix here is to distinguish between a mutable borrow and the reservation of a location to later do a mutable borrow. Locations based on local variables and their fields can be reserved, although a move out of the variable would invalidate a reservation. Other locations (e.g. the result of a function call) can't be reserved and would give an error.
- k__ 11y ago> it's basically just a straight quality-of-life improvement for Rust programmers Nice to read. Are there any other such improvements planned? Doing mostly scripting languages, Rust feels often a bit clunky with solutions like you mentioned or its unwrapping etc.
- steveklabnik 11y agoOne thing that would be helpful for those looking to write scripts is to use crates which provide a higher level API. So for example, the text-io crate wraps up the lower-level standard library interface to make it easier to use, though it gives you much less control over errors. Due to Rust's low-level nature, we've focused on the low-level stuff first: the higher-level APIs can then be built on top. I think this kind of thing would be more helpful to you than actual language improvements in many cases.
- k__ 11y agoYes, I understand that. I just had the hope, that the low level stuff is done with 1.0
- kibwen 11y agoIf you're aware of Rust's `try!()` macro for error handling, its functionality is being elevated into the language via the new `?` operator, which, in addition to being less verbose, will make for better error messages and also allow us to potentially extend the error propagation to types other than `Result` (such as `Option`). This feature is currently in nightly waiting to be stabilized, and the compiler codebase itself has been converted to use it: https://github.com/rust-lang/rust/pull/32390 https://github.com/rust-lang/rust/pull/32390 There's also a huge overhaul to compiler error message formatting that should make error messages friendlier and more informative, and should hopefully be landing in nightly sometime today: https://github.com/rust-lang/rust/pull/32756 https://github.com/rust-lang/rust/pull/32756 . Also, part of the same work that is enabling non-lexical lifetimes (MIR, discussed here last week: https://news.ycombinator.com/item?id=11581681 https://news.ycombinator.com/item?id=11581681 ) is also going to enable incremental recompilation of Rust code, which should have a dramatic effect on turnaround time when developing code, as you'll spend enormously less time waiting for things to compile.
- megrimlock 11y agoThat Niko's text-arrow lifetime annotations are so helpful here suggests a rust IDE could offer similar highlighting or annotations to explain borrow checker errors. One of the ergonomic hurdles to rust adoption by experienced C++ users is that the language looks similar enough that such people may think they fully understand the semantics even before they've fully internalized all the details. It's a tricky zone when something feels familiar enough that you don't realize the remaining mistakes in your conceptualization.
- Manishearth 11y agoThe borrow checker already outputs these arrows. This output is improving, too https://github.com/rust-lang/rust/pull/32756 https://github.com/rust-lang/rust/pull/32756 -- feedback appreciated!
- kibwen 11y agoAt this very moment there is a PR from Niko that is about to land that will overhaul borrow checker error message reporting (and error formatting in general): https://github.com/rust-lang/rust/pull/32756 https://github.com/rust-lang/rust/pull/32756 .
- mtanski 11y agoCan't happen fast enough. For the last two years in a row I try Rust every 6 months to see if it's at a place that I would consider using it. I hit the 3 problems described here every time I try it. You can work around all of them by restructuring your code it's just so tedious. I rather be doing something else versus working around this. The community hasn't really been helpful, the answer is always restructure how write things. I'm writing a column store query engine (with table blocks accessing them by columns) that leads to a lot of borrowing so the advice is not all that helpful. I want to like Rust, but I can't.
- TillE 11y agoI've also found that basic problems like how to implement typical data structures (without unsafe code) are controversial, difficult, or actually impossible. That's really not good. Still optimistic about Rust. But it has some fundamental issues to solve before it's generally usable.
- Gankro 11y agoYou can express data structures safely perfectly fine in Rust as long as you do all the things that make them safe in every other language -- aggressively garbage collect, indirect, and runtime validate. I'm not aware of any language that lets you implement an interesting high performance data structure totally safely. Heck, Rust is impressive because you can encode singly-linked-stack and binary-tree-without-parent-pointers efficiently and safely without garbage collection! You can even build iterators that are statically verified to iterate over every element at most once, and statically guaranteed to not be invalidated. Type systems that are powerful enough to ensure the safety of even moderately complex designs (doubly linked list, tree with parent pointers, singly linked stack, hashmap) appear to be unwieldy and relegated to academia.
- catnaroek 11y ago> even moderately complex designs (doubly linked list, tree with parent pointers, singly linked stack, hashmap) In other words, imperative data structures? My experience with implementing custom data structures in Rust: (0) Purely functional data structures: You don't even need borrows to express them. The main downsides are: poor locality of reference, inability to parameterize the data structure over whether its nodes are uniquely owned or reference counted. HKTs would fix the latter. (1) Imperative data structures: Not even mutable borrows will help you. You just need to drop down to `unsafe` code.
- deleted 11y ago[deleted]
- gsg 11y agoThis reminds me of an irritating problem in CPS based compilers where you can have a transformation that is obvious and legal except that after the transformation one of the relevant variables would not be in scope. The solution(s) are to decline to do that transformation, fart around with sinking and hoisting in order to get the scope right, or ditch scope and move to a graph model in which dependence is represented with explicit edges rather than being approximated by the lexical structure of the program. I'm guessing that the lexical borrow problems that Rust are facing are an instance of the same problem. It will be interesting to read the follow up post and see what relation, if any, the suggested solution has to the graph approaches that I am familiar with.
- cwzwarich 11y agoThe solution is basically the same as the CPS case: switch to dominator-based scoping rather lexical scoping. I wrote an RFC on what would need to be done to the region system for nonlexical borrows a while back: https://github.com/rust-lang/rfcs/pull/396 https://github.com/rust-lang/rfcs/pull/396.
- jeffdavis 11y ago"The lifetime of a value" Surely he means variable, not value, right?
- glass-animal 11y agoI want to like Rust, and in the abstract I do, but every time I dive in the concepts have too much complexity for me to get going, and this post reinforces that. It's not that I couldn't eventually grok Rust, it's that I don't have time to given that it's not part of my day job. I really hope that Rust leads to some of the same features being expressed in a more intuitive way by a subsequent language.
- jeffdavis 11y agoI wonder how much of this is required to use rust effectively. A lot of languages have fairly tricky semantics and type systems, and most developers don't fully understand them. C++ is obviously a big language, but even C has things like volatile pointers and pointer aliasing and numerical type promotion. And if you don't understand non-lexical lifetimes, I think that's OK. You can just be more explicit about the lifetimes and it will still work. And reading someone else's code should still make sense.