6 ms·
An Overview of Memory Management in Rust
- untothebreach 14y agoThis is very helpful to me, having not come from the c++ world and not really having a good grasp of the whole "smart pointer" stuff. Thanks!
- lmkg 14y agoSo this article mentioned that there are other types of smart pointers, which are in the standard library instead of part of the language. How do those work? I would imagine that they're generics, like how C++ smart pointers are templates, but I can't figure out how you would go about writing something like that if your existing primitives are also smart pointers. Does Rust also support C-style pointers, and if so, does it allow you to restrict their usage?
- pcwalton 14y agoYes, they're generics implemented with C-style pointers under the hood. The use of raw C-style pointers is restricted to designated "unsafe" blocks.
- haberman 14y agoAre you saying that these smart pointers (like the return of the allocation operator) are implemented in Rust itself?! If so that is extremely cool, but then how do they implement the semantics that sure references can't outlive their referent? It seems like the smart pointer would have to hook into the program analysis.
- pcwalton 14y agoThe built-in pointers with special names call out to functions written in Rust to perform the actual allocation (specifically, the functions annotated with lang="malloc"/"free"/"exchange_malloc"/"exchange_free"), so you can customize their behavior in that way. Right now we just call through to the C allocator, but there is a WIP GC written in Rust that does some extra bookkeeping in the allocation function and performs the mark and sweep periodically. But the compiler does know about the built-in smart pointers' semantics in order to perform the reference analysis. For custom smart pointers, you usually have to delimit the scope of a reference yourself with a block. For example: let p = ARC(1024); // atomically reference counted do p.ref |r: &int| { ... do stuff with r ... // The compiler ensures that r cannot leave this // scope... } So custom smart pointers cause some rightward drift. Figuring out how to remove this rightward drift is something I would like to solve in the future, but it'll require some more thought. In any case, it's a case of code being a bit uglier, not a case of loss of flexibility, so building a couple of smart pointers into the language seemed like an acceptable worse-is-better solution for Rust 1.0.
- lobster_johnson 14y agoAre you sure that ARC is an appropriate name? Apple already uses the acronym for Automatic Reference Counting. Correct me if I'm wrong, but Rust's reference counting does not look automatic, since you must wrap it in "do" blocks. It's a bit misleading to reuse a fairly well-esablished acronym for something that is nearly, but not quite, the same.
- asb 14y agoYeah, the rust devs have said a number of times they're going to change the name due to this confusion.
- fzzzy 14y agoThe rust ARC stands for Atomic Reference Counting, not Automatic Reference Counting. An unfortunate namespace collision.
- lobster_johnson 14y agoYes. That was my point (atomic vs. automatic).
- pcwalton 14y agoYeah, everyone gets confused here because of Obj-C. We should change the name.
- saidajigumi 14y agoAs someone with no Rust experience but tons of C and C++ experience, this article's a nice synopsis of Rust's memory mangagement. The patterns will be familiar to C++ developers but it's a refreshing take to see these baked into the language. I recommend reading through to punchline at the end of the "References" section. That's a nice example of how elevating these memory management patterns to a first-class language constructs results in a design win over C and C++.
- bcoates 14y agoHow does Rust resolve move semantics for ~-values at compile time? What does this do: let a: ~Point = ~Point { x: 10, y: 20 }; let b: ~Point = ~Point { x: 20, y: 40 }; if (foo()) { b = a; } println(a.x.to_str());
- pcwalton 14y agoIt's a conservative analysis, so your program would be in error as a possible attempt to use a moved value.
- bcoates 14y agoOuch. Can it handle things like conditionally swapping two values or does any conditional assignment break both values? Is there an escape hatch?
- pcwalton 14y agoThere is a swap operator built into the language, so that specific example would work (I think; I haven't checked the code). The escape hatch is to use the generic "Cell<T>" type in the standard library, which gives you dynamic behavior much like C++11 move semantics. The nice thing about the static analysis, though, is that the checks don't have to be performed at runtime if the compiler can prove that the value is always moved at the end of the block.
- lifthrasiir 14y agoYour example does not quite compile (you need to declare `a` and `b` as mutable, `println` is actually `io::println`, `fn main() { ... }` is missing etc.), but given similar code: fn main() { let mut a = ~1; let mut b = ~2; if false { b = a; } io::println(fmt!("%d", *a)); } ...the compiler will assume that `a` can be moved to `b` (although the precise analysis will reveal it can't) and therefore further use of `a` is unsafe. If you really want to assign `a` to `b`, you can do `b = copy a;` (warning: the syntax may change) to explicitly copy it without moving.
- 14y ago
- te_chris 14y agoAs someone whose only real exposure to C level programming has been iOS and Obj-C, the way I read this it works similar to ARC, am I correct in that conclusion?
- lifthrasiir 14y agoNot exactly, unique smart pointer (~-ptr) does not need the reference counting at all. It is comparable to a raw pointer with much stronger compile time check. In fact, I think using unique smart pointers does not require the runtime at all (I'm not sure as I'm still scratching the surface of rustc).
- pcwalton 14y agoThey don't require the runtime (the "embedding Rust in Ruby" post, for example, uses them), but they do require some implementation of malloc and free.
- thebear 14y agoOne issue with languages that use smart pointers (as opposed to a global garbage collector) for memory management is circular references. For example, one can create a memory leak with C++ shared pointers by having two objects each of which holds a shared pointer to the other. In C++, the solution is to use a weak pointer for one of the two pointers. This raises two questions: 1) Since the Rust compiler is more vigilant than the C++ compiler, does it do anything to prevent "pointer circles?" 2) Am I correct in assuming that the solution in Rust is to use a reference for one of the owning pointers? In other words, is the Rust reference the analogue to the C++ weak pointer?
- pcwalton 14y agoYou can use Rust references that way to some extent, although the safety check for references will probably prevent you from using it to do what you want in all cases. In general, you need to follow a stack discipline with your references. For example, you can't make a doubly linked list using unique smart pointers and references as the backpointers; the Rust compiler will be unable to prove it safe. (In general, while placing references inside data structures can be done, it requires some knowledge of the how the safety check works—this is where the "lifetime parameters" come in.) The good news is that the Rust compiler does detect and clean up cycles of `@` smart pointers. At the moment, this is done at thread death only. But Graydon is nearly done with a new tracing garbage collector (written in Rust) that correctly detects and cleans up cycles.
- thebear 14y agoThanks for the explanation, this really piques my interest in Rust. BTW, now I have a follow-up to my question whether Rust's references are the analogue to C++ weak pointers. C++ weak pointers can be dangling, but in a controlled manner, that is, one can ask them whether they're dangling or not. Am I correct in assuming that Rust will never allow anything like a dangling pointer, not even the "controlled dangling" of a C++ weak pointer?
- pcwalton 14y agoRight, Rust references are never dangling. But there might end up being true weak references in the standard library, which would be more like a C++ weak pointer. (I'd take a patch for them if anyone needed them, though it's blocked on the GC landing.)
- minamea 14y agoNot being a picky prick this is a genuine question: What's the point of smart pointers? Seems to me that the #1 reason you would use a pointer is that you're allocating memory on the heap and you don't want the memory deallocated when the pointer goes out of scope. Seems to me that the only difference between a smart pointer and declaring a variable is data on the heap vs data on the stack and there's not much difference there?
- tdg 14y agoSimple. You use unique smart pointers if you do not know the size of your data at compile time.
- steveklabnik 14y agoRust tasks cannot share anything. Smart pointers go on the 'exchange stack', which is technically shared by tasks, but since there's only one reference to anything in the stack, it's okay. So you often create some sort of communications structure, keep one end yourself, and hand the other end off to a task. The smart-ness keeps this memory from being copied, which would be slow, but keeps you safe at the same time. Like this: use task::spawn; use pipes::stream; use pipes::Port; use pipes::Chan; fn main() { let (port, chan): (Port<int>, Chan<int>) = stream(); do spawn |move chan| { chan.send(10); } io::println(int::str(port.recv())); } More: http://www.rustforrubyists.com/book/chapter-07.html http://www.rustforrubyists.com/book/chapter-07.html and http://www.rustforrubyists.com/book/chapter-08.html http://www.rustforrubyists.com/book/chapter-08.html (I use the older-as-of-yesterday term 'owned pointer' rather than 'smart pointer.')
- dbaupp 14y agoA ~ pointer can still be returned from a function without being deallocated, and the contained data doesn't have to be copied (unlike data on the stack).
- minamea 14y agoReference please? I think this is wrong.
- drivebyacct2 14y agopcwalton, I don't know if this was in response to my begging in /r/rust and mozilla/#rust but this and the previous post (http://pcwalton.github.com/blog/2013/03/09/which-pointer-should-i-use/ http://pcwalton.github.com/blog/2013/03/09/which-pointer-sho...) were exceptionally helpful. I'm still sitting on my hands waiting for a better net/http equivalent library, but I've been having fun dabbling in Rust. Thanks.
- steveklabnik 14y ago> I'm still sitting on my hands waiting for a better net/http equivalent library, I'd like to write one, but the whole IO part of the standard library is changing quickly, so I'm waiting till that is through. I have a Rust buildpack sorta-kinda working almost....
- hobbyist 14y agoIf Rust doesn't allow pointer arithmetic and conjuring up a pointer using '&' like in C, it doesn't make me feel there is anything special here.
- pcwalton 14y agoIf you have a slice of a vector (`&[T]`), then you can also move it forward, like pointer arithmetic in C except checked to ensure that you don't overflow the bounds of the vector. Of course, if you have an unsafe pointer, then you can perform raw pointer arithmetic on it as well. You can create references using `&`, although I didn't show it in this overview.
- pjmlp 14y agoPointer arithmetic should be confined to unsafe/system blocks as it only makes sense when writing low level code like device drivers.
- Executor 14y agoWhy do new languages like Rust, Go look so funky? Can't they make a language that is familiar? i.e.: - type comes after variables - for list.each |dog| <-- wtf, weird syntax?