6 ms·
> Of course they are considered problematic in Rust. > And leaks are hard to code by accident in Rust. > ... I enjoyed this gem and its descendants from the co
by hansvm 2y ago
> Of course they are considered problematic in Rust.
> And leaks are hard to code by accident in Rust.
> ...
I enjoyed this gem and its descendants from the comments. What I see instead, commonly, even in big rust projects, is that it's easy to accidentally define something with a longer lifetime than you intend. Some of that happens accidentally (not recognizing the implications of an extra reference here and there when the language frees things based on static detections of being unused). Much more of it happens because the compiler fights against interesting data structures -- e.g., the pattern of allocating a vec as pseudo-RAM, using indices as pseudo-pointers, and never freeing anything till the container itself is unused.
There's nothing wrong with those techniques per se, but the language tends to paint you into a bit of a corner if you're not very good and very careful, so leaks are a fact of life in basically every major Rust project I've seen not written by somebody like BurntSushi, even when that same sort of project would not have a leak in a GC language.
- MeetingsBrowser 2y ago> not recognizing the implications of an extra reference here and there when the language frees things based on static detections of being unused Maybe I’m misunderstanding, but isn’t the point of the borrow checker to throw errors if a reference “outlives” the memory it references? How can an extra reference extend the lifetime?
- hansvm 2y agoRust (most of the time, I'm not arguing about failure modes at all right now, let's pretend it's perfect) drops items when you're done with them. If you use the item longer, you have a longer implicit lifetime. The memory it references will also be claimed for longer (the reference does not outlive its corresponding memory). You only fix that by explicitly considering lifetimes as you write your code -- adding in concrete lifetimes and letting the compiler tell you when you make a mistake (very hard to do as a holistic strategy, so nobody does), or just "git gud" (some people do this, but it takes time, and it's not the norm; you can code at a very high level without developing that particular skill subset, with the nearly inevitable result of "leaky" rust code that's otherwise quite nice).
- worik 2y ago> drops items when you're done with them. That is what GC languages do. Too You can get serious resource leaks in any language if you are careless What Rust guarantees is you cannot derefernce a null pointer
- yawaramin 2y agoSure, but that's not the point of Rust. Many GC'd languages also guarantee that nothing can be null.
- littlestymaar 2y agoI don't know where you got this idea but this is wrong. > you use the item longer, you have a longer implicit lifetime. The memory it references will also be claimed for longer This is only true at the stack frame level, you cannot extend a variable beyond that, and there's no difference with a GC in that particular case (a GC will never remove an element for which there's a variable on the stack pointing towards it) > You only fix that by explicitly considering lifetimes as you write your code Lifetime parameters don't alter the the behavior, and their are either mandatory if the situation is ambiguous or entirely redundant. And it only works at the function boundaries, where lifetime extension cannot happen anyway. (See above) Please stop spreading bullshit criticism that have no grounding in reality (Rust has real defects like everything else, but spreading nonesense isn't OK).
- hansvm 2y agoWho said anything about extending "variables"? If you heap allocate something and store a reference to it, Rust will absolutely keep that object alive till all such references are gone. It's not "bullshit [with] no grounding in reality." That property is, yes, what you would see in a GC language (e.g., Entity Framework back in the day in C# would have a leak if you tried to create short-lived objects from long-lived objects using the built-in DI, since EF had a bug maintaining a strong reference to those objects tied to the lifetime of the outer object). My observation is that people tend to do that much more in Rust than in GC languages because the compiler makes other coding patterns harder.
- jvanderbot 2y agoThe claim was "safe", not GC equivalent or memory minimal. You're right, all things have tradeoffs and borrow checker is no different.
- kortilla 2y agoThe claim was “leaks are hard to code by accident”. I agree with gp that this is false. Preventing leaks is explicitly not a goal of rust and making lifetimes correct often involves giving up and making unnecessary static lifetimes. I see this all the time in async rpc stream stuff.
- lifthrasiir 2y agoI think leaks are indeed harder to code by accident in non-async Rust, which was the original setting Rust was developed on. I wouldn't say it is absolutely hard (that really depends on the architectural complexity), but there seems some truth in that claim.
- haileys 2y agoWhen a type is 'static, it doesn’t mean it’s leaked for the lifetime of the program. It just means it owns everything it needs and doesn’t borrow anything. It will still free everything it owns when it is freed.
- n3t 2y agohttps://doc.rust-lang.org/rust-by-example/scope/lifetime/static_lifetime.html#reference-lifetime https://doc.rust-lang.org/rust-by-example/scope/lifetime/sta... says: > As a reference lifetime 'static indicates that the data pointed to by the reference lives for the remaining lifetime of the running program.
- tczMUFlmoNk 2y agoWhat your link says is correct about `&'static T`, a lifetime on a reference. What GP says is correct about `T: 'static`, a type constraint. They both use the "keyword" 'static, but in different senses.
- mmoskal 2y agoThe vec pattern has some advantages though, in particular you can often get away with 32 bit indices (instead of 64 bit pointers), making it a little more cache-friendly. I did it for regex ASTs that are supposed to be hash-consed, so never need to die until the whole matcher dies [0]. A more contrived example is Earley items in parser that point inside a grammar rule (all rules are in flat vec) and back into previous parser state - so I have 2 u32 offsets. If I had pointers, I would be tempted to have a pointer to grammar rule, index inside of it, and pointer to previous state [1], so probably 3x the space. In both cases, pointers would be easier but slower. Annoying that Rust doesn't really let you make the choice though... [0] https://github.com/microsoft/derivre/blob/main/src/hashcons.rs https://github.com/microsoft/derivre/blob/main/src/hashcons.... [1] https://github.com/guidance-ai/llguidance/blob/main/parser/src/earley/parser.rs https://github.com/guidance-ai/llguidance/blob/main/parser/s...
- hansvm 2y agoBy all means. It's a pattern I use all the time, not just in Rust (often getting away with much less than 32-bit indices). You mentioned this and/or alluded to it, but my core complaints are: 1. You don't have much choice in the matter 2. When implementing that strategy, the default (happy path coding) is that you have almost zero choice in when the underlying memory is reclaimed It's a pattern that doesn't have to be leaky, but especially when you're implementing it in Rust to circumvent borrow-checker limitations, most versions I've seen have been very leaky, not even supporting shrink/resize/reset/... operations to try to at least let the user manually be more careful with leaks.
- baq 2y agoYou should probably use something like https://docs.rs/generational-arena/latest/generational_arena/ https://docs.rs/generational-arena/latest/generational_arena... when you want to do this. I understand the problem isn’t that the tools exist, it’s that there are Rust users who are not aware of the concept and its evolution over the years, which I’d argue is not a uniquely Rust issue.
- humanfromearth9 2y agoWhy doesn't anyone rewrite some of these tools in a language that can be compiled to a native binary by GraalVM and benefit of all Java's security guarantees? Would it be too slow? Don't the advantages outweigh the inconvenients?
- KeplerBoy 2y agoRust is trendy, Oracle is the opposite. Of course this isn't a very nuanced or clever take, but certainly part of the truth
- xedrac 2y agoIt's also better in just about every way I can think of, except maturity.
- dpe82 2y agoYou could do similar with the existing C by compiling it to WASM and then compiling that to machine code.
- nicoburns 2y agoWhy use Java when you can use Rust? In all seriousness, Rust is a joy to work with for these kind of tools which typically don't have complex lifetimes or ownership semantics. On top of that you get better performance and probably smaller binaries. But I would pick Rust over Java for CLI tools just on the strengths of the language itself.
- jeroenhd 2y agoWhy go for Java when you can go for .NET? Or Go? .NET seems to perform on par and seems to produce smaller executable, and Go seems to be faster in general. Personally I don't really care what language common tools are written in (except for programs written in C(++), but I'll gladly use those after they've had a few years to catch most of the inevitable the memory bugs). I think the difference is that there aren't many languages with projects that actually end up writing full suite replacements. There's a lot of unexpected complexity hidden within simple binaries that need to be implemented in a compatible way so scripts don't explode at you, and that's pretty tedious work. I know of projects in Rust and Zig that intend to be fully compatible, but I don't know if any in Java+GraalVM or Go. I wouldn't pick Zig for a distro until the language hits 1.0, though. If these projects do exist, someone could probably compare them all in a compatibility and performance matrix to figure out which distribution is the fastest+smallest, but I suspect Rust may just end up winning in both areas there.
- littlestymaar 2y agoI genuinely don't understand why this is the top comment, as it is almost complete BS and the author confuses the behavior of the borrow checker with the one of a GC (and ironically claim a GC would solve the problem when in fact the problem doesn't exist outside of a GC's world)
- hansvm 2y agoThere's no confusion between what a borrow checker and a GC do. The borrow checker enforces safety by adding constraints to the set of valid programs so that it can statically know where to alloc/dealloc, among other benefits. A GC dynamically figures out what memory to drop. My claim is that those Rust constraints force you to write your code differently than you otherwise would (since unsafe is frowned upon, this looks like hand-rolled pointers using vecs as one option fairly frequently), which is sometimes good, but for more interesting data structures it encourages people to write leaks which wouldn't exist if you were to write that same program without those constraints and thus wouldn't exist in a GC language.
- steveklabnik 2y agoThe borrow checker does not change when things are dropped.
- littlestymaar 2y ago> There's no confusion between what a borrow checker and a GC do There is, with Rust you cannot extend the lifetime of an object by keeping a reference to it, when this is exactly the kind of things that will happen (and cause leaks) with GC. > this looks like hand-rolled pointers using vecs as one option fairly frequently Emphasis mine. Idk where you got the idea that was something frequent, but while it is an option that's on the table in order to manage data with cyclic references and is indeed used by a bunch of crate doing this kind of stuff, it's never something you'd have to use in practice. (I say that as someone who's been writing Rust for ten years in every possible set-up, from embedded to front-end web, and who's been teaching Rust at university).
- mercurial 2y ago> the pattern of allocating a vec as pseudo-RAM, using indices as pseudo-pointers, and never freeing anything till the container itself is unused Are you talking about hand-rolled arena allocation? I don´t see how a GC language would have a different behaviour as long as you also use arena allocation and you keep a reachable reference. > There's nothing wrong with those techniques per se, but the language tends to paint you into a bit of a corner if you're not very good and very careful, so leaks are a fact of life in basically every major Rust project I've seen not written by somebody like BurntSushi If I take 3 random major Rust projects like Serde, Hyper, and Tracing, none of which are written by BurntSushi, your claim is that they all suffer from memory leaks?
- burntsushi 2y agoAnd when you put people on a pedestal, they're guaranteed to let you down. :-) https://github.com/BurntSushi/aho-corasick/commit/474393be8d6be7418699e0a9ef8e00fe2fd9cd75 https://github.com/BurntSushi/aho-corasick/commit/474393be8d... I wouldn't be surprised if that style of leak were more prevalent than one would expect. It's pretty subtle. But that link is the only such instance I'm aware of it happening to such a degree in crates I maintain. Maybe there are other instances. This is why I try to use `Box<[T]>` when possible, because you know that can't have extra capacity. I find the GP's overall argument specious. They lack concrete examples.
- tremon 2y agoCurious why you would call that a memory leak? The memory is still accounted for in the Vec and will get released properly when it deallocates, right? This looks like optimizing memory usage to me, not plugging a leak.
- burntsushi 2y agoHow do you define a leak? If you sit down and really think about it, I think you'll find that a precise definition of a leak is actually somewhat difficult. I am nowhere near the first person to make this observation. I point this out to avoid a big long thread in which we just argue about what the word "leak" means. You could absolutely define "leak" in a way that my example is not a leak. But I prefer a definition in which "leak" does include my example. I do not care to litigate the definition. If you want to recast my example as a "space" leak instead of a "memory" leak, then I don't object, and I don't think it changes the relevance of my example. (Which I think is absolutely consistent with the context of this thread.) In particular, I don't think "memory leak" in this thread is being used in a very precise manner.
- vlmutolo 2y agoI think that regardless of what references you have, Rust frees values at the end of their lexical “scope”. For example, in the linked code below, x is clearly unused past the first line, but its “Drop” implementation executes after the print statement at the end of the function. The takeaway is that if you want a value to drop early, just explicitly `drop` it. The borrow checker will make sure you don't have any dangling references. https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=16929eee379e50e56ae07627fb78fc5e https://play.rust-lang.org/?version=stable&mode=debug&editio... In general, I think "lifetimes" only exist in the context of the borrow checker and have no influence on the semantics of Rust code. The language was designed so that the borrow checker pass could be omitted and everything would compile and run identically.
- steveklabnik 2y agoThis is correct.
- pjmlp 2y agoUnless one is writing something where pauses are a no go, even a tiny µs, I don't see a reason for rushing out to affine type systems, or variations thereof. CLI applications are exactly a case, where it doesn't matter at all, see Inferno and Limbo.
- Fruitmaniac 2y agoMemory leaks are easy to code by accident in Java, so it must be even worse in Rust.