4 ms·
> In Rust, this destructor is perfectly fine: Sort of; it's very rare. You don't really write free_memory, unless you're doing some very specific unsafe things
by steveklabnik 7y ago
> In Rust, this destructor is perfectly fine:
Sort of; it's very rare. You don't really write free_memory, unless you're doing some very specific unsafe things.
> When that happens, `self.free_memory` will never be called, and memory will be leaked if `panic=unwind`, which is the default behavior.
It will be leaked if panic=unwind and if you use catch_panic. catch_panic is not used very much. Partially because panic=abort is also a valid semantic, and partially because if this Drop impl is called when panic happens, regardless of panic settings, you'll get an abort if self.cleanup() panics.
> Rust supports fallible destructors,
This isn't really true. Fallability in Rust is spelled "Result<T, E>", and drop does not return one. You're talking about a non-recoverable error, which, it is true, Rust does not give you tools to prevent, really. That's because it's very, very rarely used, because the whole intention is to end the current thread of execution.
> At least, I couldn't find any examples in the Rust book / reference / rust-by-example that show in which situations unwinding from a destructor is a good thing to do.
There isn't, which is why it isn't done, which is why your point confused me :)
- jynelson 7y agoWell, rarely done isn't the same as never done. docs.rs has exactly the tradeoffs the parent comment is talking about: we have a long-running daemon thread that uses `catch_unwind` and builder threads that occasionally panic. We've had [issues with memory leaks](https://github.com/rust-lang/docs.rs/issues/656 https://github.com/rust-lang/docs.rs/issues/656) in the past - they weren't related to unwinding that I know of, but it's still possible that they were. However I'm not in favor of the proposed solution - if the docs.rs server aborted every time a thread panicked we would have a lot of outages!
- steveklabnik 7y ago> Well, rarely done isn't the same as never done. Absolutely. But I do think the difference between "idiomatic" and "rarely done" is valuable, here. And as you said yourself, it's not clear that these leaks are caused by this kind of thing. If someone saw Rust code leaking memory all the time due to catching panics, I'd want to know about it, because it's very contradictory to my own experience, and I think that's interesting.
- cogman10 7y agoCorrect me if I'm wrong, but panics themselves are fairly rare. That's not the case in C++ where "throw" is a keyword you are expected to use for error handling. If you take this to a Java example, a panic is more like an "Error" and less like an "Exception". That is to say, in rust, if something panics it is a sign of a major bug that shouldn't have any option for recovery. With all that said, the same concept exists in C++. Try doing a 1/0 in C++ and see what happens. You don't get some nice exception to catch and you can't add a "noexcept" clause to stop it from happening. In the same original example, if you have that 1/0 error in the underlying method and you handle it instead of crashing by tying into the OS specific "div by zero" garbage. You to can create a memory leak in code thought to be safe. It goes to show that you can't (and shouldn't) expect to recover from everything. Crashing, IMO, is usually far safer than trying to fix things up and move forward.
- steveklabnik 7y agoConceptually, panics should be rare, because one firing means that some sort of unexpected problem has occurred. However, the real world is not "conceptually." I don't think there's any real data about how often they happen, but at least my experience is that tools I write in Rust rarely end up showing me panic output. > That's not the case in C++ where "throw" is a keyword you are expected to use for error handling. Yes, that's correct. The intended semantics of the two features are very, very different.
- matklad 7y agoA fun example here is rust-analyzer: we implement cancellation via unwinding. This is not technically a panic, but the mechanism is the same, and it more or less is invoked every time a user types something in a file.
- fluffything 7y agoWhere and why does rust-analyzer unwind from destructors ?
- richardwhiuk 7y agoThe solution for docs.rs is handling errors correctly and not panicing.
- jynelson 7y agoI was going to talk about some panics we've had recently, but those will hopefully be fixed soon so I don't think that quite fits my message. Instead I want to talk about why 'handling errors correctly and not panicking' wouldn't work in general. In order for that work, we'd have to have 0 panics - not just few, but none. That requires none of our code to panic, none of our dependencies to panic, and none of our uses of the standard library to panic. If you gave me a limited time frame to run the server - say a week - I think it's possible to make docs.rs that robust. However, for a server that's meant to run 24/7 for weeks on end, I just don't think that's realistic. What `catch_unwind` lets us do is localize panics to a single web request or crate build instead of it affecting the whole server. Of course we don't want to have 500s for any user, but we _especially_ don't want the whole server to be down until a team member has time to ssh in and restart it manually. Of course, after that there's whole question of whether making the server that robust is a good use of time in the first place. Is it better to fix a few 500s every week or to fix long-standing bugs? I don't think it's clear that the 500s are more important if they don't affect many users.
- nine_k 7y agoLook how Erlang handles this. Crashing on error is the encouraged policy. A managing process will notice a crashed process and restart it. Basically crashing is the safe way to release all resources in a problematic situation, at the cost of terminating the process. It's easy when you don't have shared resources at all (Erlang's case), and harder with threads: one threads crashes and frees its resources, another tries to use a shared resource already deallocated. If this can be avoided, life becomes vastly easier, at the cost of higher resource consumption.
- jynelson 7y agoI think we're saying the same thing from two different perspectives :) The 'managing process' is the daemon thread. The 'crashed process' is a worker thread. There's no need to worry about corrupted memory since all state is shared through the database.
- fluffything 7y ago> However I'm not in favor of the proposed solution - if the docs.rs server aborted every time a thread panicked we would have a lot of outages! Where is the proposed solution? Where and why does docs.rs unwind from destructors?
- jkoudys 7y agoIt seems like they're trying to solve a problem (manual destructor call to free memory) that is important in other languages but not rust on account of borrowing. Even in the author's examples I feel like it's a problem directly solved by lifetimes, since all the reasoning about freeing that memory is already there. Interestingly they picked the best possible case to showcase rust's strengths, where the free happens on drop. If it were a case where you needed strict control over where that memory came from and _when_ it gets freed, C would seem like a nice fit, but rust is great when you only care that the memory is gone once your variable's out of scope.
- fluffything 7y ago> It will be leaked if panic=unwind and if you use catch_panic. This isn't fully accurate. It is leaked if you use `panic=unwind` and the panic unwinds the function. At that point, the program is still running, but the memory is unreachable through a pointer in the program, and therefore leaked. Whether you actually catch the panic afterwards doesn't matter. Not catching it will terminate the program, but then the program will be terminated with "leaked" memory, and tools like valgrind will report it as such.
- steveklabnik 7y ago(This comment is basically a duplicate of one of your other ones)
- fluffything 7y ago> Fallability in Rust is spelled "Result<T, E>", and drop does not return one. This isn't really true. All Rust functions that can panic are fallible, independently of whether their return type is `Result` or not. There are idioms to use `Result` for recoverable failures and panics for "harder-to-recover" failures, but that's about it, and people do use `catch_unwind` on `main` to make their web-servers live forever, and they are doing it right, IMO (this is one of the many justified usecases of `catch_unwind`). If the errors would actually be non-recoverable, `catch_unwind` wouldn't need to exist. Also, even if you think as panics as "errors that will kill this thread", your program has multiple threads, and resources leaked by one thread continues to be leaked (e.g. memory is not reclaimed by the OS until the whole process exits), so you still have a leak.
- steveklabnik 7y ago> This isn't really true. All Rust functions that can panic are fallible, Yes, in the abstract sense of fallible, but not in the terminology of Rust, or how the features are used generally. Defaults and language matter, and we've seen the actual usage follow. I don't ever remember seeing a crate suggest catch_unwind for error handling. > our program has multiple threads, and resources leaked by one thread continues to be leaked Yep, my bad, I simply made a mistake here.
- fluffything 7y ago> There isn't, which is why it isn't done, which is why your point confused me :) This shows why it was a bad decision. It allows doing something in the language (unwinding from destructors), that a lot of code needs to protect against (e.g. all the standard library collections, all collections in general, all types that own a resource, etc.), for absolutely no added value (doing that isn't a useful thing to do). Aborting the process if `Drop::drop` unwinds would have simplified both the language and the programs written in the language without downsides. That doing that was a good idea was known (CERT C++ requires non-throwing destructors), and C++ actually made a backward incompatible change in C++11 to fix that (making destructors `noexcept(true)` by default, e.g., see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3166.html http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n316...).
- steveklabnik 7y ago> for absolutely no added value (doing that isn't a useful thing to do). You yourself said that web servers are a place where this is a good idea. > That doing that was a good idea was known (CERT C++ requires non-throwing destructors), and C++ actually made a backward incompatible change in C++11 to fix that (making destructors `noexcept(true)` by default, e.g., see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n316... http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n316...). Again, throwing in C++ and panicking in Rust are used very differently idiomatically, and in practice, due to the double panic issue, as I mentioned, this is effectively the same for Rust. Yes in theory you can panic in Drop but it will often end in an abort, so doing it for some kind of recoverable problem makes nearly no sense.
- fluffything 7y ago> You yourself said that web servers are a place where this is a good idea. I said that "panicking from Drop::drop is a bad idea, but that panicking is a good idea in general (e.g. in web servers)". We are in agreement that panicking is a good idea in general, but you seem to be turning that around, arguing that "Panicking is a good idea in general, therefore panicking from Drop::drop is a good idea". One does not follow from the other. If you believe that unwinding from Drop::drop is a good idea, enumerate the value this feature adds, its costs, and make a case about why this trade-off is worth it. All languages with unwinding in the same space as Rust do not allow unwinding from destructors, because it adds no value, and adds significant costs. For example, C++ and D do not support it. C++ used to support it, like Rust, but considered the value it added as "negative" (that code deserved to be broken), and changed its semantics to forbid this by default. AFAICT, the same arguments that apply there, apply 1:1 to Rust. In Rust, the cost of this feature is real, e.g., from basic impls like the impl of `Drop` for slices, reused by most collections, to pretty much every Drop impl of every type guarding an important resource in a module that uses unsafe (e.g. the many Iterator drop guards, etc.). So I stand by my original claim: panicking from Drop::drop is a bad idea: there are no worthy use-cases for it, its costs are real, pervasive, and unnecessary, because in practice nobody does this, but every writer of unsafe code must write code to defend from it, which makes trickier code even trickier to write, read, and properly test. C++ is the sane language here. Rust is not.
- CodesInChaos 7y ago> It will be leaked if panic=unwind and if you use catch_panic. An uncaught panic terminates the thread. If that's the main thread the process terminates, but on other threads it continues with leaked memory, even without `catch_unwind`. > catch_panic is not used very much. Are you sure about that? I'd expect most multi-threaded (web)servers to continue after a panic. Otherwise any bug causing a panic (common in my experience) would force a potentially expensive restart of the server.
- steveklabnik 7y ago> If that's the main thread the process terminates, but on other threads it continues with leaked memory Yep, my bad, I simply made a mistake here. > Are you sure about that? I'd expect most multi-threaded (web)servers to continue after a panic. The two main use-cases for panics seem to be: 1. web servers 2. FFI, since panic across the boundary is UB On 1, well, this is actually a contentious point. In general, the web world has moved more and more to more and more disposable web servers. You have to be resilient to something killing your web serer process, so you need the infrastructure here anyway. Servers re-starting isn't very expensive because you have a bunch of them already, and they don't start serving requests until they're up. Of course, "defense in depth" is a good idea, so doing both makes more sense than just one, but you can't get away with just catching panics if you want a robust service. And, a lot of web servers are single threaded these days.... Regardless, while web services are a big market for Rust, they're only one thing that it does, so I still think of this as "not that popular." Maybe that's wrong :)
- CodesInChaos 7y agoA reproducible panic can easily take down your whole fleet of servers, so you'd end up with a one process per request model without planning for this. Plus if a webserver handles more than one request in parallel (affects both multi-threaded and node.js style async designs), you want to finish handling all the other requests before shutting down gracefully. In my experience with C# even exceptions caused by bugs/failed assertions (which would map to panics in rust) almost never cause issues which require a server restart. It's a bit worse in Rust, since a GC collects memory more reliably in case of errors than Rust (but even in Rust leaks should rarely happen on panic)