10 ms·

Wrong by Default

105 points by nalgeon 4y ago

duped 4y ago
The problem with RAII is that it is semantically coupled to destruction in most (all?) of the languages it is implemented in, which is tied to lexical scope. In GC'd languages where lifetime is not meaningfully coupled to scope, RAII cannot work the way you expect it to. The most trivial way to handle it is to couple to finalizers, which is tricky for two reasons: firstly it is not deterministic when (or if, in case of an exception/fatal error) the finalizer runs, and secondly it means the GC must keep track of user defined finalizers to execute on collection (what if they trigger an error? What if they take a long time? What if they block?). The solution is to find some way to tie RAII to lifetimes, which may require exit analysis and can be extremely difficult, if not impossible, to implement in the language. The "least bad" way to deal with the problem is defer-like semantics. I'm not a huge fan of defer, but it's important to understand why it exists. It's hardly "wrong by default" and more "understandably different."
- kevincox 4y ago
  > which is tied to lexical scope This isn't really true. It is kinda true in C++ but fairly well fixed with moves. In Rust it isn't true at all because moves are the default so any time you return a value or put it in a data structure the lifetime is no longer lexically bound. > It's hardly "wrong by default" and more "understandably different." I think it is both. What I mean by "wrong by default" is that if you write f = open(name) The code is now wrong. This is the default behaviour. To make the code corred you need to add a second bit of code (a close call). Defer is a powerful mechanism to do that, but it is a tool to solve the core wrongness.
  - masklinn 4y ago
    Note that they said the destruction is tied to lexical scope, in general, not necessarily the lexical scope of creation. Their point is mostly that once you "drop" an object in a GC'd language, it will be collected at some later time independent of lexical scope. Which I don't think is much of an issue, technically the language could "drop" (close) such objects lexically and leave memory reclamation for later. Not a huge issue. The bigger issue is that GC-based objects have a strong tendency to share, a lot. That makes RAII way more challenging, when using the objects directly for starters, but even more so when composing RAII objects and non-RAII objects.
    - marcosdumay 4y ago
      On both C++ and Rust the destruction is tied to the value's lifetime. On the two languages that everybody remember when they talk about RAII, the destruction is not tied to lexical scope. (Although, on C++ that requires extra code for your class to behave correctly. The resource managing class is wrong by default, but the code that uses it is not.)
      - tialaramex 4y ago
        In Rust what you wrote is just how it works because it's how move was defined in the language, but I think in C++ at most you could say that this is how people conceptually try to arrange for things to happen, and at that point we're back to "Wrong by Default". All the C++ compiler actually gives you is destruction at end of lexical scope, and then it's your job as programmer to do what people actually wanted instead. Historically what you do in C++ is copy things a lot, or work with pointers. Modern C++ offers a move, but for backwards compatibility reasons it had to preserve that behaviour where the destruction happens precisely when you leave scope, so, move "hollows out" objects, leaving something which can be destroyed later. Thus it's your job, when implementing a C++ type, to ensure that the "real" work happens whenever a programmer would expect, leaving just some shell to be properly "destroyed" later when scope ends. Whereas for Rust you just implement Drop correctly and the compiler arranges for it to happen promptly.
        MauranKilom 4y ago
        > All the C++ compiler actually gives you is destruction at end of lexical scope, and then it's your job as programmer to do what people actually wanted instead. `new` and `delete` are built into the language (and smart pointers into the standard library). The statement that object lifetime is tied to lexical scope is plainly untrue. > Thus it's your job, when implementing a C++ type, to ensure that the "real" work happens whenever a programmer would expect The auto-generated (`= default`) copy and move constructors do the right thing by default if you compose your type from other (correct) types. Yes, in the rare case that you write a type that has to implement RAII, the compiler can do the wrong thing by default ("copy just copies the pointer"), but then you write a wrapper class once and you're done. If that's even necessary at all, since unique_ptr can have custom deleters. (You wouldn't violate Single Responsibility by messing with RAII in a class that also does other things, right?)
- ComputerGuru 4y ago
  This is only technically true, in practice GC languages have workarounds. C# is almost entirely GC but has the IDisposable pattern and using statements to make up for this.
- cryptonector 4y ago
  Also, `defer` is a patch applied to lame languages like C. It's fine as such -- the problem is the language, and that can't be fixed because the installed base is large.
  - thayne 4y ago
    Go and zig have had defer pretty much from the beginning.
- thayne 4y ago
  Even in GC languages, you almost always want to free the resource at the end of a lexical scope, usually the same one where it was acquired. What if instead of requiring special syntax (defer, try-with-reaource, with etc.) for the normal case where you want it freed at the end of scope, you use special syntax to specify it shouldn't be automatically cleaned up, because it is persisted somewhere else?
dataangel 4y ago
I've been so annoyed with people falsely equating defer and RAII that it didn't occur to me this is yet another way it's inferior! The thing that usually comes to mind for me is that defer only works for local scope. RAII is more general because RAII objects can be put inside other objects that have different lifetimes. I can have an OpenFile inside a struct inside a reference counted smart pointer which will make it so that when the last user using that file disconnects the file is automatically closed. Defer can't do that.
- masklinn 4y ago
  Indeed `defer` quickly gets real complicated when you need to return an object but you also may need to clean it up e.g. f, err := os.Open("file.go") if err != nil { return nil, err } data := make([]byte, 100) count, err := f.Read(data) if err != nil { return nil, err } _, err = f.Seek(0, 0) if err != nil { return nil, err } return f, nil You can't just `defer f.Close()` after the first condition (regardless of flushing / write loss issues, we'll assume read-only) because you want to return it. You need to either: * defer a callback and use a flag * or specifically not use defer, remember to close the file on every path, and hope your linter supports that pattern RAII? RAII just does the right hing there (disregarding the same flushing issue on writes). In case of error you drop the file and that closes it. In case of no-error you return the file object, that automatically makes it the responsibility of the caller.
  - sirwhinesalot 4y ago
    Some languages (like D and Zig) distinguish between "error defer" and "success defer", which solves the issue above. Not as automatic as RAII, but better than the single defer (which sucks). EDIT: My personal preference is actually something like Python's "with" or C#'s "using", since they rely on "destructors" rather than explicit calls to close. But they have the exact same issue as simple defer. If you ever need to store or return the thing, you need to handle it manually instead. In my own usage I almost never keep stuff open for long or store it in complex data structures where this is an issue in practice, simple "using" is almost always enough. Rust and C++ need RAII because they also manage memory like any other resource, otherwise it would get incredibly painful. (Zig uses arena and pool allocators to manage memory, if you had to manage all memory with defers I think people would go nuts)
deleted 4y ago
[deleted]
gilbetron 4y ago
It's just as easy to not write a destructor as it is to not write a defer.
- CodeIsTheEnd 4y ago
  But you only need to write the destructor once.
  - gilbetron 4y ago
    Fair point, but on the other hand, the destructor has to work for all contexts. The usual argument is that the creator of the class has the best vision for what should be in the destructor, but I don't agree with that either.
- masklinn 4y ago
  Except the defer stuff is in addition to the "writing the destructor" part. When you have defer, you still need to write code you'll be deferring. That's what the destructor is, no more. `defer` is strictly additional overhead. Furthermore: 1. If you're just composing in destructible objects the compiler does the work for you, if I create a struct which contains a File, i don't have to do anything, rustc will make my struct implicitly Drop and release the `File`. If you do that in Go... you have to remember to write a deferrable method which'll close the underlying file. 2. You're only defining a type once, and you're thinking about its invariants right there, it's a lot easier to remember those invariants when creating the type than every time you're using an instance. Unless you have linear structural subtyping and the compiler forces you to do it. Then it's fine.
david2ndaccount 4y ago
I can’t believe that people still use opening/closing files as examples of RAII when that is one of the cases that RAII can’t handle correctly. A call to close is the last chance the operating system has to signal that an error occurred when writing. You can’t easily signal errors in destructors. In C++, you basically can’t throw an exception and there is no return value from a destructor. Rust suffers the same kind of problem. The only way to properly close a file is to use regular old explicit control flow with if statements. Then you can handle the errors in the normal way.
- richardwhiuk 4y ago
  Handling an error from close is a bad way to handle the error, because the file is already closed, so you are pretty much stuffed. You want to call fsync() instead, and grab the error there, which will prevent you from being able to get an error from close() - see "Dealing with error returns from close()" in https://www.man7.org/linux/man-pages/man2/close.2.html https://www.man7.org/linux/man-pages/man2/close.2.html If you get it on close, you have ~no idea which operation failed. Obviously doing it on the operation allows you to wrap it up in the handling of the operation and allows you to do RAII (or DIRR as it probably should be known). (The EBADF option is nonsense in a properly developed application, and should cause a panic. The EINTR position is just a mess.)
  - richardwhiuk 4y ago
    Of course, other flow control options are available which also prevent this mess - e.g: int failed = with_file("foo", |file|{ write(file, "bar"); }); fn with_file(char* foo, lambda: Fn(File)){ let f = open(foo); if !f { return f; } let err = lamda(f); let cerr = close(f); if !err { return err; } return cerr; }
    - metadat 4y ago
      Is this Rust or a different language?
revskill 4y ago
This concept is almost the same as trx = new DbTransaction(); ... trx.commit() or trx.rollback(); It's easy to be missed, but it's hard to detect without manually remembering where to put it in the correct place.
- deathanatos 4y ago
  It's the same concept. RAII can (and I have seen DB libraries that do) model a DB transaction as a resource. In the cases that I know of, they take the safe default of rolling the transaction back if the code fails to commit it. But the transaction is never leaked nor is the connection left in some unspecified state.
adrianmonk 4y ago
Not the main point here, but I think you could almost fix the defer approach with types. Instead of: File f = File.open(...); defer f; You could have: PleaseDeferMe<File> p = File.open(...); File f = defer p; If you want to actually access that file you've created, your only path is through using defer. One case this doesn't solve is creating an object but never using it. When writing new code, you probably won't make that error, but when ripping out old code you might miss a piece you should have deleted. If the compiler warns about unused variables (for every possible code path), it could catch that too. (This could be generalized a bit. Call it CleanMeUp<T>, have a CleanMeUp.takeResponsibility() method that returns T, and maybe defer isn't the only thing that can call takeResponsibility().)
- shhsshs 4y ago
  Java’s `throws` keyword comes to mind. It forces you to handle exceptions or explicitly declare that you expect the exception to be handled by a method higher up the call stack. Your strongly-typed suggestion seems better. But you’d have to somehow allow for cases where a function opens a file, does something with it, and also expects the file to be closed by a higher function. Perhaps something like: PleaseDeferMe<File> BeginWrite() { PleaseDeferMe<File> p = File.open(...); // unsafe! Make sure to return p to the calling function so this gets called with “defer” at some point File f = unwrapDefer p; f.Write(some header) return p }
  - adrianmonk 4y ago
    Yeah, or I guess you could pass some object in: File BeginWrite(DeferTrustee t) { PleaseDeferMe<File> p = File.open(...); File f = t.defer p; f.Write(some header); return f; } And then the caller would look something like: void WriteWholeFile() { DeferTrustee t = defer; // defers to the end of this function File f = BeginWrite(t); f.Write(the body); } Then you can still defer before anything else and avoid unsafe situations.
ComputerGuru 4y ago
This actually brings me to one of the few (current?) limitations of rust that drive me nuts! In all the other modern languages I work with, my big projects end up with a utility RAII class that wraps a closure to execute when the scope expires. It’s great for cleaning up logical operations as opposed to what is traditionally done by RAII destructors in oop languages. Eg if I’m creating a temporary file or directory in a function to handle some IO logic, I’ll create a RAII wrapper around a closure that deletes that folder when the scope expires - this might be a one-off thing so instead of creating an actual RAII “TemporaryFolder” object that wraps around a path, creates the folder in its constructor, and deletes the folder and its contents in the destructor (and have to reference the inner path instead of the object itself when needing it, etc), I can just stick a closure to delete the folder in my wrapper and call it a day. You can’t really do that in rust because you’d have to give ownership of the file/folder/whatever to the closure - meaning you can’t use it yourself afterwards! Technically the compiler just has to guarantee that the object persists to the end of scope (isn’t destroyed or moved out earlier) and then there would be no borrow conflicts (since it’s only run after everything else has finished and can’t be used for anything else in the RAII wrapper) but it can’t really know that, so you’re disallowed from sharing it between your actual code and the cleanup closure.
- masklinn 4y ago
  > You can’t really do that in rust because you’d have to give ownership of the file/folder/whatever to the closure - meaning you can’t use it yourself afterwards! The way to do this is pretty much what C++ has in unique_ptr(p, deleter): have the utility RAII class own both the object and the closure, dereference to the wrapped object, and on drop pass the wrapped object to the closure. Altho > Eg if I’m creating a temporary file or directory in a function to handle some IO logic, I’ll create a RAII wrapper around a closure that deletes that folder when the scope expires There are crates for lots of those situations. https://crates.io/crates/tempfile https://crates.io/crates/tempfile for instance.
  - ComputerGuru 4y ago
    > have the utility RAII class own both the object and the closure Yeah, but the ergonomics suck. Especially for random cleanup tasks that aren’t necessarily one-object, one-task. The temp file thing was just an example, actual use case was more domain-specific.
the_gipsy 4y ago
> Even simply passing references to multiple places creates ambiguity on who is responsible for cleaning it up. Learning rust, realizing how value ownership (borrow checker stuff) overlaps with conceptual ownership has been enlightening.
car_analogy 4y ago
> But the problem with most garbage collectors is that they only clean up memory! Is this an inherent limitation of garbage collectors, or could they be made to clean up arbitrary resources?
- fancyfredbot 4y ago
  Actually he later goes on to say the garbage collector can clean other resources by using the finalise method. The problem is that it doesn't necessarily clean them up at the right time. The garbage collector isn't triggered by having too many open file descriptors, it runs when you are low on memory. In theory it could look at other resources too, but for certain resources you don't want to wait at all.
  - arcticbull 4y ago
    There's also generally no guarantee that finalizers ever run, period.
    - slaymaker1907 4y ago
      That really depends a lot on the language. You could have a language which guarantees all finalizers are run so long as the process terminates normally (i.e. not by a power outage).
- Sesse__ 4y ago
  It is obvious if you think of what a garbage collector is; it's a simulation of having infinite memory. So objects in a GC world live forever, and thus are never destroyed. (Of course, behind the scenes, the garbage collector will _deallocate_ them, but only when your program can no longer care because it doesn't have any way of reaching them.) Of course, a garbage collector can be made to run finalizers, but then you either have to do a pure refcounting GC, at least for those objects (which is highly suboptimal for speed), or live with unpredictable finalization (which is incompatible with using RAII for e.g. locks, as the article points out).
- pornel 4y ago
anyfoo 4y ago
It depends on what you are working on, and then even "not wrong by default" might be... wrong. For low level code, destructors/RAII are much harder to notice than a very explicit defer. Even the "kernel C" style "goto done;" to a unified exit point (sometimes multiple labels that fall into each other to clean up according to how far you got) wins out by explicitly telling you what will happen. If you are working on projects where you often spend much more time reading code than subsequently writing code, and if the thing you are working on is not a high level application where the meat is in some business logic, then anything that reduces readability in the sense of "I can see everything that is going on" is very bad. In low level code, the potential for icky problems is also much higher: You don't want to find out that some stray destructor did something that should not have happened in e.g. an interrupt context, or while the current thread's memory management is in an intermediate state. That is not correct by default, it is "wrong and you didn't even tell me". In high level application software development, e.g. CRUD SaaS or a smartphone app, the opposite may be the case, and decluttering might well win out.
- kllrnohj 4y ago
  > then anything that reduces readability in the sense of "I can see everything that is going on" is very bad. Nonsense. So called "kernel C" still uses functions, and lots of them, scattered over thousands of files. You never "see everything that is going on." And "kernel C" also uses function pointers and macros, further preventing you from "seeing everything that is going on." Having written and read lots and lots of C++, RAII being "hidden" is hugely overblown by the diehard C crowd. It's really not hidden at all, first of all (it's often right there in the type - pretty hard to forget that unique_ptr is going to call delete, for example). And second it goes so so far in preventing resource leak bugs, which is also something kernels and low level code care a lot about. In fact, they almost always care far more about runtime correctness on this than readibility. The benefits RAII provide here so massively outweigh the "hidden" concerns it's almost laughable. > You don't want to find out that some stray destructor did something that should not have happened in e.g. an interrupt context, or while the current thread's memory management is in an intermediate state. That is not correct by default, it is "wrong and you didn't even tell me". RAII isn't just randomly injected. It's governed by the types you use. If a given type cannot be safely cleaned up in a given context, then it probably also wasn't safe to get in that context in the first place. Which the type should be communicating. So no it's not "wrong by default" since the initial (explicit!) acquisition was the "wrong" part, not the cleanup. Regardless, it's not like C helps here at all. You still have to call the cleanup function for your given data, and that cleanup function may change without your interrupt handler knowing about that change.
nyanpasu64 4y ago
Some people criticize RAII for introducing hidden destructor calls into close-braces. I don't find this a problem in my experience (aside from having to make destructors do nothing in C++'s moved-from objects), but nonetheless I want to try using a language where instead of RAII as an automation system, it serves as a compiler check rejecting code which does not call a destructor exactly-once when needed (which may even return a value, as in linear types or fclose or Vale's Higher RAII). Linear types work better in a language without exception/panic unwinding (but even then you could use the standard destructor in case of crashes), and relegating RAII to purely a static analyzer is conceptually incompatible with having the presence of destructors affect stack unwinding. I'm fine with that since I don't like stack unwinding and don't find it useful in most (not all) cases.
slaymaker1907 4y ago
Eh, I get why people often don't like things like defer, but I like how versatile it is. In general, my ideal language would have both something like defer as well as proper RAII. In addition to being easier to write one-off cleanup statements, defer can be more readable if the cleanup is simple since it doesn't require looking at any data structure's code. Finally, defer can be advantageous in certain scenarios since, at least in Go, it is somewhat dynamically scoped. Therefore, you can defer something in an if statement or in the middle of a loop and the defer will still be executed at the end of the function. This can also be a disadvantage with loops since if you do something like "defer C.free(tempMem)", the memory won't actually be freed until the end of the function.
- boardwaalk 4y ago
  The thing is, you can trivially implement a form of defer with RAII. Where I work we have a decades old C++ macro called ON_BLOCK_EXIT that people use to clean up C-style things that don't have a destructor. Today it's much better with C++ having lambdas. It won't be "function-scoped" like Go, but (IMO) that's fairly useless feature if you have destructors anyways (you could just collect everything in a vector at function-scope, if you really want that behavior).
Ericson2314 4y ago
The best case is linear types + defer, so you can clean up however you want, but if you forget to do so it's an error. (Also, hi! Kevin Cox also does good Nix things, folks. Be like Kevin and do so too.)
- masklinn 4y ago
  defer seems completely unnecessary there. It’s at best a trivial convenience, since the typesystem should flag any path where the resource is not cleaned.
  - Ericson2314 4y ago
    Defer is still nice to avoid verbose boilerplate.
    - masklinn 4y ago
      Defers are verbose boilerplate. Instead you allow relaxing linear types to affine.
      - Ericson2314 4y ago
        You are assuming the thing can drop itself. When that is fine, sure, do RAII. But not all cases can be done that way. For example, the Rust allocators working group has been contorting itself about deallocation. The right thing to do is to not store a pointer to a (de)allocator in e.g. every Box<T, A>, but just pass the (de)allocator when it comes time to deallocate a box. But Rust is completely incapable of expressing this and so we will get some crazy epicycle thing instead, or no good support for custom allocators with non-flat data structures. Even if RAII is the 90% solution I think it is still good to layer the feautres in this way: 1. Linear types. Fundamental building block for that some types one cannot just forget the values. 2. Defer, sugar for scope-based running some code that will eliminate a linear variable. avoiding an error. 3. RAII, a way to fill provide a default defer command if none was manually provided. Default is "type driven" --- we associate a default defer command with the linear type that would be dropping. From a Rust Perspective, we would want to distinguish Clone (an arbitrary convention for cloning) from Copy (and indication we can in fact copy for free. Saying types that need a destructor and types that don't are both affine is like adding some .clone() sugar and erasing the Copy--Clone distinction --- It is erasing a lot of essential nuance/complexity and I don't think a good idea.
Animats 4y ago
There's a lot to be said for the semantics of the Python "with" statement as a way to do scope-based closeout. You get to handle errors at closeout. Nesting works. Nested errors work. Interaction with exceptions works. It's the only construct I've seen which handles the hard error cases.

105 points by nalgeon 4y ago

The problem with RAII is that it is semantically coupled to destruction in most (all?) of the languages it is implemented in, which is tied to lexical scope. In GC'd languages where lifetime is not meaningfully coupled to scope, RAII cannot work the way you expect it to. The most trivial way to handle it is to couple to finalizers, which is tricky for two reasons: firstly it is not deterministic when (or if, in case of an exception/fatal error) the finalizer runs, and secondly it means the GC must keep track of user defined finalizers to execute on collection (what if they trigger an error? What if they take a long time? What if they block?). The solution is to find some way to tie RAII to lifetimes, which may require exit analysis and can be extremely difficult, if not impossible, to implement in the language. The "least bad" way to deal with the problem is defer-like semantics. I'm not a huge fan of defer, but it's important to understand why it exists. It's hardly "wrong by default" and more "understandably different."

kevincox 4y ago
> which is tied to lexical scope This isn't really true. It is kinda true in C++ but fairly well fixed with moves. In Rust it isn't true at all because moves are the default so any time you return a value or put it in a data structure the lifetime is no longer lexically bound. > It's hardly "wrong by default" and more "understandably different." I think it is both. What I mean by "wrong by default" is that if you write f = open(name) The code is now wrong. This is the default behaviour. To make the code corred you need to add a second bit of code (a close call). Defer is a powerful mechanism to do that, but it is a tool to solve the core wrongness.
- masklinn 4y ago
  Note that they said the destruction is tied to lexical scope, in general, not necessarily the lexical scope of creation. Their point is mostly that once you "drop" an object in a GC'd language, it will be collected at some later time independent of lexical scope. Which I don't think is much of an issue, technically the language could "drop" (close) such objects lexically and leave memory reclamation for later. Not a huge issue. The bigger issue is that GC-based objects have a strong tendency to share, a lot. That makes RAII way more challenging, when using the objects directly for starters, but even more so when composing RAII objects and non-RAII objects.
  - marcosdumay 4y ago
    On both C++ and Rust the destruction is tied to the value's lifetime. On the two languages that everybody remember when they talk about RAII, the destruction is not tied to lexical scope. (Although, on C++ that requires extra code for your class to behave correctly. The resource managing class is wrong by default, but the code that uses it is not.)
    - tialaramex 4y ago
      In Rust what you wrote is just how it works because it's how move was defined in the language, but I think in C++ at most you could say that this is how people conceptually try to arrange for things to happen, and at that point we're back to "Wrong by Default". All the C++ compiler actually gives you is destruction at end of lexical scope, and then it's your job as programmer to do what people actually wanted instead. Historically what you do in C++ is copy things a lot, or work with pointers. Modern C++ offers a move, but for backwards compatibility reasons it had to preserve that behaviour where the destruction happens precisely when you leave scope, so, move "hollows out" objects, leaving something which can be destroyed later. Thus it's your job, when implementing a C++ type, to ensure that the "real" work happens whenever a programmer would expect, leaving just some shell to be properly "destroyed" later when scope ends. Whereas for Rust you just implement Drop correctly and the compiler arranges for it to happen promptly.
      - MauranKilom 4y ago
        > All the C++ compiler actually gives you is destruction at end of lexical scope, and then it's your job as programmer to do what people actually wanted instead. `new` and `delete` are built into the language (and smart pointers into the standard library). The statement that object lifetime is tied to lexical scope is plainly untrue. > Thus it's your job, when implementing a C++ type, to ensure that the "real" work happens whenever a programmer would expect The auto-generated (`= default`) copy and move constructors do the right thing by default if you compose your type from other (correct) types. Yes, in the rare case that you write a type that has to implement RAII, the compiler can do the wrong thing by default ("copy just copies the pointer"), but then you write a wrapper class once and you're done. If that's even necessary at all, since unique_ptr can have custom deleters. (You wouldn't violate Single Responsibility by messing with RAII in a class that also does other things, right?)
ComputerGuru 4y ago
This is only technically true, in practice GC languages have workarounds. C# is almost entirely GC but has the IDisposable pattern and using statements to make up for this.
cryptonector 4y ago
Also, `defer` is a patch applied to lame languages like C. It's fine as such -- the problem is the language, and that can't be fixed because the installed base is large.
- thayne 4y ago
  Go and zig have had defer pretty much from the beginning.
thayne 4y ago
Even in GC languages, you almost always want to free the resource at the end of a lexical scope, usually the same one where it was acquired. What if instead of requiring special syntax (defer, try-with-reaource, with etc.) for the normal case where you want it freed at the end of scope, you use special syntax to specify it shouldn't be automatically cleaned up, because it is persisted somewhere else?

dataangel 4y ago

I've been so annoyed with people falsely equating defer and RAII that it didn't occur to me this is yet another way it's inferior! The thing that usually comes to mind for me is that defer only works for local scope. RAII is more general because RAII objects can be put inside other objects that have different lifetimes. I can have an OpenFile inside a struct inside a reference counted smart pointer which will make it so that when the last user using that file disconnects the file is automatically closed. Defer can't do that.

masklinn 4y ago
Indeed `defer` quickly gets real complicated when you need to return an object but you also may need to clean it up e.g. f, err := os.Open("file.go") if err != nil { return nil, err } data := make([]byte, 100) count, err := f.Read(data) if err != nil { return nil, err } _, err = f.Seek(0, 0) if err != nil { return nil, err } return f, nil You can't just `defer f.Close()` after the first condition (regardless of flushing / write loss issues, we'll assume read-only) because you want to return it. You need to either: * defer a callback and use a flag * or specifically not use defer, remember to close the file on every path, and hope your linter supports that pattern RAII? RAII just does the right hing there (disregarding the same flushing issue on writes). In case of error you drop the file and that closes it. In case of no-error you return the file object, that automatically makes it the responsibility of the caller.
- sirwhinesalot 4y ago
  Some languages (like D and Zig) distinguish between "error defer" and "success defer", which solves the issue above. Not as automatic as RAII, but better than the single defer (which sucks). EDIT: My personal preference is actually something like Python's "with" or C#'s "using", since they rely on "destructors" rather than explicit calls to close. But they have the exact same issue as simple defer. If you ever need to store or return the thing, you need to handle it manually instead. In my own usage I almost never keep stuff open for long or store it in complex data structures where this is an issue in practice, simple "using" is almost always enough. Rust and C++ need RAII because they also manage memory like any other resource, otherwise it would get incredibly painful. (Zig uses arena and pool allocators to manage memory, if you had to manage all memory with defers I think people would go nuts)

deleted 4y ago

[deleted]

gilbetron 4y ago

It's just as easy to not write a destructor as it is to not write a defer.

CodeIsTheEnd 4y ago
But you only need to write the destructor once.
- gilbetron 4y ago
  Fair point, but on the other hand, the destructor has to work for all contexts. The usual argument is that the creator of the class has the best vision for what should be in the destructor, but I don't agree with that either.
masklinn 4y ago
Except the defer stuff is in addition to the "writing the destructor" part. When you have defer, you still need to write code you'll be deferring. That's what the destructor is, no more. `defer` is strictly additional overhead. Furthermore: 1. If you're just composing in destructible objects the compiler does the work for you, if I create a struct which contains a File, i don't have to do anything, rustc will make my struct implicitly Drop and release the `File`. If you do that in Go... you have to remember to write a deferrable method which'll close the underlying file. 2. You're only defining a type once, and you're thinking about its invariants right there, it's a lot easier to remember those invariants when creating the type than every time you're using an instance. Unless you have linear structural subtyping and the compiler forces you to do it. Then it's fine.

david2ndaccount 4y ago

I can’t believe that people still use opening/closing files as examples of RAII when that is one of the cases that RAII can’t handle correctly. A call to close is the last chance the operating system has to signal that an error occurred when writing. You can’t easily signal errors in destructors. In C++, you basically can’t throw an exception and there is no return value from a destructor. Rust suffers the same kind of problem. The only way to properly close a file is to use regular old explicit control flow with if statements. Then you can handle the errors in the normal way.

richardwhiuk 4y ago
Handling an error from close is a bad way to handle the error, because the file is already closed, so you are pretty much stuffed. You want to call fsync() instead, and grab the error there, which will prevent you from being able to get an error from close() - see "Dealing with error returns from close()" in https://www.man7.org/linux/man-pages/man2/close.2.html https://www.man7.org/linux/man-pages/man2/close.2.html If you get it on close, you have ~no idea which operation failed. Obviously doing it on the operation allows you to wrap it up in the handling of the operation and allows you to do RAII (or DIRR as it probably should be known). (The EBADF option is nonsense in a properly developed application, and should cause a panic. The EINTR position is just a mess.)
- richardwhiuk 4y ago
  Of course, other flow control options are available which also prevent this mess - e.g: int failed = with_file("foo", |file|{ write(file, "bar"); }); fn with_file(char* foo, lambda: Fn(File)){ let f = open(foo); if !f { return f; } let err = lamda(f); let cerr = close(f); if !err { return err; } return cerr; }
  - metadat 4y ago
    Is this Rust or a different language?

revskill 4y ago

This concept is almost the same as trx = new DbTransaction(); ... trx.commit() or trx.rollback(); It's easy to be missed, but it's hard to detect without manually remembering where to put it in the correct place.

deathanatos 4y ago
It's the same concept. RAII can (and I have seen DB libraries that do) model a DB transaction as a resource. In the cases that I know of, they take the safe default of rolling the transaction back if the code fails to commit it. But the transaction is never leaked nor is the connection left in some unspecified state.

adrianmonk 4y ago

Not the main point here, but I think you could almost fix the defer approach with types. Instead of: File f = File.open(...); defer f; You could have: PleaseDeferMe<File> p = File.open(...); File f = defer p; If you want to actually access that file you've created, your only path is through using defer. One case this doesn't solve is creating an object but never using it. When writing new code, you probably won't make that error, but when ripping out old code you might miss a piece you should have deleted. If the compiler warns about unused variables (for every possible code path), it could catch that too. (This could be generalized a bit. Call it CleanMeUp<T>, have a CleanMeUp.takeResponsibility() method that returns T, and maybe defer isn't the only thing that can call takeResponsibility().)

shhsshs 4y ago
Java’s `throws` keyword comes to mind. It forces you to handle exceptions or explicitly declare that you expect the exception to be handled by a method higher up the call stack. Your strongly-typed suggestion seems better. But you’d have to somehow allow for cases where a function opens a file, does something with it, and also expects the file to be closed by a higher function. Perhaps something like: PleaseDeferMe<File> BeginWrite() { PleaseDeferMe<File> p = File.open(...); // unsafe! Make sure to return p to the calling function so this gets called with “defer” at some point File f = unwrapDefer p; f.Write(some header) return p }
- adrianmonk 4y ago
  Yeah, or I guess you could pass some object in: File BeginWrite(DeferTrustee t) { PleaseDeferMe<File> p = File.open(...); File f = t.defer p; f.Write(some header); return f; } And then the caller would look something like: void WriteWholeFile() { DeferTrustee t = defer; // defers to the end of this function File f = BeginWrite(t); f.Write(the body); } Then you can still defer before anything else and avoid unsafe situations.

ComputerGuru 4y ago

This actually brings me to one of the few (current?) limitations of rust that drive me nuts! In all the other modern languages I work with, my big projects end up with a utility RAII class that wraps a closure to execute when the scope expires. It’s great for cleaning up logical operations as opposed to what is traditionally done by RAII destructors in oop languages. Eg if I’m creating a temporary file or directory in a function to handle some IO logic, I’ll create a RAII wrapper around a closure that deletes that folder when the scope expires - this might be a one-off thing so instead of creating an actual RAII “TemporaryFolder” object that wraps around a path, creates the folder in its constructor, and deletes the folder and its contents in the destructor (and have to reference the inner path instead of the object itself when needing it, etc), I can just stick a closure to delete the folder in my wrapper and call it a day. You can’t really do that in rust because you’d have to give ownership of the file/folder/whatever to the closure - meaning you can’t use it yourself afterwards! Technically the compiler just has to guarantee that the object persists to the end of scope (isn’t destroyed or moved out earlier) and then there would be no borrow conflicts (since it’s only run after everything else has finished and can’t be used for anything else in the RAII wrapper) but it can’t really know that, so you’re disallowed from sharing it between your actual code and the cleanup closure.

masklinn 4y ago
> You can’t really do that in rust because you’d have to give ownership of the file/folder/whatever to the closure - meaning you can’t use it yourself afterwards! The way to do this is pretty much what C++ has in unique_ptr(p, deleter): have the utility RAII class own both the object and the closure, dereference to the wrapped object, and on drop pass the wrapped object to the closure. Altho > Eg if I’m creating a temporary file or directory in a function to handle some IO logic, I’ll create a RAII wrapper around a closure that deletes that folder when the scope expires There are crates for lots of those situations. https://crates.io/crates/tempfile https://crates.io/crates/tempfile for instance.
- ComputerGuru 4y ago
  > have the utility RAII class own both the object and the closure Yeah, but the ergonomics suck. Especially for random cleanup tasks that aren’t necessarily one-object, one-task. The temp file thing was just an example, actual use case was more domain-specific.

the_gipsy 4y ago

> Even simply passing references to multiple places creates ambiguity on who is responsible for cleaning it up. Learning rust, realizing how value ownership (borrow checker stuff) overlaps with conceptual ownership has been enlightening.

car_analogy 4y ago

> But the problem with most garbage collectors is that they only clean up memory! Is this an inherent limitation of garbage collectors, or could they be made to clean up arbitrary resources?

fancyfredbot 4y ago
Actually he later goes on to say the garbage collector can clean other resources by using the finalise method. The problem is that it doesn't necessarily clean them up at the right time. The garbage collector isn't triggered by having too many open file descriptors, it runs when you are low on memory. In theory it could look at other resources too, but for certain resources you don't want to wait at all.
- arcticbull 4y ago
  There's also generally no guarantee that finalizers ever run, period.
  - slaymaker1907 4y ago
    That really depends a lot on the language. You could have a language which guarantees all finalizers are run so long as the process terminates normally (i.e. not by a power outage).
Sesse__ 4y ago
It is obvious if you think of what a garbage collector is; it's a simulation of having infinite memory. So objects in a GC world live forever, and thus are never destroyed. (Of course, behind the scenes, the garbage collector will _deallocate_ them, but only when your program can no longer care because it doesn't have any way of reaching them.) Of course, a garbage collector can be made to run finalizers, but then you either have to do a pure refcounting GC, at least for those objects (which is highly suboptimal for speed), or live with unpredictable finalization (which is incompatible with using RAII for e.g. locks, as the article points out).
pornel 4y ago

anyfoo 4y ago

It depends on what you are working on, and then even "not wrong by default" might be... wrong. For low level code, destructors/RAII are much harder to notice than a very explicit defer. Even the "kernel C" style "goto done;" to a unified exit point (sometimes multiple labels that fall into each other to clean up according to how far you got) wins out by explicitly telling you what will happen. If you are working on projects where you often spend much more time reading code than subsequently writing code, and if the thing you are working on is not a high level application where the meat is in some business logic, then anything that reduces readability in the sense of "I can see everything that is going on" is very bad. In low level code, the potential for icky problems is also much higher: You don't want to find out that some stray destructor did something that should not have happened in e.g. an interrupt context, or while the current thread's memory management is in an intermediate state. That is not correct by default, it is "wrong and you didn't even tell me". In high level application software development, e.g. CRUD SaaS or a smartphone app, the opposite may be the case, and decluttering might well win out.

kllrnohj 4y ago
> then anything that reduces readability in the sense of "I can see everything that is going on" is very bad. Nonsense. So called "kernel C" still uses functions, and lots of them, scattered over thousands of files. You never "see everything that is going on." And "kernel C" also uses function pointers and macros, further preventing you from "seeing everything that is going on." Having written and read lots and lots of C++, RAII being "hidden" is hugely overblown by the diehard C crowd. It's really not hidden at all, first of all (it's often right there in the type - pretty hard to forget that unique_ptr is going to call delete, for example). And second it goes so so far in preventing resource leak bugs, which is also something kernels and low level code care a lot about. In fact, they almost always care far more about runtime correctness on this than readibility. The benefits RAII provide here so massively outweigh the "hidden" concerns it's almost laughable. > You don't want to find out that some stray destructor did something that should not have happened in e.g. an interrupt context, or while the current thread's memory management is in an intermediate state. That is not correct by default, it is "wrong and you didn't even tell me". RAII isn't just randomly injected. It's governed by the types you use. If a given type cannot be safely cleaned up in a given context, then it probably also wasn't safe to get in that context in the first place. Which the type should be communicating. So no it's not "wrong by default" since the initial (explicit!) acquisition was the "wrong" part, not the cleanup. Regardless, it's not like C helps here at all. You still have to call the cleanup function for your given data, and that cleanup function may change without your interrupt handler knowing about that change.

nyanpasu64 4y ago

Some people criticize RAII for introducing hidden destructor calls into close-braces. I don't find this a problem in my experience (aside from having to make destructors do nothing in C++'s moved-from objects), but nonetheless I want to try using a language where instead of RAII as an automation system, it serves as a compiler check rejecting code which does not call a destructor exactly-once when needed (which may even return a value, as in linear types or fclose or Vale's Higher RAII). Linear types work better in a language without exception/panic unwinding (but even then you could use the standard destructor in case of crashes), and relegating RAII to purely a static analyzer is conceptually incompatible with having the presence of destructors affect stack unwinding. I'm fine with that since I don't like stack unwinding and don't find it useful in most (not all) cases.

slaymaker1907 4y ago

Eh, I get why people often don't like things like defer, but I like how versatile it is. In general, my ideal language would have both something like defer as well as proper RAII. In addition to being easier to write one-off cleanup statements, defer can be more readable if the cleanup is simple since it doesn't require looking at any data structure's code. Finally, defer can be advantageous in certain scenarios since, at least in Go, it is somewhat dynamically scoped. Therefore, you can defer something in an if statement or in the middle of a loop and the defer will still be executed at the end of the function. This can also be a disadvantage with loops since if you do something like "defer C.free(tempMem)", the memory won't actually be freed until the end of the function.

boardwaalk 4y ago
The thing is, you can trivially implement a form of defer with RAII. Where I work we have a decades old C++ macro called ON_BLOCK_EXIT that people use to clean up C-style things that don't have a destructor. Today it's much better with C++ having lambdas. It won't be "function-scoped" like Go, but (IMO) that's fairly useless feature if you have destructors anyways (you could just collect everything in a vector at function-scope, if you really want that behavior).

Ericson2314 4y ago

The best case is linear types + defer, so you can clean up however you want, but if you forget to do so it's an error. (Also, hi! Kevin Cox also does good Nix things, folks. Be like Kevin and do so too.)

masklinn 4y ago
defer seems completely unnecessary there. It’s at best a trivial convenience, since the typesystem should flag any path where the resource is not cleaned.
- Ericson2314 4y ago
  Defer is still nice to avoid verbose boilerplate.
  - masklinn 4y ago
    Defers are verbose boilerplate. Instead you allow relaxing linear types to affine.
    - Ericson2314 4y ago
      You are assuming the thing can drop itself. When that is fine, sure, do RAII. But not all cases can be done that way. For example, the Rust allocators working group has been contorting itself about deallocation. The right thing to do is to not store a pointer to a (de)allocator in e.g. every Box<T, A>, but just pass the (de)allocator when it comes time to deallocate a box. But Rust is completely incapable of expressing this and so we will get some crazy epicycle thing instead, or no good support for custom allocators with non-flat data structures. Even if RAII is the 90% solution I think it is still good to layer the feautres in this way: 1. Linear types. Fundamental building block for that some types one cannot just forget the values. 2. Defer, sugar for scope-based running some code that will eliminate a linear variable. avoiding an error. 3. RAII, a way to fill provide a default defer command if none was manually provided. Default is "type driven" --- we associate a default defer command with the linear type that would be dropping. From a Rust Perspective, we would want to distinguish Clone (an arbitrary convention for cloning) from Copy (and indication we can in fact copy for free. Saying types that need a destructor and types that don't are both affine is like adding some .clone() sugar and erasing the Copy--Clone distinction --- It is erasing a lot of essential nuance/complexity and I don't think a good idea.

Animats 4y ago

There's a lot to be said for the semantics of the Python "with" statement as a way to do scope-based closeout. You get to handle errors at closeout. Nesting works. Nested errors work. Interaction with exceptions works. It's the only construct I've seen which handles the hard error cases.