7 ms·

I've been so annoyed with people falsely equating defer and RAII that it didn't occur to me this is yet another way it's inferior! The thing that usually comes

I've been so annoyed with people falsely equating defer and RAII that it didn't occur to me this is yet another way it's inferior! The thing that usually comes to mind for me is that defer only works for local scope. RAII is more general because RAII objects can be put inside other objects that have different lifetimes. I can have an OpenFile inside a struct inside a reference counted smart pointer which will make it so that when the last user using that file disconnects the file is automatically closed. Defer can't do that.

masklinn 4y ago
Indeed `defer` quickly gets real complicated when you need to return an object but you also may need to clean it up e.g. f, err := os.Open("file.go") if err != nil { return nil, err } data := make([]byte, 100) count, err := f.Read(data) if err != nil { return nil, err } _, err = f.Seek(0, 0) if err != nil { return nil, err } return f, nil You can't just `defer f.Close()` after the first condition (regardless of flushing / write loss issues, we'll assume read-only) because you want to return it. You need to either: * defer a callback and use a flag * or specifically not use defer, remember to close the file on every path, and hope your linter supports that pattern RAII? RAII just does the right hing there (disregarding the same flushing issue on writes). In case of error you drop the file and that closes it. In case of no-error you return the file object, that automatically makes it the responsibility of the caller.
- sirwhinesalot 4y ago
  Some languages (like D and Zig) distinguish between "error defer" and "success defer", which solves the issue above. Not as automatic as RAII, but better than the single defer (which sucks). EDIT: My personal preference is actually something like Python's "with" or C#'s "using", since they rely on "destructors" rather than explicit calls to close. But they have the exact same issue as simple defer. If you ever need to store or return the thing, you need to handle it manually instead. In my own usage I almost never keep stuff open for long or store it in complex data structures where this is an issue in practice, simple "using" is almost always enough. Rust and C++ need RAII because they also manage memory like any other resource, otherwise it would get incredibly painful. (Zig uses arena and pool allocators to manage memory, if you had to manage all memory with defers I think people would go nuts)
  - shakna 4y ago
    > My personal preference is actually something like Python's "with" or C#'s "using", since they rely on "destructors" rather than explicit calls to close. But they have the exact same issue as simple defer. If you ever need to store or return the thing, you need to handle it manually instead. Python's "with" doesn't. That is to say, when the with-block closes, it doesn't call any kind of destructor against the variable. It calls the __exit__ method, but it still leaves the variable in-scope. And if the __exit__ method tries to call an actual destructor against the variable, you'll hit a double-free bug when the GC runs later. You can safely return to the variable after storing it, and do anything it supports after-closing. Including returning the variable. def retIOWrapper(x): with open(x) as f: pass assert(f.closed == True) return f Python's with isn't introducing a scope lifetime.
    - MereInterest 4y ago
      This can sometimes be useful, though I haven't yet decided if I think it's a dirty hack or not. At one point, I made a timer whose __enter__ method started the timer, and whose __exit__ method stopped the timer. After the "with", the elapsed time could be queried.
      - masklinn 4y ago
        That is used by several context managers, mostly those related to intercepting exceptions and warnings.
      - andreareina 4y ago
        I feel like that would be better as t = timer() with t: ... (I've done this exact thing for this exact use-case)
    - sirwhinesalot 4y ago
      C#'s using is also not calling the "destructor", it's calling .Dispose() C# and Python have actual destructors "nobody" uses because they are GCed languages which make this discussion confusing (well, in Python they do sometimes, because it is reference counted and more predictable, making the discussion even more confusing). I'm treating __exit__() and .Dispose() as "destructors" here, in the sense that they are implicitly called methods that free the resource (like drop in Rust), vs defering a call to close, which is explicit and "configurable" (you have to call the right method). You get to keep the variable around, but the resource itself is gone after the with/using block, you have to reacquire it. With some changes to the way the APIs were written, it would make a bit more sense (IMO): files = [File("a.txt"), File("b.txt"), File("c.txt")] for f in files: with f.open('r'): print(f.read()) return files I personally extremely dislike the fact that "as f" is not scoped.
- svnpenn 4y ago
  > You can't just `defer f.Close()` after the first condition (regardless of flushing / write loss issues, we'll assume read-only) because you want to return it You would never include Close in this function, period. In this example above, it would always be the caller closing the file: package main import "os" func hello() (*os.File, error) { return os.Open("file.go") } func main() { file, err := hello() if err != nil { panic(err) } defer file.Close() }
  - geraneum 4y ago
    The problem is, the programmer (caller) forgets and that’s the whole point of having a more or less automated way of releasing/closing/etc. Apart from that, it’s bad design IMO (leaky abstraction).
  - jjnoakes 4y ago
    The example above has an operation that might fail between the Open and the return. Leaving that out of course makes the example useless, but if you keep that part in, you can't use go's simple defer pattern to help you close the file if that intermediate operation fails.
  - tialaramex 4y ago
    Now though this extra work (the thing I got has to have deferred work done when I don't need it any more) is given to your caller. For a File Handle this might feel natural, but what about if I hand you a... 3D printer? a Bank Account? a Youtube Video? a Package Delivery? Which of these need some sort of "deferred close" action ? You probably need to read the docs and arrange to close or not close things accordingly right? You're going to be Wrong by Default again.
  - masklinn 4y ago
    > In this example above, it would always be the caller closing the file: You missed the entire point, literally. Maybe try reading the original comment again?
- TheDong 4y ago
  RAII is superior, but there's also another go pattern for that which reads better than your two alternatives sometimes: // You _must_ 'defer closeFunc()' (with closeFunc being the second return value) after this function, // even if this function returns an error. // You do not need to call File.Close() on the returned file, only closeFunc() func openFileAndPeakHeader(path string) (*os.File, func() error, error) { closeF := func() error { return nil } f, err := os.Open(path) if err != nil { return nil, closeF, err } closeF = f.Close // ... if _, err := f.Read(data); err != nil { return nil, closeF, err } return f, closeF, nil } Unfortunately, it relies on someone reading docs for them to determine that this isn't an Either<(File, func), error>, but rather an Either<File, (func, error)>, but that's how go works anyways. You have to read doc comments to know what types an 'error' might be, you have to read docs to know if return values are Either or not (i.e. file.Read returns n, err, and both are meaningful, not just one of them). It's not good, but I think it's better in some cases than the options you presented.
- pphysch 4y ago
  What's so "complicated" about sticking a f.Close() before those latter two nil, err returns?
  - MereInterest 4y ago
    Getting it right once, in an example, where everything is visible within ten lines? Not complicated at all. Getting it right every time, across thousands of lines of code, interspersed with other functionality? Very complicated. If you design a system that requires multiple locations to be in sync with each other for correctness, that's a system that will quickly become incorrect.
    - pphysch 4y ago
      If you're the 1% building wrappers around syscalls and IO, you're gonna have to think carefully about all these failure conditions. Meanwhile, the other 99% of IO-invoking code gets to benefit from the syntactic sugar of `defer`.

masklinn 4y ago

Indeed `defer` quickly gets real complicated when you need to return an object but you also may need to clean it up e.g. f, err := os.Open("file.go") if err != nil { return nil, err } data := make([]byte, 100) count, err := f.Read(data) if err != nil { return nil, err } _, err = f.Seek(0, 0) if err != nil { return nil, err } return f, nil You can't just `defer f.Close()` after the first condition (regardless of flushing / write loss issues, we'll assume read-only) because you want to return it. You need to either: * defer a callback and use a flag * or specifically not use defer, remember to close the file on every path, and hope your linter supports that pattern RAII? RAII just does the right hing there (disregarding the same flushing issue on writes). In case of error you drop the file and that closes it. In case of no-error you return the file object, that automatically makes it the responsibility of the caller.

sirwhinesalot 4y ago
Some languages (like D and Zig) distinguish between "error defer" and "success defer", which solves the issue above. Not as automatic as RAII, but better than the single defer (which sucks). EDIT: My personal preference is actually something like Python's "with" or C#'s "using", since they rely on "destructors" rather than explicit calls to close. But they have the exact same issue as simple defer. If you ever need to store or return the thing, you need to handle it manually instead. In my own usage I almost never keep stuff open for long or store it in complex data structures where this is an issue in practice, simple "using" is almost always enough. Rust and C++ need RAII because they also manage memory like any other resource, otherwise it would get incredibly painful. (Zig uses arena and pool allocators to manage memory, if you had to manage all memory with defers I think people would go nuts)
- shakna 4y ago
  > My personal preference is actually something like Python's "with" or C#'s "using", since they rely on "destructors" rather than explicit calls to close. But they have the exact same issue as simple defer. If you ever need to store or return the thing, you need to handle it manually instead. Python's "with" doesn't. That is to say, when the with-block closes, it doesn't call any kind of destructor against the variable. It calls the __exit__ method, but it still leaves the variable in-scope. And if the __exit__ method tries to call an actual destructor against the variable, you'll hit a double-free bug when the GC runs later. You can safely return to the variable after storing it, and do anything it supports after-closing. Including returning the variable. def retIOWrapper(x): with open(x) as f: pass assert(f.closed == True) return f Python's with isn't introducing a scope lifetime.
  - MereInterest 4y ago
    This can sometimes be useful, though I haven't yet decided if I think it's a dirty hack or not. At one point, I made a timer whose __enter__ method started the timer, and whose __exit__ method stopped the timer. After the "with", the elapsed time could be queried.
    - masklinn 4y ago
      That is used by several context managers, mostly those related to intercepting exceptions and warnings.
    - andreareina 4y ago
      I feel like that would be better as t = timer() with t: ... (I've done this exact thing for this exact use-case)
  - sirwhinesalot 4y ago
    C#'s using is also not calling the "destructor", it's calling .Dispose() C# and Python have actual destructors "nobody" uses because they are GCed languages which make this discussion confusing (well, in Python they do sometimes, because it is reference counted and more predictable, making the discussion even more confusing). I'm treating __exit__() and .Dispose() as "destructors" here, in the sense that they are implicitly called methods that free the resource (like drop in Rust), vs defering a call to close, which is explicit and "configurable" (you have to call the right method). You get to keep the variable around, but the resource itself is gone after the with/using block, you have to reacquire it. With some changes to the way the APIs were written, it would make a bit more sense (IMO): files = [File("a.txt"), File("b.txt"), File("c.txt")] for f in files: with f.open('r'): print(f.read()) return files I personally extremely dislike the fact that "as f" is not scoped.
svnpenn 4y ago
> You can't just `defer f.Close()` after the first condition (regardless of flushing / write loss issues, we'll assume read-only) because you want to return it You would never include Close in this function, period. In this example above, it would always be the caller closing the file: package main import "os" func hello() (*os.File, error) { return os.Open("file.go") } func main() { file, err := hello() if err != nil { panic(err) } defer file.Close() }
- geraneum 4y ago
  The problem is, the programmer (caller) forgets and that’s the whole point of having a more or less automated way of releasing/closing/etc. Apart from that, it’s bad design IMO (leaky abstraction).
- jjnoakes 4y ago
  The example above has an operation that might fail between the Open and the return. Leaving that out of course makes the example useless, but if you keep that part in, you can't use go's simple defer pattern to help you close the file if that intermediate operation fails.
- tialaramex 4y ago
  Now though this extra work (the thing I got has to have deferred work done when I don't need it any more) is given to your caller. For a File Handle this might feel natural, but what about if I hand you a... 3D printer? a Bank Account? a Youtube Video? a Package Delivery? Which of these need some sort of "deferred close" action ? You probably need to read the docs and arrange to close or not close things accordingly right? You're going to be Wrong by Default again.
- masklinn 4y ago
  > In this example above, it would always be the caller closing the file: You missed the entire point, literally. Maybe try reading the original comment again?
TheDong 4y ago
RAII is superior, but there's also another go pattern for that which reads better than your two alternatives sometimes: // You _must_ 'defer closeFunc()' (with closeFunc being the second return value) after this function, // even if this function returns an error. // You do not need to call File.Close() on the returned file, only closeFunc() func openFileAndPeakHeader(path string) (*os.File, func() error, error) { closeF := func() error { return nil } f, err := os.Open(path) if err != nil { return nil, closeF, err } closeF = f.Close // ... if _, err := f.Read(data); err != nil { return nil, closeF, err } return f, closeF, nil } Unfortunately, it relies on someone reading docs for them to determine that this isn't an Either<(File, func), error>, but rather an Either<File, (func, error)>, but that's how go works anyways. You have to read doc comments to know what types an 'error' might be, you have to read docs to know if return values are Either or not (i.e. file.Read returns n, err, and both are meaningful, not just one of them). It's not good, but I think it's better in some cases than the options you presented.
pphysch 4y ago
What's so "complicated" about sticking a f.Close() before those latter two nil, err returns?
- MereInterest 4y ago
  Getting it right once, in an example, where everything is visible within ten lines? Not complicated at all. Getting it right every time, across thousands of lines of code, interspersed with other functionality? Very complicated. If you design a system that requires multiple locations to be in sync with each other for correctness, that's a system that will quickly become incorrect.
  - pphysch 4y ago
    If you're the 1% building wrappers around syscalls and IO, you're gonna have to think carefully about all these failure conditions. Meanwhile, the other 99% of IO-invoking code gets to benefit from the syntactic sugar of `defer`.