5 ms·

Reminds me of `diags` from [0]. Side note: can we make 'file not found' errors that fail to mention what wasn't found illegal? What's the point of 'file not fo

Reminds me of `diags` from [0]. Side note: can we make 'file not found' errors that fail to mention what wasn't found illegal? What's the point of 'file not found', then keeping silent about the exact file? Seen it in various ecosystems now. [0]: https://www.lurklurk.org/effective-rust/macros.html#when-to-use-macros https://www.lurklurk.org/effective-rust/macros.html#when-to-...

dathinab 3y ago
I wonder if not having `what` comes from two things: 1. the underlying OS error not containing what 2. some edge cases in which you don't have a what, or at least not a clear one 3. paths, especially absolute paths, always being prone on leaking personal user information
thristian 3y ago
The Unix kernel API only returns a numeric error code, no additional information. Nine times out of ten¹ the file that wasn't found is the file that you just tried to open or rename or whatever, so a language runtime can make a copy of that and attach it to their error representation, but (a) that's more work so it gets shunted out of the MVP, and (b) it requires allocating memory to store the copy, and some languages (like Rust) do not want silent memory allocation as a default behaviour. ¹: The one time out of ten is things like a script with a #! line that refers to a file that doesn't exist. The file you asked for (the script) definitely exists, but you'll get a file-not-found error if you ask the kernel to run it. Modern versions of bash seem to detect this situation and display a more helpful error message, but this is specific to bash, not other shells or languages.
- deleted 3y ago
  [deleted]
- littlestymaar 3y ago
  > (b) it requires allocating memory to store the copy, and some languages (like Rust) do not want silent memory allocation as a default behaviour. There's no need for an allocation: you already have the path allocated somewhere since you're making the call. So you could just have a pointer to that path in the error. That would obviously be very dangerous in C, because then you'd have to be careful not to have your error outlive your allocated path, but in Rust you'd have a lifetime annotation on your Error guaranteeing that your Error does not outlive your allocated path.
  - duped 3y ago
    > but in Rust you'd have a lifetime annotation on your Error guaranteeing that your Error does not outlive your allocated path. That's a really heavy constraint that doesn't work in practice - errors usually bubble up past the scope of the arguments of whatever failed.
    - littlestymaar 3y ago
      This design would force you to make a choice: if you want to bubble the error up[1], then you either: - discard the file path - allocate This way, you control the performance/UX trade-off. And it doesn't require much work: it's just a From implementation to add to your error enum. [1] only higher than the actual owned path actually.
      - insanitybit 3y ago
        What you're talking about is very practical imo and likely the right solution, but unfortunately would be a breaking change (you can't just add a lifetime annotation to the error).
        littlestymaar 3y ago
        It's not even my biggest complaint about std::fs, the entire module feels very much not like the rest of Rust and way too close to the underlying libc (for legitimate reasons: they couldn't release Rust 1.0 without file access and likely didn't have much bandwidth to work on an alternative API). Maybe some day Rust will support multiple versions of the stdlib (in a compatible way, like what's done with editions) but until then we'll have to work with this half-baked corner of the language.
  - voidnap 3y ago
    > you'd have a lifetime annotation on your Error guaranteeing that your Error does not outlive your allocated path This has implications that are obviously undesirable, like prohibiting returning errors borrowing file paths allocated on the stack.
- duped 3y ago
  > The one time out of ten is things like a script with a #! line that refers to a file that doesn't exist. More generally, the interpreter doesn't exist. For a script with a shebang that's whatever is under the shebang. For executables it's whatever is in the PT_INTERP header in the binary (in ELF, or whatever the equivalent is for PE/Mach-O). For example you might have an ELF binary with PT_INTERP set to /usr/lib/ld-linux.so when it's actually /lib/x86_64-unknown-linux-gnu/ld-linux.so. You'll get a "exec: File not found" error, even if all the files you think you need exist. The loader itself is what's missing.
junon 3y ago
No, because that's not desired behavior in all cases. There can be security/privacy issues stemming from that. Making it the default means you have to do string manipulation to cut out information you don't otherwise want, which is way less robust and more fragile than simply not including it in the first place.
- wizofaus 3y ago
  For the maybe 2 or 3 times I've thought "we probably shouldn't include that in an error visible to end-users" there's probably 100 times I've wished it had been included though, and would've been able to save said end users considerable time if we/they had known which file/resource was being accessed at the point a critical error occurred. At any rate at most I'd expect the file name to be written to a log file, to which only relatively privileged users would have access.
  - junon 3y ago
    Yes sure but to make it the default asserts that it's the correct thing to do in all cases. Those aren't assertions rust folks tend to make, and for things like logging that's really up to the application how things are logged anyway. Plus if you're issuing a filesystem call you're almost definitely in possession of the filename anyway so it's just a matter of format!("failed to write: {:?}: {:?}", err, filename).
maxbond 3y ago
My related pet peeve is a failure to give a good error message when the disk runs out of space. This often manifests in multiple applications failing silently or with cryptic error numbers. Sometimes you can't log in to the computer, the password is accepted but you're returned to the login prompt with no error message. I've learned to immediately suspect disk exhaustion when I'm experiencing unexplained failures, but in the past it's taken me forever to figure it out.
- aomix 3y ago
  I remember spending an afternoon figuring out why Docker images stopped building with legitimate errors because it ran out of disk space. It never even occurred to me to check and I found the virtual disk was full by accident.
littlestymaar 3y ago
> Seen it in various ecosystems now. It's basically libc's behavior so any ecosystem that just offers a thin layer on top of libc for file access without thinking about it too much is going to do it this way.
cdogl 3y ago
Including the filename is a highly opinionated decision. First of all it means your error value needs to have a string in it at minimum, which means extra overhead compared (eg) an enum that could compile down to an int. Maybe you don’t normally care, but the standard library has to. The caller always has the context needed to know what file wasn’t found and propagate it as appropriate. This also introduces a whole angle around ownership of that memory in a language like Rust, and would be a huge headache in C. It also increases the surface area of the API which makes everything around it less portable and may mean it’s awkward to mock in tests.
conradludgate 3y ago
For the file not found errors, I recommend the fs-err crate. If you ask for async support, you should use spawn_blocking anyway as that's what tokio::fs does too

dathinab 3y ago

I wonder if not having `what` comes from two things: 1. the underlying OS error not containing what 2. some edge cases in which you don't have a what, or at least not a clear one 3. paths, especially absolute paths, always being prone on leaking personal user information

thristian 3y ago

The Unix kernel API only returns a numeric error code, no additional information. Nine times out of ten¹ the file that wasn't found is the file that you just tried to open or rename or whatever, so a language runtime can make a copy of that and attach it to their error representation, but (a) that's more work so it gets shunted out of the MVP, and (b) it requires allocating memory to store the copy, and some languages (like Rust) do not want silent memory allocation as a default behaviour. ¹: The one time out of ten is things like a script with a #! line that refers to a file that doesn't exist. The file you asked for (the script) definitely exists, but you'll get a file-not-found error if you ask the kernel to run it. Modern versions of bash seem to detect this situation and display a more helpful error message, but this is specific to bash, not other shells or languages.

deleted 3y ago
[deleted]
littlestymaar 3y ago
> (b) it requires allocating memory to store the copy, and some languages (like Rust) do not want silent memory allocation as a default behaviour. There's no need for an allocation: you already have the path allocated somewhere since you're making the call. So you could just have a pointer to that path in the error. That would obviously be very dangerous in C, because then you'd have to be careful not to have your error outlive your allocated path, but in Rust you'd have a lifetime annotation on your Error guaranteeing that your Error does not outlive your allocated path.
- duped 3y ago
  > but in Rust you'd have a lifetime annotation on your Error guaranteeing that your Error does not outlive your allocated path. That's a really heavy constraint that doesn't work in practice - errors usually bubble up past the scope of the arguments of whatever failed.
  - littlestymaar 3y ago
    This design would force you to make a choice: if you want to bubble the error up[1], then you either: - discard the file path - allocate This way, you control the performance/UX trade-off. And it doesn't require much work: it's just a From implementation to add to your error enum. [1] only higher than the actual owned path actually.
    - insanitybit 3y ago
      What you're talking about is very practical imo and likely the right solution, but unfortunately would be a breaking change (you can't just add a lifetime annotation to the error).
      - littlestymaar 3y ago
        It's not even my biggest complaint about std::fs, the entire module feels very much not like the rest of Rust and way too close to the underlying libc (for legitimate reasons: they couldn't release Rust 1.0 without file access and likely didn't have much bandwidth to work on an alternative API). Maybe some day Rust will support multiple versions of the stdlib (in a compatible way, like what's done with editions) but until then we'll have to work with this half-baked corner of the language.
- voidnap 3y ago
  > you'd have a lifetime annotation on your Error guaranteeing that your Error does not outlive your allocated path This has implications that are obviously undesirable, like prohibiting returning errors borrowing file paths allocated on the stack.
duped 3y ago
> The one time out of ten is things like a script with a #! line that refers to a file that doesn't exist. More generally, the interpreter doesn't exist. For a script with a shebang that's whatever is under the shebang. For executables it's whatever is in the PT_INTERP header in the binary (in ELF, or whatever the equivalent is for PE/Mach-O). For example you might have an ELF binary with PT_INTERP set to /usr/lib/ld-linux.so when it's actually /lib/x86_64-unknown-linux-gnu/ld-linux.so. You'll get a "exec: File not found" error, even if all the files you think you need exist. The loader itself is what's missing.

junon 3y ago

No, because that's not desired behavior in all cases. There can be security/privacy issues stemming from that. Making it the default means you have to do string manipulation to cut out information you don't otherwise want, which is way less robust and more fragile than simply not including it in the first place.

wizofaus 3y ago
For the maybe 2 or 3 times I've thought "we probably shouldn't include that in an error visible to end-users" there's probably 100 times I've wished it had been included though, and would've been able to save said end users considerable time if we/they had known which file/resource was being accessed at the point a critical error occurred. At any rate at most I'd expect the file name to be written to a log file, to which only relatively privileged users would have access.
- junon 3y ago
  Yes sure but to make it the default asserts that it's the correct thing to do in all cases. Those aren't assertions rust folks tend to make, and for things like logging that's really up to the application how things are logged anyway. Plus if you're issuing a filesystem call you're almost definitely in possession of the filename anyway so it's just a matter of format!("failed to write: {:?}: {:?}", err, filename).

maxbond 3y ago

My related pet peeve is a failure to give a good error message when the disk runs out of space. This often manifests in multiple applications failing silently or with cryptic error numbers. Sometimes you can't log in to the computer, the password is accepted but you're returned to the login prompt with no error message. I've learned to immediately suspect disk exhaustion when I'm experiencing unexplained failures, but in the past it's taken me forever to figure it out.

aomix 3y ago
I remember spending an afternoon figuring out why Docker images stopped building with legitimate errors because it ran out of disk space. It never even occurred to me to check and I found the virtual disk was full by accident.

littlestymaar 3y ago

> Seen it in various ecosystems now. It's basically libc's behavior so any ecosystem that just offers a thin layer on top of libc for file access without thinking about it too much is going to do it this way.

cdogl 3y ago

Including the filename is a highly opinionated decision. First of all it means your error value needs to have a string in it at minimum, which means extra overhead compared (eg) an enum that could compile down to an int. Maybe you don’t normally care, but the standard library has to. The caller always has the context needed to know what file wasn’t found and propagate it as appropriate. This also introduces a whole angle around ownership of that memory in a language like Rust, and would be a huge headache in C. It also increases the surface area of the API which makes everything around it less portable and may mean it’s awkward to mock in tests.

conradludgate 3y ago

For the file not found errors, I recommend the fs-err crate. If you ask for async support, you should use spawn_blocking anyway as that's what tokio::fs does too