4 ms·

> File descriptors getting reused I don't know how you could avoid this, and it has me wondering if you understand the issue. This is basically the file descri

> File descriptors getting reused I don't know how you could avoid this, and it has me wondering if you understand the issue. This is basically the file descriptor version of a use-after-free bug. I'm not one of those "rust fixes everything" people, so I don't answer this about just anything, but I think this issue would completely disappear in a memory safe language and especially one where the wrapper for file descriptors had good prevention of race conditions closing a file, so it hardly seems fair to blame the filesystem API. I mean, in some cases you could blame the API for high frequency of application bugs, but using a closed file seems like a bug category that is unreasonable to do so for. The issue of use-after-free is larger than the filesystem API.

rfoo 5y ago
Writing to arbitrary file descriptor (because, well, a fd is just an integer, right?) is safe Rust. So, no.
- asveikau 5y ago
  > and especially one where the wrapper for file descriptors had good prevention of race conditions closing a file This is an ownership issue is my point. It's not exactly memory safety but the same concept applied to an integer rather than a pointer.
- chrismorgan 5y ago
  If you want to write to an arbitrary file descriptor in Rust, you need to use unsafe code (std::os::unix::io::FromRawFd, unsafe fn from_raw_fd). … though if you’ve got a procfs around, /proc/self/fd/* is ripe for the writing.
  - mst 5y ago
    I have previously used memfd_create plus /proc/self/fd/* to emulate fdlopen on systems that only provide dlopen. I strongly recommend finding a different approach ... but it totally worked.
moleperson 5y ago
It sounded like the underlying issue was the OS reusing the same int value for the file descriptor (2), rather than a dangling FILE*, which AFIK is not something rust could prevent. The only way you could really prevent this is if the OS generated GUIDs for newly opened files.
zvrba 5y ago
> I don't know how you could avoid this, and it has me wondering if you understand the issue. I understand the issue. Someone else has already mentioned how you could avoid this: allocate fds randomly instead of sequentially. This would significantly reduce probability of accidental reuse. If you widen fds to 64 bits then you're pretty much good. Yes, it'd break syscalls that rely on sequential allocation like select. It just shows how POSIX has painted itself into a historical corner, alongside with many interactions with threads being a kludge.
- the_duke 5y ago
  With 64 bits a system can create 59973028745 file descriptors per second for 10 years before running, out, so no need for randomizing.
  - zvrba 5y ago
    Depends on what you're defending against. Sequential ids are easy to guess, and that's why we have ASLR for memory management.
- asveikau 5y ago
  Windows does not allocate handles sequentially, and I've seen this pop up as a race condition and bug. I don't think changing the way they're assigned is necessarily good enough.
  - zvrba 5y ago
    > Windows does not allocate handles sequentially Indeed it does not, but it's not "random" either, i.e., they do get reused, probably rather quickly, so it does not solve the problem. Mark Russinovich or Raymond Chen (I think) had an article about why it's a bad idea to forcibly close files that are in use and they gave exactly that as a reason: the closed handle gets valid again, but for a different object, and, voila - corruption!
- tedunangst 5y ago
  So now the failure case still happens, but only in production, and it's impossible to repro in testing. Great.
gizmo686 5y ago
The filesystem might not be able to block use-after-free bugs, but it could greatly mitigate them. In particular, if the fs avoided reusing file descriptors, it could return an error on use-after-free instead of silently corrupting a random data file.
- blibble 5y ago
  the filesystem doesn't know anything about file descriptors you're talking about the kernel interface that eventually maps onto a filesystem

rfoo 5y ago

Writing to arbitrary file descriptor (because, well, a fd is just an integer, right?) is safe Rust. So, no.

asveikau 5y ago
> and especially one where the wrapper for file descriptors had good prevention of race conditions closing a file This is an ownership issue is my point. It's not exactly memory safety but the same concept applied to an integer rather than a pointer.
chrismorgan 5y ago
If you want to write to an arbitrary file descriptor in Rust, you need to use unsafe code (std::os::unix::io::FromRawFd, unsafe fn from_raw_fd). … though if you’ve got a procfs around, /proc/self/fd/* is ripe for the writing.
- mst 5y ago
  I have previously used memfd_create plus /proc/self/fd/* to emulate fdlopen on systems that only provide dlopen. I strongly recommend finding a different approach ... but it totally worked.

moleperson 5y ago

It sounded like the underlying issue was the OS reusing the same int value for the file descriptor (2), rather than a dangling FILE*, which AFIK is not something rust could prevent. The only way you could really prevent this is if the OS generated GUIDs for newly opened files.

zvrba 5y ago

> I don't know how you could avoid this, and it has me wondering if you understand the issue. I understand the issue. Someone else has already mentioned how you could avoid this: allocate fds randomly instead of sequentially. This would significantly reduce probability of accidental reuse. If you widen fds to 64 bits then you're pretty much good. Yes, it'd break syscalls that rely on sequential allocation like select. It just shows how POSIX has painted itself into a historical corner, alongside with many interactions with threads being a kludge.

the_duke 5y ago
With 64 bits a system can create 59973028745 file descriptors per second for 10 years before running, out, so no need for randomizing.
- zvrba 5y ago
  Depends on what you're defending against. Sequential ids are easy to guess, and that's why we have ASLR for memory management.
asveikau 5y ago
Windows does not allocate handles sequentially, and I've seen this pop up as a race condition and bug. I don't think changing the way they're assigned is necessarily good enough.
- zvrba 5y ago
  > Windows does not allocate handles sequentially Indeed it does not, but it's not "random" either, i.e., they do get reused, probably rather quickly, so it does not solve the problem. Mark Russinovich or Raymond Chen (I think) had an article about why it's a bad idea to forcibly close files that are in use and they gave exactly that as a reason: the closed handle gets valid again, but for a different object, and, voila - corruption!
tedunangst 5y ago
So now the failure case still happens, but only in production, and it's impossible to repro in testing. Great.

gizmo686 5y ago

The filesystem might not be able to block use-after-free bugs, but it could greatly mitigate them. In particular, if the fs avoided reusing file descriptors, it could return an error on use-after-free instead of silently corrupting a random data file.

blibble 5y ago
the filesystem doesn't know anything about file descriptors you're talking about the kernel interface that eventually maps onto a filesystem