4 ms·

I was expecting that the process writing to the now deleted file would throw some kind of error/exception that the file doesn't exist anymore. I would have noti

by 323 4y ago

I was expecting that the process writing to the now deleted file would throw some kind of error/exception that the file doesn't exist anymore. I would have noticed that. But instead it silently continued writing into the void.

trelane 4y ago
I still don't understand the loss. The original data that was going into the zip was still there. Or did you the write pipeline delete it at the end of something? More precisely, what data was lost, and how did it get lost?
- scbrg 4y ago
  There are three processes involved. Process P, the data producer, writes to file foo. Process Z, the zip-process, reads file foo and produces foo.zip Process rm, removes file foo. I imagine it went a bit like this: P --output foo & Z < foo > foo.zip rm foo # time passes # process P terminates Any data written to foo since Z was run is now lost. User 323 assumed that either rm would fail, letting them know that foo cannot be deleted, or that P would throw an error when the file was deleted. (I don't think this is a reasonable expectation, and it's a failure on 323's hand of not learning the UNIX file model, but there's the situation).
  - trelane 4y ago
    In this case, it would have been deleted anyway. I think I can see the loss: 1. Start zip. 2. In parallel, copy the intermediate zip file. 3. Delete the zip 4. Delete originals The data was deleted in step 4 and is the loss. It notably would also had occurred had the zip terminated early or not zipped _all_ the files for some reason. The process was not resilient.
  - 323 4y ago
    I understand the UNIX file model. My mistake was that I thought that the data producer was writing to a different file (think something like log-rotation). I think it's debatable if it's reasonable to continue allowing writing to an unlinked file. There was discussion some time ago about Postgres data loss because of something similar, where the linux kernel returned successfully from fsync despite the movable media not being present anymore. > How is it possible that PostgreSQL used fsync incorrectly for 20 years, and what we'll do about it https://archive.fosdem.org/2019/schedule/event/postgresql_fsync/ https://archive.fosdem.org/2019/schedule/event/postgresql_fs...
jonhohle 4y ago
As others have mentioned, it was still writing to disk, but its directory entry was removed. This is considered a feature since the data will be automatically cleaned up once the process holding the last file descriptor closes it or terminates. Also as others have mentioned, Linux provides an additional directory entry through /proc, something not available on most other OSs. In this case, you can have your cake (delete the directory entry) and eat it too (get the data back through /proc).
- phendrenad2 4y ago
  What if something else writes a file to that location on disk? Can you still have your cake?
  - trelane 4y ago
    The "location on disk" (blocks) are owned by the file until all descriptors to it are closed. So nothing else can write to those blocks (actually, inodes) until all file handles to it are closed. At that point, the inodes in the file will be freed to be reclaimed by another file.
    - trelane 4y ago
      This is actually really handy behavior to have in a number of cases. A prominent example is temp files. They will automatically be cleaned up on process exit by the filesystem if you open and then delete them. It also helps prevent (not completely, but greatly reduces the window) others from accessing the temp file. you can also use this for intermediate files. Process A creates and writes, B opens for read and immediately deletes the file. A can continue writing and B continue reading until both close their file descriptor, at which point the inodes will get freed.
    - phendrenad2 4y ago
      That doesn't answer my question. I'm saying that you can't recover the file in any realistic way, in a reliable way. Once you close the file descriptor, the file is gone.
chasil 4y ago
POSIX does not work this way. When the last directory entry for a file is passed to the unlink() system call, the file is removed from the directory hierarchy. However, the data on media will be deleted only after all open file descriptors are closed. One exploitation of this feature that I have seen is the SQLite temporary tablespace, which is created and immediately unlinked, ensuring that it will not persist after the program terminates.
- noasaservice 4y ago
  > POSIX does not work this way. Yes, it does. Windows is POSIX/FIPS-151 https://www.quora.com/Is-Windows-POSIX-compliant https://www.quora.com/Is-Windows-POSIX-compliant
  - chasil 4y ago
    It was, but the POSIX subsystem was removed. In any case, I have never seen any way to coerce POSIX unlink() behavior on Windows. 'Broad software compatibility was initially achieved with support for several API "personalities", including Windows API, POSIX, and OS/2 APIs – the latter two were phased out starting with Windows XP.' https://en.wikipedia.org/wiki/Windows_NT https://en.wikipedia.org/wiki/Windows_NT

by 323 4y ago

trelane 4y ago

I still don't understand the loss. The original data that was going into the zip was still there. Or did you the write pipeline delete it at the end of something? More precisely, what data was lost, and how did it get lost?

scbrg 4y ago
There are three processes involved. Process P, the data producer, writes to file foo. Process Z, the zip-process, reads file foo and produces foo.zip Process rm, removes file foo. I imagine it went a bit like this: P --output foo & Z < foo > foo.zip rm foo # time passes # process P terminates Any data written to foo since Z was run is now lost. User 323 assumed that either rm would fail, letting them know that foo cannot be deleted, or that P would throw an error when the file was deleted. (I don't think this is a reasonable expectation, and it's a failure on 323's hand of not learning the UNIX file model, but there's the situation).
- trelane 4y ago
  In this case, it would have been deleted anyway. I think I can see the loss: 1. Start zip. 2. In parallel, copy the intermediate zip file. 3. Delete the zip 4. Delete originals The data was deleted in step 4 and is the loss. It notably would also had occurred had the zip terminated early or not zipped _all_ the files for some reason. The process was not resilient.
- 323 4y ago
  I understand the UNIX file model. My mistake was that I thought that the data producer was writing to a different file (think something like log-rotation). I think it's debatable if it's reasonable to continue allowing writing to an unlinked file. There was discussion some time ago about Postgres data loss because of something similar, where the linux kernel returned successfully from fsync despite the movable media not being present anymore. > How is it possible that PostgreSQL used fsync incorrectly for 20 years, and what we'll do about it https://archive.fosdem.org/2019/schedule/event/postgresql_fsync/ https://archive.fosdem.org/2019/schedule/event/postgresql_fs...

jonhohle 4y ago

As others have mentioned, it was still writing to disk, but its directory entry was removed. This is considered a feature since the data will be automatically cleaned up once the process holding the last file descriptor closes it or terminates. Also as others have mentioned, Linux provides an additional directory entry through /proc, something not available on most other OSs. In this case, you can have your cake (delete the directory entry) and eat it too (get the data back through /proc).

phendrenad2 4y ago
What if something else writes a file to that location on disk? Can you still have your cake?
- trelane 4y ago
  The "location on disk" (blocks) are owned by the file until all descriptors to it are closed. So nothing else can write to those blocks (actually, inodes) until all file handles to it are closed. At that point, the inodes in the file will be freed to be reclaimed by another file.
  - trelane 4y ago
    This is actually really handy behavior to have in a number of cases. A prominent example is temp files. They will automatically be cleaned up on process exit by the filesystem if you open and then delete them. It also helps prevent (not completely, but greatly reduces the window) others from accessing the temp file. you can also use this for intermediate files. Process A creates and writes, B opens for read and immediately deletes the file. A can continue writing and B continue reading until both close their file descriptor, at which point the inodes will get freed.
  - phendrenad2 4y ago
    That doesn't answer my question. I'm saying that you can't recover the file in any realistic way, in a reliable way. Once you close the file descriptor, the file is gone.

chasil 4y ago

POSIX does not work this way. When the last directory entry for a file is passed to the unlink() system call, the file is removed from the directory hierarchy. However, the data on media will be deleted only after all open file descriptors are closed. One exploitation of this feature that I have seen is the SQLite temporary tablespace, which is created and immediately unlinked, ensuring that it will not persist after the program terminates.

noasaservice 4y ago
> POSIX does not work this way. Yes, it does. Windows is POSIX/FIPS-151 https://www.quora.com/Is-Windows-POSIX-compliant https://www.quora.com/Is-Windows-POSIX-compliant
- chasil 4y ago
  It was, but the POSIX subsystem was removed. In any case, I have never seen any way to coerce POSIX unlink() behavior on Windows. 'Broad software compatibility was initially achieved with support for several API "personalities", including Windows API, POSIX, and OS/2 APIs – the latter two were phased out starting with Windows XP.' https://en.wikipedia.org/wiki/Windows_NT https://en.wikipedia.org/wiki/Windows_NT