4 ms·

As a native windows user I don't really understand this grief - if the file is open we're kind of assuming we need it, aren't we? What happens if the system tr

As a native windows user I don't really understand this grief - if the file is open we're kind of assuming we need it, aren't we? What happens if the system tries to read the file on disc and it's been replaced or removed? Why shouldn't that end in an earth-shattering kaboom? Isn't the problem actually not being able to effectively close it to then replace it?

kevin_thibedeau 7y ago
This was solved in Unix land 50 years ago. Unlinked handles remain usable until the process releases them but the filesystem isn't held hostage. Doesn't require hunt the process to find out who has a file open. The OS can update itself live without a reboot.
pjc50 7y ago
There are three kinds of semantics for non-collaborating programs that people want from files that are completely incompatible: - update immediately(ish) seen by other users of the file (e.g. database); the default UNIX semantics, even for executables - update seen by other users at the next point they open the file (what you want for update-without reboot, can be achieved on UNIX because deleted inodes hang around) - it's completely impossible for other users to write, rename etc the file (Windows default)
- SketchySeaBeast 7y ago
  Thanks! I guess each come with their own tradeoffs - immediate updates have a good chance of blowing away the current users work as well as continually searching for file updates, and a versioning system then needs to figure out what to do when the user saves a stale file.
  - BubRoss 7y ago
    gvim already handles this. If a file changes it prompts you to update or not. If you try to save an old file over a new one, it makes you confirm what you are doing.
    - SketchySeaBeast 7y ago
      As it should, and as all apps have to, otherwise you end up with problems in any shared workspace.
BubRoss 7y ago
A lot of times the problem actually comes down to not being able to find out what process is keeping a file open and/or not being able to close the process or file.
- SketchySeaBeast 7y ago
  That is true. We've all been there with someone having a word file open somewhere on the network.
- SAI_Peregrinus 7y ago
  Process Explorer is the workarond. Use its search for handle, find the filename. You'll get a list of all handles to the file, and thus all processes with it open. You can close those (safest way) or force-close the handle (more likely to crash the process in question).
  - oefrha 7y ago
    I use Process Explorer when I have to, but 1. It’s not built into Windows, requires separate download; 2. Searching is hit or miss.
    - SAI_Peregrinus 7y ago
      That's why I call it a workaround, not a solution. It's good enough most of the time, but not every time.
vetinari 7y ago
This is easy, actually. File and it's directory entry are two separate things. So you can remove and replace open file, but the old handle will still point to the old file - it just won't be open-able using the original filename. Once all the processes will close the handle to the old file, and no directory entry points to it, it will be removed from the file system.
- SketchySeaBeast 7y ago
  What happens if the user tries to save the stale file? They have to give it a new name?
  - pjc50 7y ago
    Default UNIX semantics: you don't actually know this has happened to a file you're using on most filesystems, so you can continue writing to it and it will vanish on close. Many programs do the "write to temp file and rename on top of old file" trick, which is almost-atomic, and you can have multiple programs doing that to the same file at the same time without noticing. (If you're on NFS writing to a deleted inode may do something different and probably broken)
    - SketchySeaBeast 7y ago
      Doesn't that end up being dangerous as far as updates? You keep blowing away the file? I think part of my problem is I'm visualizing this as a shared file being updated by users, and not, say, system updates. System updates make sense - we can assume only the system will be modifying them, and it'll be relatively low touch, but I keep visualizing a shared text document.
      - SAI_Peregrinus 7y ago
        Yes, it's very dangerous. Files in Unix-style systems can have invisibly shared mutable state. Like having shared memory without any mutexes.
      - 7y ago
oefrha 7y ago
When a file is renamed or removed (replace is basically remove followed by rename -- often in one atomic action, of course), the open file handle and the corresponding inode can be left around and remain usable until the handle is closed. That is orthogonal to whether the filesystem path should be bound to the inode throughout its lifetime. On *ix systems it's not, so for instance a program could easily replace itself with a new version. > Isn't the problem actually not being able to effectively close it to then replace it? Not sure what this is supposed to mean. The program that's keeping the file open could effectively close it. The problem is other programs can't, plus it's hard to hunt down the process that's hanging on to a file.
snoopen 7y ago
there's a lot of instances where the system can the file locked when you don't think you have it open. preview pane in file explorer used to open an instance of the relevant software to generate a preview thereby locking the file. most Adobe products can fail to release a lock. weird filenames can prevent deletion in file explorer, but cmd can delete. sometimes windows locks the file while a thumbnail is being generated. these are the things that keep me up at night
alpaca128 7y ago
This makes sense in theory, unless it's like on Windows where the file/storage is accessed by some unknown, hidden process that you don't know anything about except that it really likes to use this specific file, and you only have the choice between shutting down the computer to safely remove some thumb drive or to just pull it and hope for the best.
cesarb 7y ago
> if the file is open we're kind of assuming we need it, aren't we? What happens if the system tries to read the file on disc and it's been replaced or removed? Why shouldn't that end in an earth-shattering kaboom? On Unix, the directory entry does not point directly to the file contents. It points to an "inode", which contains the file metadata (owner, permissions, timestamps) and points to the file contents. When you remove a file, you are actually removing the directory entry; the inode will only be removed when nothing else points to or is using it. An open file keeps a reference to the inode, so the system can still read the file just fine, even if the directory entry has been removed. Some other operating systems, like MS-DOS and Windows 9x, do not have that distinction. On them, the directory entry has the same role as the inode on Unix-style systems, which means it cannot be removed while the file is in use. The Windows NT line, when using NTFS, does have something similar to the directory entry and inode split from Unix, but it still denies removing or replacing in-use files for compatibility with Windows 9x. > Isn't the problem actually not being able to effectively close it to then replace it? The problem is when another software (for instance, a file indexer or an antivirus) has the file open. You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot.
- SAI_Peregrinus 7y ago
  Or close the other program. Also, "You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot. " is somewhat wrong. You can close the handle the other program holds on the file, it just might crash that other program.
  - cesarb 7y ago
    > You can close the handle the other program holds on the file, it just might crash that other program. It might do more than just crash that other program. Some other component within that program might open a new file (or even things which are not really files), and happen to get the same handle you had just closed (once the handle is closed, its number can be reused). Then the original component finishes what it was doing and closes the handle; the second component thinks it's still open. A third component opens a new file, and again gets the same handle. Now the second component tries to write to the handle (it might be something like a log file), and the write goes to the wrong place. Force closing a handle on a different process leads to undefined behavior, and as always with undefined behavior, anything can happen. Raymond Chen also has an article about this: http://technet.microsoft.com/en-us/magazine/2009.04.windowsconfidential.aspx http://technet.microsoft.com/en-us/magazine/2009.04.windowsc...
setr 7y ago
90% of my headache with it is that windows won't tell you which program(s) has it open

kevin_thibedeau 7y ago

This was solved in Unix land 50 years ago. Unlinked handles remain usable until the process releases them but the filesystem isn't held hostage. Doesn't require hunt the process to find out who has a file open. The OS can update itself live without a reboot.

pjc50 7y ago

There are three kinds of semantics for non-collaborating programs that people want from files that are completely incompatible: - update immediately(ish) seen by other users of the file (e.g. database); the default UNIX semantics, even for executables - update seen by other users at the next point they open the file (what you want for update-without reboot, can be achieved on UNIX because deleted inodes hang around) - it's completely impossible for other users to write, rename etc the file (Windows default)

SketchySeaBeast 7y ago
Thanks! I guess each come with their own tradeoffs - immediate updates have a good chance of blowing away the current users work as well as continually searching for file updates, and a versioning system then needs to figure out what to do when the user saves a stale file.
- BubRoss 7y ago
  gvim already handles this. If a file changes it prompts you to update or not. If you try to save an old file over a new one, it makes you confirm what you are doing.
  - SketchySeaBeast 7y ago
    As it should, and as all apps have to, otherwise you end up with problems in any shared workspace.

BubRoss 7y ago

A lot of times the problem actually comes down to not being able to find out what process is keeping a file open and/or not being able to close the process or file.

SketchySeaBeast 7y ago
That is true. We've all been there with someone having a word file open somewhere on the network.
SAI_Peregrinus 7y ago
Process Explorer is the workarond. Use its search for handle, find the filename. You'll get a list of all handles to the file, and thus all processes with it open. You can close those (safest way) or force-close the handle (more likely to crash the process in question).
- oefrha 7y ago
  I use Process Explorer when I have to, but 1. It’s not built into Windows, requires separate download; 2. Searching is hit or miss.
  - SAI_Peregrinus 7y ago
    That's why I call it a workaround, not a solution. It's good enough most of the time, but not every time.

vetinari 7y ago

This is easy, actually. File and it's directory entry are two separate things. So you can remove and replace open file, but the old handle will still point to the old file - it just won't be open-able using the original filename. Once all the processes will close the handle to the old file, and no directory entry points to it, it will be removed from the file system.

SketchySeaBeast 7y ago
What happens if the user tries to save the stale file? They have to give it a new name?
- pjc50 7y ago
  Default UNIX semantics: you don't actually know this has happened to a file you're using on most filesystems, so you can continue writing to it and it will vanish on close. Many programs do the "write to temp file and rename on top of old file" trick, which is almost-atomic, and you can have multiple programs doing that to the same file at the same time without noticing. (If you're on NFS writing to a deleted inode may do something different and probably broken)
  - SketchySeaBeast 7y ago
    Doesn't that end up being dangerous as far as updates? You keep blowing away the file? I think part of my problem is I'm visualizing this as a shared file being updated by users, and not, say, system updates. System updates make sense - we can assume only the system will be modifying them, and it'll be relatively low touch, but I keep visualizing a shared text document.
    - SAI_Peregrinus 7y ago
      Yes, it's very dangerous. Files in Unix-style systems can have invisibly shared mutable state. Like having shared memory without any mutexes.
    - 7y ago

oefrha 7y ago

When a file is renamed or removed (replace is basically remove followed by rename -- often in one atomic action, of course), the open file handle and the corresponding inode can be left around and remain usable until the handle is closed. That is orthogonal to whether the filesystem path should be bound to the inode throughout its lifetime. On *ix systems it's not, so for instance a program could easily replace itself with a new version. > Isn't the problem actually not being able to effectively close it to then replace it? Not sure what this is supposed to mean. The program that's keeping the file open could effectively close it. The problem is other programs can't, plus it's hard to hunt down the process that's hanging on to a file.

snoopen 7y ago

there's a lot of instances where the system can the file locked when you don't think you have it open. preview pane in file explorer used to open an instance of the relevant software to generate a preview thereby locking the file. most Adobe products can fail to release a lock. weird filenames can prevent deletion in file explorer, but cmd can delete. sometimes windows locks the file while a thumbnail is being generated. these are the things that keep me up at night

alpaca128 7y ago

This makes sense in theory, unless it's like on Windows where the file/storage is accessed by some unknown, hidden process that you don't know anything about except that it really likes to use this specific file, and you only have the choice between shutting down the computer to safely remove some thumb drive or to just pull it and hope for the best.

cesarb 7y ago

> if the file is open we're kind of assuming we need it, aren't we? What happens if the system tries to read the file on disc and it's been replaced or removed? Why shouldn't that end in an earth-shattering kaboom? On Unix, the directory entry does not point directly to the file contents. It points to an "inode", which contains the file metadata (owner, permissions, timestamps) and points to the file contents. When you remove a file, you are actually removing the directory entry; the inode will only be removed when nothing else points to or is using it. An open file keeps a reference to the inode, so the system can still read the file just fine, even if the directory entry has been removed. Some other operating systems, like MS-DOS and Windows 9x, do not have that distinction. On them, the directory entry has the same role as the inode on Unix-style systems, which means it cannot be removed while the file is in use. The Windows NT line, when using NTFS, does have something similar to the directory entry and inode split from Unix, but it still denies removing or replacing in-use files for compatibility with Windows 9x. > Isn't the problem actually not being able to effectively close it to then replace it? The problem is when another software (for instance, a file indexer or an antivirus) has the file open. You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot.

SAI_Peregrinus 7y ago
Or close the other program. Also, "You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot. " is somewhat wrong. You can close the handle the other program holds on the file, it just might crash that other program.
- cesarb 7y ago
  > You can close the handle the other program holds on the file, it just might crash that other program. It might do more than just crash that other program. Some other component within that program might open a new file (or even things which are not really files), and happen to get the same handle you had just closed (once the handle is closed, its number can be reused). Then the original component finishes what it was doing and closes the handle; the second component thinks it's still open. A third component opens a new file, and again gets the same handle. Now the second component tries to write to the handle (it might be something like a log file), and the write goes to the wrong place. Force closing a handle on a different process leads to undefined behavior, and as always with undefined behavior, anything can happen. Raymond Chen also has an article about this: http://technet.microsoft.com/en-us/magazine/2009.04.windowsconfidential.aspx http://technet.microsoft.com/en-us/magazine/2009.04.windowsc...

setr 7y ago

90% of my headache with it is that windows won't tell you which program(s) has it open