5 ms·

At least 20% of my grief while using Windows stems from not being able to remove or replace open files.

by oefrha 7y ago

Kye 7y ago
Sometimes Windows will release a stuck file if you kill explorer and restart it from task manager. I found it mostly happens while working with files in a program while the folder is open in explorer.
SketchySeaBeast 7y ago
As a native windows user I don't really understand this grief - if the file is open we're kind of assuming we need it, aren't we? What happens if the system tries to read the file on disc and it's been replaced or removed? Why shouldn't that end in an earth-shattering kaboom? Isn't the problem actually not being able to effectively close it to then replace it?
- kevin_thibedeau 7y ago
  This was solved in Unix land 50 years ago. Unlinked handles remain usable until the process releases them but the filesystem isn't held hostage. Doesn't require hunt the process to find out who has a file open. The OS can update itself live without a reboot.
- pjc50 7y ago
  There are three kinds of semantics for non-collaborating programs that people want from files that are completely incompatible: - update immediately(ish) seen by other users of the file (e.g. database); the default UNIX semantics, even for executables - update seen by other users at the next point they open the file (what you want for update-without reboot, can be achieved on UNIX because deleted inodes hang around) - it's completely impossible for other users to write, rename etc the file (Windows default)
  - SketchySeaBeast 7y ago
    Thanks! I guess each come with their own tradeoffs - immediate updates have a good chance of blowing away the current users work as well as continually searching for file updates, and a versioning system then needs to figure out what to do when the user saves a stale file.
    - BubRoss 7y ago
      gvim already handles this. If a file changes it prompts you to update or not. If you try to save an old file over a new one, it makes you confirm what you are doing.
      - SketchySeaBeast 7y ago
        As it should, and as all apps have to, otherwise you end up with problems in any shared workspace.
- BubRoss 7y ago
  A lot of times the problem actually comes down to not being able to find out what process is keeping a file open and/or not being able to close the process or file.
  - SketchySeaBeast 7y ago
    That is true. We've all been there with someone having a word file open somewhere on the network.
  - SAI_Peregrinus 7y ago
    Process Explorer is the workarond. Use its search for handle, find the filename. You'll get a list of all handles to the file, and thus all processes with it open. You can close those (safest way) or force-close the handle (more likely to crash the process in question).
    - oefrha 7y ago
      I use Process Explorer when I have to, but 1. It’s not built into Windows, requires separate download; 2. Searching is hit or miss.
      - SAI_Peregrinus 7y ago
        That's why I call it a workaround, not a solution. It's good enough most of the time, but not every time.
- vetinari 7y ago
  This is easy, actually. File and it's directory entry are two separate things. So you can remove and replace open file, but the old handle will still point to the old file - it just won't be open-able using the original filename. Once all the processes will close the handle to the old file, and no directory entry points to it, it will be removed from the file system.
  - SketchySeaBeast 7y ago
    What happens if the user tries to save the stale file? They have to give it a new name?
    - pjc50 7y ago
      Default UNIX semantics: you don't actually know this has happened to a file you're using on most filesystems, so you can continue writing to it and it will vanish on close. Many programs do the "write to temp file and rename on top of old file" trick, which is almost-atomic, and you can have multiple programs doing that to the same file at the same time without noticing. (If you're on NFS writing to a deleted inode may do something different and probably broken)
      - SketchySeaBeast 7y ago
        Doesn't that end up being dangerous as far as updates? You keep blowing away the file? I think part of my problem is I'm visualizing this as a shared file being updated by users, and not, say, system updates. System updates make sense - we can assume only the system will be modifying them, and it'll be relatively low touch, but I keep visualizing a shared text document.
        SAI_Peregrinus 7y ago
        Yes, it's very dangerous. Files in Unix-style systems can have invisibly shared mutable state. Like having shared memory without any mutexes.
        7y ago
- oefrha 7y ago
  When a file is renamed or removed (replace is basically remove followed by rename -- often in one atomic action, of course), the open file handle and the corresponding inode can be left around and remain usable until the handle is closed. That is orthogonal to whether the filesystem path should be bound to the inode throughout its lifetime. On *ix systems it's not, so for instance a program could easily replace itself with a new version. > Isn't the problem actually not being able to effectively close it to then replace it? Not sure what this is supposed to mean. The program that's keeping the file open could effectively close it. The problem is other programs can't, plus it's hard to hunt down the process that's hanging on to a file.
- snoopen 7y ago
  there's a lot of instances where the system can the file locked when you don't think you have it open. preview pane in file explorer used to open an instance of the relevant software to generate a preview thereby locking the file. most Adobe products can fail to release a lock. weird filenames can prevent deletion in file explorer, but cmd can delete. sometimes windows locks the file while a thumbnail is being generated. these are the things that keep me up at night
- alpaca128 7y ago
  This makes sense in theory, unless it's like on Windows where the file/storage is accessed by some unknown, hidden process that you don't know anything about except that it really likes to use this specific file, and you only have the choice between shutting down the computer to safely remove some thumb drive or to just pull it and hope for the best.
- cesarb 7y ago
  > if the file is open we're kind of assuming we need it, aren't we? What happens if the system tries to read the file on disc and it's been replaced or removed? Why shouldn't that end in an earth-shattering kaboom? On Unix, the directory entry does not point directly to the file contents. It points to an "inode", which contains the file metadata (owner, permissions, timestamps) and points to the file contents. When you remove a file, you are actually removing the directory entry; the inode will only be removed when nothing else points to or is using it. An open file keeps a reference to the inode, so the system can still read the file just fine, even if the directory entry has been removed. Some other operating systems, like MS-DOS and Windows 9x, do not have that distinction. On them, the directory entry has the same role as the inode on Unix-style systems, which means it cannot be removed while the file is in use. The Windows NT line, when using NTFS, does have something similar to the directory entry and inode split from Unix, but it still denies removing or replacing in-use files for compatibility with Windows 9x. > Isn't the problem actually not being able to effectively close it to then replace it? The problem is when another software (for instance, a file indexer or an antivirus) has the file open. You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot.
  - SAI_Peregrinus 7y ago
    Or close the other program. Also, "You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot. " is somewhat wrong. You can close the handle the other program holds on the file, it just might crash that other program.
    - cesarb 7y ago
      > You can close the handle the other program holds on the file, it just might crash that other program. It might do more than just crash that other program. Some other component within that program might open a new file (or even things which are not really files), and happen to get the same handle you had just closed (once the handle is closed, its number can be reused). Then the original component finishes what it was doing and closes the handle; the second component thinks it's still open. A third component opens a new file, and again gets the same handle. Now the second component tries to write to the handle (it might be something like a log file), and the write goes to the wrong place. Force closing a handle on a different process leads to undefined behavior, and as always with undefined behavior, anything can happen. Raymond Chen also has an article about this: http://technet.microsoft.com/en-us/magazine/2009.04.windowsconfidential.aspx http://technet.microsoft.com/en-us/magazine/2009.04.windowsc...
- setr 7y ago
  90% of my headache with it is that windows won't tell you which program(s) has it open
pjmlp 7y ago
Which happens to be what every sane OS besides UNIX does.
- ubercow13 7y ago
  As a user, every interaction I have had with Windows' file locking semantics have not made me think it was 'sane'
  - pjmlp 7y ago
    First, there are more OSes out there besides Windows and UNIX. Examples of insane behavior of UNIX's lack of locking: - A process can get its file corrupted because flock() is cooperative, so there isn't any enforcement that even if file locking is used, it actually works. Hence the dance with temporary lockfiles as workaround. - A process can happily store important information into a file just to have it deleted under its feet after the usage count of the inode goes to zero, and good luck bringing the data back to life without forensics tools. Quite sane indeed.
    - ubercow13 7y ago
      Yes, but as a user rather than developer, I have never had a problem with it in practice, unlike Windows. And it is honestly much more convenient (as a user) to be able to manipulate open documents on the filesystem if you can keep in mind how the semantics work.
      - pjmlp 7y ago
        The user of a corrupted document won't be very happy when it happens.
        7786655 7y ago
        I've been using Linux for years and have encountered this problem exactly zero times.
userbinator 7y ago
You can rename them, which is the way "in-place" updates like these are usually done - the old file is renamed, the new file is copied into place, and an entry is set up in the registry to delete the old one upon the next reboot.
YeGoblynQueenne 7y ago
More than 20% of my grief comes from having to confirm the change of a file's extension, every time. This happens when I start a script or a LaTex file etc by right-clicking in an empty folder to create a new text file, then changing the name of the file and its extension. It's an empty file with a .txt extension, yet windows has to ask me whether I'm sure I really want to perform this desperate and dangerous act that might cause some programs to not work any more.

At least 20% of my grief while using Windows stems from not being able to remove or replace open files.

by oefrha 7y ago

At least 20% of my grief while using Windows stems from not being able to remove or replace open files.

Kye 7y ago

Sometimes Windows will release a stuck file if you kill explorer and restart it from task manager. I found it mostly happens while working with files in a program while the folder is open in explorer.

SketchySeaBeast 7y ago

As a native windows user I don't really understand this grief - if the file is open we're kind of assuming we need it, aren't we? What happens if the system tries to read the file on disc and it's been replaced or removed? Why shouldn't that end in an earth-shattering kaboom? Isn't the problem actually not being able to effectively close it to then replace it?

kevin_thibedeau 7y ago
This was solved in Unix land 50 years ago. Unlinked handles remain usable until the process releases them but the filesystem isn't held hostage. Doesn't require hunt the process to find out who has a file open. The OS can update itself live without a reboot.
pjc50 7y ago
There are three kinds of semantics for non-collaborating programs that people want from files that are completely incompatible: - update immediately(ish) seen by other users of the file (e.g. database); the default UNIX semantics, even for executables - update seen by other users at the next point they open the file (what you want for update-without reboot, can be achieved on UNIX because deleted inodes hang around) - it's completely impossible for other users to write, rename etc the file (Windows default)
- SketchySeaBeast 7y ago
  Thanks! I guess each come with their own tradeoffs - immediate updates have a good chance of blowing away the current users work as well as continually searching for file updates, and a versioning system then needs to figure out what to do when the user saves a stale file.
  - BubRoss 7y ago
    gvim already handles this. If a file changes it prompts you to update or not. If you try to save an old file over a new one, it makes you confirm what you are doing.
    - SketchySeaBeast 7y ago
      As it should, and as all apps have to, otherwise you end up with problems in any shared workspace.
BubRoss 7y ago
A lot of times the problem actually comes down to not being able to find out what process is keeping a file open and/or not being able to close the process or file.
- SketchySeaBeast 7y ago
  That is true. We've all been there with someone having a word file open somewhere on the network.
- SAI_Peregrinus 7y ago
  Process Explorer is the workarond. Use its search for handle, find the filename. You'll get a list of all handles to the file, and thus all processes with it open. You can close those (safest way) or force-close the handle (more likely to crash the process in question).
  - oefrha 7y ago
    I use Process Explorer when I have to, but 1. It’s not built into Windows, requires separate download; 2. Searching is hit or miss.
    - SAI_Peregrinus 7y ago
      That's why I call it a workaround, not a solution. It's good enough most of the time, but not every time.
vetinari 7y ago
This is easy, actually. File and it's directory entry are two separate things. So you can remove and replace open file, but the old handle will still point to the old file - it just won't be open-able using the original filename. Once all the processes will close the handle to the old file, and no directory entry points to it, it will be removed from the file system.
- SketchySeaBeast 7y ago
  What happens if the user tries to save the stale file? They have to give it a new name?
  - pjc50 7y ago
    Default UNIX semantics: you don't actually know this has happened to a file you're using on most filesystems, so you can continue writing to it and it will vanish on close. Many programs do the "write to temp file and rename on top of old file" trick, which is almost-atomic, and you can have multiple programs doing that to the same file at the same time without noticing. (If you're on NFS writing to a deleted inode may do something different and probably broken)
    - SketchySeaBeast 7y ago
      Doesn't that end up being dangerous as far as updates? You keep blowing away the file? I think part of my problem is I'm visualizing this as a shared file being updated by users, and not, say, system updates. System updates make sense - we can assume only the system will be modifying them, and it'll be relatively low touch, but I keep visualizing a shared text document.
      - SAI_Peregrinus 7y ago
        Yes, it's very dangerous. Files in Unix-style systems can have invisibly shared mutable state. Like having shared memory without any mutexes.
      - 7y ago
oefrha 7y ago
When a file is renamed or removed (replace is basically remove followed by rename -- often in one atomic action, of course), the open file handle and the corresponding inode can be left around and remain usable until the handle is closed. That is orthogonal to whether the filesystem path should be bound to the inode throughout its lifetime. On *ix systems it's not, so for instance a program could easily replace itself with a new version. > Isn't the problem actually not being able to effectively close it to then replace it? Not sure what this is supposed to mean. The program that's keeping the file open could effectively close it. The problem is other programs can't, plus it's hard to hunt down the process that's hanging on to a file.
snoopen 7y ago
there's a lot of instances where the system can the file locked when you don't think you have it open. preview pane in file explorer used to open an instance of the relevant software to generate a preview thereby locking the file. most Adobe products can fail to release a lock. weird filenames can prevent deletion in file explorer, but cmd can delete. sometimes windows locks the file while a thumbnail is being generated. these are the things that keep me up at night
alpaca128 7y ago
This makes sense in theory, unless it's like on Windows where the file/storage is accessed by some unknown, hidden process that you don't know anything about except that it really likes to use this specific file, and you only have the choice between shutting down the computer to safely remove some thumb drive or to just pull it and hope for the best.
cesarb 7y ago
> if the file is open we're kind of assuming we need it, aren't we? What happens if the system tries to read the file on disc and it's been replaced or removed? Why shouldn't that end in an earth-shattering kaboom? On Unix, the directory entry does not point directly to the file contents. It points to an "inode", which contains the file metadata (owner, permissions, timestamps) and points to the file contents. When you remove a file, you are actually removing the directory entry; the inode will only be removed when nothing else points to or is using it. An open file keeps a reference to the inode, so the system can still read the file just fine, even if the directory entry has been removed. Some other operating systems, like MS-DOS and Windows 9x, do not have that distinction. On them, the directory entry has the same role as the inode on Unix-style systems, which means it cannot be removed while the file is in use. The Windows NT line, when using NTFS, does have something similar to the directory entry and inode split from Unix, but it still denies removing or replacing in-use files for compatibility with Windows 9x. > Isn't the problem actually not being able to effectively close it to then replace it? The problem is when another software (for instance, a file indexer or an antivirus) has the file open. You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot.
- SAI_Peregrinus 7y ago
  Or close the other program. Also, "You cannot force the other program to release its grip on the file; the workaround is to set the "pending delete" flag for the file and schedule a reboot. " is somewhat wrong. You can close the handle the other program holds on the file, it just might crash that other program.
  - cesarb 7y ago
    > You can close the handle the other program holds on the file, it just might crash that other program. It might do more than just crash that other program. Some other component within that program might open a new file (or even things which are not really files), and happen to get the same handle you had just closed (once the handle is closed, its number can be reused). Then the original component finishes what it was doing and closes the handle; the second component thinks it's still open. A third component opens a new file, and again gets the same handle. Now the second component tries to write to the handle (it might be something like a log file), and the write goes to the wrong place. Force closing a handle on a different process leads to undefined behavior, and as always with undefined behavior, anything can happen. Raymond Chen also has an article about this: http://technet.microsoft.com/en-us/magazine/2009.04.windowsconfidential.aspx http://technet.microsoft.com/en-us/magazine/2009.04.windowsc...
setr 7y ago
90% of my headache with it is that windows won't tell you which program(s) has it open

pjmlp 7y ago

Which happens to be what every sane OS besides UNIX does.

ubercow13 7y ago
As a user, every interaction I have had with Windows' file locking semantics have not made me think it was 'sane'
- pjmlp 7y ago
  First, there are more OSes out there besides Windows and UNIX. Examples of insane behavior of UNIX's lack of locking: - A process can get its file corrupted because flock() is cooperative, so there isn't any enforcement that even if file locking is used, it actually works. Hence the dance with temporary lockfiles as workaround. - A process can happily store important information into a file just to have it deleted under its feet after the usage count of the inode goes to zero, and good luck bringing the data back to life without forensics tools. Quite sane indeed.
  - ubercow13 7y ago
    Yes, but as a user rather than developer, I have never had a problem with it in practice, unlike Windows. And it is honestly much more convenient (as a user) to be able to manipulate open documents on the filesystem if you can keep in mind how the semantics work.
    - pjmlp 7y ago
      The user of a corrupted document won't be very happy when it happens.
      - 7786655 7y ago
        I've been using Linux for years and have encountered this problem exactly zero times.

userbinator 7y ago

You can rename them, which is the way "in-place" updates like these are usually done - the old file is renamed, the new file is copied into place, and an entry is set up in the registry to delete the old one upon the next reboot.

YeGoblynQueenne 7y ago

More than 20% of my grief comes from having to confirm the change of a file's extension, every time. This happens when I start a script or a LaTex file etc by right-clicking in an empty folder to create a new text file, then changing the name of the file and its extension. It's an empty file with a .txt extension, yet windows has to ask me whether I'm sure I really want to perform this desperate and dangerous act that might cause some programs to not work any more.