5 ms·

...or maybe Windows should just offer an API for marking a file for deletion once it's not in use anymore (I understand unlink semantics may not be possible, bu

by cafxx 3y ago

...or maybe Windows should just offer an API for marking a file for deletion once it's not in use anymore (I understand unlink semantics may not be possible, but that's not what my suggestion above is saying)

deleted 3y ago
[deleted]
fodkodrasz 3y ago
I thought I'll be the guy to point out that once again mandatory file locking is to blame, but you beat me to it. I never digged into the question, but why is it used, what benefits did it provide over the UNIX unlink behaviour?
- dataflow 3y ago
  > I never digged into the question, but why is it used, what benefits did it provide over the UNIX unlink behaviour? How do you defragment/move files that are unreachable on the file system? How do you shrink volumes when you can't move files that need to be moved? Edit: Actually, hmm... as I type this, I suddenly recall you can also open a file by its ID on NTFS. And you can enumerate its streams as well. So presumably this could work on NTFS if you loop through all file IDs? Though then that would make these files still accessible, not truly unreachable.
  - tedunangst 3y ago
    Using the same API that lets you move file blocks around at will.
    - dataflow 3y ago
      Huh? That API requires a file handle. Which you get by opening a file. Which you can't do because you can't find it on the filesystem when it's not there. Edit: Actually, hmm... see edit above.
      - yrro 3y ago
        While a process still has an unlinked file open, /proc/<pid>/fd can be used to obtain a handle to the file so that you can mess around with it.
        dataflow 3y ago
        You're suggesting opening every single FD of every single process (which might not even point to a file, let alone a file on that volume) and querying it just to do this? I mean, sure, I guess that's usually not physically impossible (unless e.g. /proc is unavailable/unmounted)... but it's clearly a hack. In fact, I think it's not just a (slow!) hack, but a buggy one too. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
        yrro 3y ago
        No I'm just saying it's possible. I can count on the fingers of 0 hands the number of times I've needed to do this to edit a deleted file out from under a process that has the only reference to an unlinked file open so at least in my experience it's merely acadamic knowledge!
  - fodkodrasz 3y ago
    You don't? Those are either free space, or held by handle by a running process, so you just leave them be and assume they will be released sooner or later. Worst case you defragment on boot. https://unix.stackexchange.com/questions/68523/find-and-remove-large-files-that-are-open-but-have-been-deleted https://unix.stackexchange.com/questions/68523/find-and-remo... This is how it works on UNIX. Generally better then apps randomly failing because a file(name) is held open somewhere by something.
    - dataflow 3y ago
      > You don't? Those are either free space, or held by handle by a running process, so you just leave them be and assume they will be released sooner or later. Well that's what I was getting at, it would suck to not be able to move around file blocks just because a process is using the file. That "sooner or later" might well be "until the next reboot". The current strategy makes it possible to live-shrink and live-defragment volumes on Windows - ironically, saving you a reboot in those cases compared to Linux. But actually, maybe not - see the edit in my original comment.
      - fodkodrasz 3y ago
        I'm yet to want to defrag my computer and worrying about still open deleted files. I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often, and annoys me more. (or being unable to replace a running service binary of a service being developed/tested, needing to stop the service, replace it, and start again. Or failing log rotations because a logfile is open in a Notepad. Or...) Also see my link for a solution on unix, where you can indeed fix this problem, or simply kill the process holding the file. I didn't need to defrag my computer in the last 20 years, neither on Linux, nor on Windows, but hey, it makes me happy that my daily work is hindered for this hypothetical possibility. (which could and is solved in other OSs with appropriate APIs for the job) Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
        dataflow 3y ago
- pjmlp 3y ago
  Locking mechanism that actually works, like in any sane OS besides UNIX. And with it, less data corruption issues.
CodeHz 3y ago
Windows do have this API, NtDeleteFile, AND it could be used to delete current running exe. https://twitter.com/jonaslyk/status/1345167613643661312 https://twitter.com/jonaslyk/status/1345167613643661312 but it is undocumented...

by cafxx 3y ago

deleted 3y ago

[deleted]

fodkodrasz 3y ago

I thought I'll be the guy to point out that once again mandatory file locking is to blame, but you beat me to it. I never digged into the question, but why is it used, what benefits did it provide over the UNIX unlink behaviour?

dataflow 3y ago
> I never digged into the question, but why is it used, what benefits did it provide over the UNIX unlink behaviour? How do you defragment/move files that are unreachable on the file system? How do you shrink volumes when you can't move files that need to be moved? Edit: Actually, hmm... as I type this, I suddenly recall you can also open a file by its ID on NTFS. And you can enumerate its streams as well. So presumably this could work on NTFS if you loop through all file IDs? Though then that would make these files still accessible, not truly unreachable.
- tedunangst 3y ago
  Using the same API that lets you move file blocks around at will.
  - dataflow 3y ago
    Huh? That API requires a file handle. Which you get by opening a file. Which you can't do because you can't find it on the filesystem when it's not there. Edit: Actually, hmm... see edit above.
    - yrro 3y ago
      While a process still has an unlinked file open, /proc/<pid>/fd can be used to obtain a handle to the file so that you can mess around with it.
      - dataflow 3y ago
        You're suggesting opening every single FD of every single process (which might not even point to a file, let alone a file on that volume) and querying it just to do this? I mean, sure, I guess that's usually not physically impossible (unless e.g. /proc is unavailable/unmounted)... but it's clearly a hack. In fact, I think it's not just a (slow!) hack, but a buggy one too. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
        yrro 3y ago
        No I'm just saying it's possible. I can count on the fingers of 0 hands the number of times I've needed to do this to edit a deleted file out from under a process that has the only reference to an unlinked file open so at least in my experience it's merely acadamic knowledge!
- fodkodrasz 3y ago
  You don't? Those are either free space, or held by handle by a running process, so you just leave them be and assume they will be released sooner or later. Worst case you defragment on boot. https://unix.stackexchange.com/questions/68523/find-and-remove-large-files-that-are-open-but-have-been-deleted https://unix.stackexchange.com/questions/68523/find-and-remo... This is how it works on UNIX. Generally better then apps randomly failing because a file(name) is held open somewhere by something.
  - dataflow 3y ago
    > You don't? Those are either free space, or held by handle by a running process, so you just leave them be and assume they will be released sooner or later. Well that's what I was getting at, it would suck to not be able to move around file blocks just because a process is using the file. That "sooner or later" might well be "until the next reboot". The current strategy makes it possible to live-shrink and live-defragment volumes on Windows - ironically, saving you a reboot in those cases compared to Linux. But actually, maybe not - see the edit in my original comment.
    - fodkodrasz 3y ago
      I'm yet to want to defrag my computer and worrying about still open deleted files. I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often, and annoys me more. (or being unable to replace a running service binary of a service being developed/tested, needing to stop the service, replace it, and start again. Or failing log rotations because a logfile is open in a Notepad. Or...) Also see my link for a solution on unix, where you can indeed fix this problem, or simply kill the process holding the file. I didn't need to defrag my computer in the last 20 years, neither on Linux, nor on Windows, but hey, it makes me happy that my daily work is hindered for this hypothetical possibility. (which could and is solved in other OSs with appropriate APIs for the job) Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
      - dataflow 3y ago
pjmlp 3y ago
Locking mechanism that actually works, like in any sane OS besides UNIX. And with it, less data corruption issues.

CodeHz 3y ago

Windows do have this API, NtDeleteFile, AND it could be used to delete current running exe. https://twitter.com/jonaslyk/status/1345167613643661312 https://twitter.com/jonaslyk/status/1345167613643661312 but it is undocumented...