4 ms·

EDIT: Misinformed, see wfunction's comment below. I can understand their frustration, because this is probably really hard to fix. I guess there would not be

EDIT: Misinformed, see wfunction's comment below. I can understand their frustration, because this is probably really hard to fix. I guess there would not be problems if Windows apps were not overwriting files by first deleting them and then re-creating them with new contents. It's easy to see why this leads to data loss (file permission metadata is data in the Unix world). You rarely see Unix apps re-creating files: there you instead you specify the behaviour in case the file exists with the flags of the open(2) call. I guess such contract is absent / not as strict in the Windows world (disc: I have never programmed for Windows), which would make solving the conflicting APIs almost impossible: if a re-created file would inherit a previously deleted synonymous file's permissions, what happens to a program who is deleting a file exactly to get rid of its permissions? I think MS has done pretty impressive work with the Linux subsystem. It is certainly not a trivial task, and I am looking forward to seeing more awesome stuff running on Windows.

deleted 10y ago
[deleted]
wfunction 10y ago
> You rarely see Unix apps re-creating files: there you instead you specify the behaviour in case the file exists with the flags of the open(2) call. I think your lack of knowledge about this problem is betraying you. This is outright wrong. First, look up CreateFile() in Windows; it has flags to specify things like this as well. Second, the reason programs delete files in the first place has nothing to do with the lack of such flags. It has to do with the fact that they want to write new contents but don't want to lose data in the event of an abnormal termination. If you truncate the file that you're opening instead, then you lose that data if something goes wrong. So they create a new file and replace it with the old one when it's written. Finally, as far as I know, 'nix tools have a tendency to outright replace files MORE than Windows tools do. That's why the underlying Windows kernel API function (NtCreateFile/ZwCreateFile) has a FILE_SUPERSEDE parameter... whose entire intention is to mimic POSIX behavior: > The CreateDisposition value FILE_SUPERSEDE requires that the caller have DELETE access to a existing file object. If so, a successful call to ZwCreateFile with FILE_SUPERSEDE on an existing file effectively deletes that file, and then recreates it. [...] Note that this type of disposition is consistent with the POSIX style of overwriting files. https://msdn.microsoft.com/en-us/library/windows/hardware/ff566424.aspx https://msdn.microsoft.com/en-us/library/windows/hardware/ff...
- paulddraper 10y ago
  Exactly. Most file systems have an atomic move, that you can leverage into an "atomic write".
- vesinisa 10y ago
  Thanks for correcting. That makes it sound almost like this could be actually fixed by swimming deep enough to the NTFS driver in Windows.
  - wfunction 10y ago
    There is such a thing as NTFS tunneling, if that's what you mean. Not sure what you have in mind if you mean otherwise.
    - asveikau 10y ago
      NTFS tunneling is a hilarious feature. To those who are not well versed in this: This was a hack for 2 things that exist in the universe: 1. Apps that save their documents by "delete document, re-create document by the same name". 2. The fact that said apps (especially when you consider that they might be written for Win3.1 or DOS) could be accessing the documents by 8.3 names (eg. MICROS~1.TXT) What ntfs.sys will do is: for some period of time after deleting a file, remember the "long filename" of a corresponding short name, and re-hydrate it when you re-create. So when you delete MICROS~1.TXT and create a new MICROS~1.TXT, ntfs.sys will remember that this file is also called "Microsoft.txt" and re-create that name.
- asveikau 10y ago
  I would go so far to say (though I've met few people who grok this, as Windows filesystems is pretty rare as a specialty these days) that delete-and-recreate is a huge antipattern and recipe for failure on Windows. The reason: Delete behavior. In NT, when you delete a file and handles are open, the name sticks around until all handles are closed. Any attempt to re-open (including re-create) with the same name will fail with STATUS_DELETE_PENDING while this is the case. Combine that with the fact that most operations in NT (including delete or rename) happen via opening handles to the file first and you have lots of chaos and race conditions. So the delete + recreate pattern is very likely to bite you later as a "re-create fails randomly".
cestith 10y ago
Actually it's fairly common when making changes within a file, especially a text file, to read from one file, write the changed data stream to a new file, then do an atomic rename of the target file to the name of the source file.
pritambaral 10y ago
> You rarely see Unix apps re-creating files Vim's default behaviour is to save via complete re-creation and atomic rename. I wouldn't be surprised if other classic editors used the same technique.
- jovdg 10y ago
  That is also what "sed -i" does.
kalleboo 10y ago
> because this is probably really hard to fix. The alternative I can think of is spreading hidden metadata files all over the place, à la Apple with their .DS_Store and AppleDouble files that everyone always complains about. That comes with its own set of problems (moving files around will lose the metadata when they're no longer in the source directory along with the metadata)
userbinator 10y ago
It's easy to see why this leads to data loss (file permission metadata is data in the Unix world). Completely disagree. Considering that some file systems which have been around for decades don't even have this permissions metadata, I would say that the file contents are the most important to users. MS should absolutely give a warning that the extended metadata can be lost, but the "If WSL can’t find Linux file metadata for a given file, it assumes the file is corrupted/damaged" behaviour noted in the article is completely contrary to user expectations. The filesystem model in Linux and Windows is not identical, but there is considerable overlap. It therefore makes perfect sense --- and is expected --- for the overlapping parts to work.

deleted 10y ago

[deleted]

wfunction 10y ago

> You rarely see Unix apps re-creating files: there you instead you specify the behaviour in case the file exists with the flags of the open(2) call. I think your lack of knowledge about this problem is betraying you. This is outright wrong. First, look up CreateFile() in Windows; it has flags to specify things like this as well. Second, the reason programs delete files in the first place has nothing to do with the lack of such flags. It has to do with the fact that they want to write new contents but don't want to lose data in the event of an abnormal termination. If you truncate the file that you're opening instead, then you lose that data if something goes wrong. So they create a new file and replace it with the old one when it's written. Finally, as far as I know, 'nix tools have a tendency to outright replace files MORE than Windows tools do. That's why the underlying Windows kernel API function (NtCreateFile/ZwCreateFile) has a FILE_SUPERSEDE parameter... whose entire intention is to mimic POSIX behavior: > The CreateDisposition value FILE_SUPERSEDE requires that the caller have DELETE access to a existing file object. If so, a successful call to ZwCreateFile with FILE_SUPERSEDE on an existing file effectively deletes that file, and then recreates it. [...] Note that this type of disposition is consistent with the POSIX style of overwriting files. https://msdn.microsoft.com/en-us/library/windows/hardware/ff566424.aspx https://msdn.microsoft.com/en-us/library/windows/hardware/ff...

paulddraper 10y ago
Exactly. Most file systems have an atomic move, that you can leverage into an "atomic write".
vesinisa 10y ago
Thanks for correcting. That makes it sound almost like this could be actually fixed by swimming deep enough to the NTFS driver in Windows.
- wfunction 10y ago
  There is such a thing as NTFS tunneling, if that's what you mean. Not sure what you have in mind if you mean otherwise.
  - asveikau 10y ago
    NTFS tunneling is a hilarious feature. To those who are not well versed in this: This was a hack for 2 things that exist in the universe: 1. Apps that save their documents by "delete document, re-create document by the same name". 2. The fact that said apps (especially when you consider that they might be written for Win3.1 or DOS) could be accessing the documents by 8.3 names (eg. MICROS~1.TXT) What ntfs.sys will do is: for some period of time after deleting a file, remember the "long filename" of a corresponding short name, and re-hydrate it when you re-create. So when you delete MICROS~1.TXT and create a new MICROS~1.TXT, ntfs.sys will remember that this file is also called "Microsoft.txt" and re-create that name.
asveikau 10y ago
I would go so far to say (though I've met few people who grok this, as Windows filesystems is pretty rare as a specialty these days) that delete-and-recreate is a huge antipattern and recipe for failure on Windows. The reason: Delete behavior. In NT, when you delete a file and handles are open, the name sticks around until all handles are closed. Any attempt to re-open (including re-create) with the same name will fail with STATUS_DELETE_PENDING while this is the case. Combine that with the fact that most operations in NT (including delete or rename) happen via opening handles to the file first and you have lots of chaos and race conditions. So the delete + recreate pattern is very likely to bite you later as a "re-create fails randomly".

cestith 10y ago

Actually it's fairly common when making changes within a file, especially a text file, to read from one file, write the changed data stream to a new file, then do an atomic rename of the target file to the name of the source file.

pritambaral 10y ago

> You rarely see Unix apps re-creating files Vim's default behaviour is to save via complete re-creation and atomic rename. I wouldn't be surprised if other classic editors used the same technique.

jovdg 10y ago
That is also what "sed -i" does.

kalleboo 10y ago

> because this is probably really hard to fix. The alternative I can think of is spreading hidden metadata files all over the place, à la Apple with their .DS_Store and AppleDouble files that everyone always complains about. That comes with its own set of problems (moving files around will lose the metadata when they're no longer in the source directory along with the metadata)

userbinator 10y ago

It's easy to see why this leads to data loss (file permission metadata is data in the Unix world). Completely disagree. Considering that some file systems which have been around for decades don't even have this permissions metadata, I would say that the file contents are the most important to users. MS should absolutely give a warning that the extended metadata can be lost, but the "If WSL can’t find Linux file metadata for a given file, it assumes the file is corrupted/damaged" behaviour noted in the article is completely contrary to user expectations. The filesystem model in Linux and Windows is not identical, but there is considerable overlap. It therefore makes perfect sense --- and is expected --- for the overlapping parts to work.