4 ms·

What do you mean by perfect copies here? Do you mean the file content itself or are you also including the filesystem attributes related to the file in your def

What do you mean by perfect copies here? Do you mean the file content itself or are you also including the filesystem attributes related to the file in your definition?

adrian_b 2y ago
A file consists of data and various metadata, e.g. file name, timestamps, access rights, user-defined file attributes. By default, a file copy should include everything that is contained in the original file. Sometimes the destination file system cannot store all the original metadata, but in such cases a file copying utility must give a warning that some file metadata has been lost, e.g. like when copying to a FAT file system or to a tmpfs file system as implemented by older Linux kernels. (Many file copy or archiving utilities fail to warn the user when metadata cannot be preserved.) Some times you may no longer need some of the file metadata, but the user should be the one who chooses to loose some information, it should not be the default behavior, especially when this unexpected behavior is not advertised anywhere in the documentation. The origin of the problem is that the old UNIX file systems did not support many kinds of modern file metadata, i.e. they did not have access control lists or extended file attributes and the file timestamps had a very low resolution. When the file systems were modernized (XFS was the first Linux file system supporting such features, then slowly also the other file systems were modernized), most UNIX utilities have not been updated until many years later, and even then the additional features remained disabled by default. Copying like rsync, between different computers, creates additional problems, because even if e.g. both Windows and Linux have extended file attributes, access control lists and high-resolution file timestamps, the APIs used for accessing file metadata differ between operating systems, so a utility like rsync must contain code able to handle all such APIs, otherwise it will not be able to preserve all file metadata.
- scrapheap 2y ago
  But what you're referring to here are the attributes that the file system stores about the file, not the file itself. By default I wouldn't expect a copy of a file to have identical file system attributes, just an identical content for the file. I would expect some of the file system attributes to be copied, but not all of them. Take the file owner for example if I take a copy of a file then by default I should be the owner of that file as it's my copy of the file, and not the original file owner's copy. An alternative way of looking at it is if I have created a file on my local machine that's owned by root and has the setuid bit set on it's file permissions then there's no way that I should be able to copy that file up to a server with my normal user account and have those atttibutes still set on the copy.
  - LoganDark 2y ago
    "File" means an entry in the file system, and so includes the metadata. It is not only the data. When a copy a file you will be the owner because the new copy is your copy. Other attributes however like modification date for example will remain the same. It's not as if you wrote the contents of the file anew, especially not for copy-on-write architectures like Apple's APFS.
    - scrapheap 2y ago
      So you also would expect some of the file system attributes to be copied, but not all of them. :D
      - LoganDark 2y ago
        I expect all of them to be copied except for specifically the owner and group. Created date, modified date, ACLs, extended attributes, eeeverything else. My expectations are more specific than "not all of them", so please don't misrepresent them.
        scrapheap 2y ago
        Out of interest, why wouldn't you expect the created timestamp for a file that you've created by copying another file to be the point in time which the copy was made? After all, before that moment the file didn't exist, and after that moment it did.
        LoganDark 2y ago
        macOS has "date added" for this, which is the date the file was added to its containing folder. It's not the exact same as the date created that you're talking about, though. I honestly don't have a strong preference either way on this. I don't use date created except for misbehaving media downloaders that think the file modified date is a good place to put the video publication date. I'm sure there's a flag somewhere that I don't care enough to find.
- prmoustache 2y ago
  The cp command does copy the file data but not the metadata. There is a reason we have come up with 2 words to distinguish them. Rsync only cp the metadata when you specifically ask it to anyway. I haven't had a look at openrsync man page but I would assume it is the same in the case of the later.
  - adrian_b 2y ago
    Nope. Openrsync lacks the options of rsync for making exact copies. Moreover, the OpenBSD file systems are unable to store all metadata that can accompany files in Linux filesystems or Windows filesystems, so that is the likely reason for removing the rsync options. I also doubt that the developers of an utility for OpenBSD are also interested in taking care to preserve file metadata when copying to/from Windows, because the metadata access API is not portable, so a complete "rsync" utility must include specific code paths at least for Windows, for Linux and for FreeBSD. I do not know if the API of MacOS is also specific to it, or it is compatible with anything else.
fhars 2y ago
It maens that if you copy a file from NTFS to ext4, ext4 will magically sprout support for alternate data streams.
- johnisgood 2y ago
  And all files from NTFS have +x. :|

What do you mean by perfect copies here? Do you mean the file content itself or are you also including the filesystem attributes related to the file in your definition?

adrian_b 2y ago

A file consists of data and various metadata, e.g. file name, timestamps, access rights, user-defined file attributes. By default, a file copy should include everything that is contained in the original file. Sometimes the destination file system cannot store all the original metadata, but in such cases a file copying utility must give a warning that some file metadata has been lost, e.g. like when copying to a FAT file system or to a tmpfs file system as implemented by older Linux kernels. (Many file copy or archiving utilities fail to warn the user when metadata cannot be preserved.) Some times you may no longer need some of the file metadata, but the user should be the one who chooses to loose some information, it should not be the default behavior, especially when this unexpected behavior is not advertised anywhere in the documentation. The origin of the problem is that the old UNIX file systems did not support many kinds of modern file metadata, i.e. they did not have access control lists or extended file attributes and the file timestamps had a very low resolution. When the file systems were modernized (XFS was the first Linux file system supporting such features, then slowly also the other file systems were modernized), most UNIX utilities have not been updated until many years later, and even then the additional features remained disabled by default. Copying like rsync, between different computers, creates additional problems, because even if e.g. both Windows and Linux have extended file attributes, access control lists and high-resolution file timestamps, the APIs used for accessing file metadata differ between operating systems, so a utility like rsync must contain code able to handle all such APIs, otherwise it will not be able to preserve all file metadata.

scrapheap 2y ago
But what you're referring to here are the attributes that the file system stores about the file, not the file itself. By default I wouldn't expect a copy of a file to have identical file system attributes, just an identical content for the file. I would expect some of the file system attributes to be copied, but not all of them. Take the file owner for example if I take a copy of a file then by default I should be the owner of that file as it's my copy of the file, and not the original file owner's copy. An alternative way of looking at it is if I have created a file on my local machine that's owned by root and has the setuid bit set on it's file permissions then there's no way that I should be able to copy that file up to a server with my normal user account and have those atttibutes still set on the copy.
- LoganDark 2y ago
  "File" means an entry in the file system, and so includes the metadata. It is not only the data. When a copy a file you will be the owner because the new copy is your copy. Other attributes however like modification date for example will remain the same. It's not as if you wrote the contents of the file anew, especially not for copy-on-write architectures like Apple's APFS.
  - scrapheap 2y ago
    So you also would expect some of the file system attributes to be copied, but not all of them. :D
    - LoganDark 2y ago
      I expect all of them to be copied except for specifically the owner and group. Created date, modified date, ACLs, extended attributes, eeeverything else. My expectations are more specific than "not all of them", so please don't misrepresent them.
      - scrapheap 2y ago
        Out of interest, why wouldn't you expect the created timestamp for a file that you've created by copying another file to be the point in time which the copy was made? After all, before that moment the file didn't exist, and after that moment it did.
        LoganDark 2y ago
        macOS has "date added" for this, which is the date the file was added to its containing folder. It's not the exact same as the date created that you're talking about, though. I honestly don't have a strong preference either way on this. I don't use date created except for misbehaving media downloaders that think the file modified date is a good place to put the video publication date. I'm sure there's a flag somewhere that I don't care enough to find.
prmoustache 2y ago
The cp command does copy the file data but not the metadata. There is a reason we have come up with 2 words to distinguish them. Rsync only cp the metadata when you specifically ask it to anyway. I haven't had a look at openrsync man page but I would assume it is the same in the case of the later.
- adrian_b 2y ago
  Nope. Openrsync lacks the options of rsync for making exact copies. Moreover, the OpenBSD file systems are unable to store all metadata that can accompany files in Linux filesystems or Windows filesystems, so that is the likely reason for removing the rsync options. I also doubt that the developers of an utility for OpenBSD are also interested in taking care to preserve file metadata when copying to/from Windows, because the metadata access API is not portable, so a complete "rsync" utility must include specific code paths at least for Windows, for Linux and for FreeBSD. I do not know if the API of MacOS is also specific to it, or it is compatible with anything else.

fhars 2y ago

It maens that if you copy a file from NTFS to ext4, ext4 will magically sprout support for alternate data streams.

johnisgood 2y ago
And all files from NTFS have +x. :|