4 ms·

Arguably, principle of least surprise is very Apple. If I point "tape archive" at a file system, I want that file system archived to tape. And so, tar does.

Arguably, principle of least surprise is very Apple. If I point "tape archive" at a file system, I want that file system archived to tape. And so, tar does. If I don't, well, that's a fine option, and there's a fine option for that. So it's less of a "workaround" or something that "gets worse", than, "No, I don't really want a tape archive of this filesystem, only of some of it." And that's supported. That said, never seeing another .DS_Store should be a system-wide option!

taftster 6mo ago
> That said, never seeing another .DS_Store should be a system-wide option! Yes please.
- ryandrake 6mo ago
  .DS_Store, .fseventsd, .Spotlight-V100, .Trashes, and ._this and ._that These can all die in a fire too, as far as I am concerned. macOS loves to treat the user's filesystem as its own personal garbage dump.
  - gerdesj 6mo ago
    thumbs.db and those weird MS alternative stream files for recording origination. filesystem attributes are for decorating files with meaning. Anything else that attempts to use filesystems in "interesting" ways is silly. Apple and MS really ought to consider why they do this sort of fragile, idiosyncratic nonsense.
    - Joker_vD 6mo ago
      But... thumbs.db is precisely not an "attempt to use filesystems in "interesting" ways" — it's literally a just hidden file with previews stored in it. Storing the preview in the alternative stream of the file with the picture itself would be "an interesting way".
      - kstrauser 6mo ago
        Agreed. Where else would you put that stuff? It’s gotta go somewhere, and this is the least surprising place IMO. Anywhere else would have to be a parallel store that follows filesystem mounts and unmounts, renaming directories, etc so that it alway perfectly mirrors the thing it’s configuring.
        noisem4ker 5mo ago
        > Where else would you put that stuff? A "Centralized thumbnail cache" in the user profile folder, where it's been for a long while. https://en.wikipedia.org/wiki/Windows_thumbnail_cache https://en.wikipedia.org/wiki/Windows_thumbnail_cache > so that it alway perfectly mirrors Who cares? It's a cache.
        kstrauser 5mo ago
        That windows takes an approach does not mean it’s a good idea. And what about things like folder settings, such as whether to display is as a list or as icons, or how to sort it, etc? That’s more important than a thumbnail cache.
      - mook 6mo ago
        In the particular case of thumbs.db, storing them in NTFS alternate data streams would have been a good idea; they're essentially caches for the main data stream, so if they fail to copy to different filesystems it's totally fine. Of course, that wasn't viable because 1) IIRC that was before the widespread adoption of NTFS, and 2) they probably still need the cache somewhere for vFAT USB drives.
      - fingerlocks 5mo ago
        And .DS_Store is just your folder level preferences in Finder. If you don’t use Finder they won’t be created
        taftster 5mo ago
        Yes. And truthfully, I try to remember to only ever navigate my project folders (particularly those under revision control) using command line and/or IDE folder views. But eventually, for whatever reason, I use Finder to go looking into a directory structure and bam, now I have .DS_Store. gitignore takes care of it, I know, but still, it's annoying.
    - noisem4ker 5mo ago
      > Thumbs.db Windows has been storing thumbnail cache in the user profile folder since Vista (2006). It's been 20 years. Time to let it go.
  - emmelaich 6mo ago
    OTOH, If you want the information contained in those files, where else would you save it?
    - ajxs 6mo ago
      To me it seems more sensible to store information relevant only to this OS in a specific cache somewhere within that OS. It would even make cache-like functionality such as evicting old entries super easy.
      - Gigachad 6mo ago
        There are some tradeoffs. Like if you used a usb and set up folder colours or any of the other things stored in the file, they would not move along with the usb when used on another computer.
        ajxs 5mo ago
        If I set a folder colour in Finder on my work MacBook, and then plug that USB drive into my personal computer which uses Thunar as a file browser on Debian, nothing would happen.
        Someone 5mo ago
        And? If you mount a Unix file system on another system, you may see ‘invisible’ fuels whose name starts with a period, may even see weird files named “.” or “..”, may not see ACLs, and may not see any file attributes such as user and group information. In 1970 it already was not true that one could treat all filesystems the way Unix did, but it certainly isn’t true anymore today.
      - Someone 5mo ago
        > sensible to store information relevant only to this OS in a specific cache somewhere within that OS. For most of these files, this isn’t information that can be reconstructed, so caching isn’t an option. Also, the information has to move with the disk, if it is moved to or mounted on another system.
JoshTriplett 6mo ago
> Arguably, principle of least surprise is very Apple. Principle of least surprise is good engineering practice. The question is always whose surprise. Someone who expects tar to behave like other UNIX systems is going to be surprised by this. Someone who expects tar on Apple to have perfect fidelity would be surprised by not-this. I increasingly feel like build systems should never be relying on any "native" utilities from the host system, and should instead be bringing them in via dependencies. You can't have this problem if your packaging system pulls in a specific portable `tar` library.
- amarant 6mo ago
  Nixos has a pretty solid solution to this issue: key your dependencies with checksums of the content. That way you get the best of both worlds: you always get the exact version you want, and you can share a copy of that exact version with other software that wants to use that exact version too!
  - JoshTriplett 6mo ago
    Yeah, Nix-like distributions (e.g. guix, lix) do for Linux systems what some language package managers (e.g. cargo) do for individual projects.
  - altairprime 6mo ago
    Are the xattr / chattr / umask checksums rolled into the main data fork content or are they hashed separately (or not at all)?
    - a_t48 6mo ago
      IIRC Nix is checksummed in the hash of the source of the content, not the results.
      - microtonal 6mo ago
        Hash of a normalization of the derivation, so this roughly means source, dependencies and the ‘build recipe’. The exception are fixed-output derivations, which are typically content-hashed. That said, a lot of work is done in content-addressed hashing, but AFAIK it’s not the default yet.
saghm 6mo ago
If you think that most people who run the tar command are assuming it will work like a tape archive, you'll probably be the one surprised
adrian_b 5mo ago
> I want that file system archived to tape. And so, tar does. The traditional UNIX tar and cpio utilities cannot archive the modern Linux file systems without loss of metadata. Most modern tar programs implement various file format extensions as a workaround for this, but the extensions may be incompatible between distinct tar programs and frequently they are very poorly documented. Some years in the past, libarchive was the only archiver available on Linux that guaranteed lossless backups for the Linux file systems, e.g. xfs or ext4 (and also lossless file transfers between Linux file systems and FreeBSD file systems). Therefore that is what I have been using on Linux since then. Presumably since then GNU tar and other tar programs should have caught up with it, but I have not verified this. Whichever tar program was used in TFA, it was an obsolete tar program, and that was the real problem, not that the archives had been created on an Apple computer.

taftster 6mo ago

> That said, never seeing another .DS_Store should be a system-wide option! Yes please.

ryandrake 6mo ago
.DS_Store, .fseventsd, .Spotlight-V100, .Trashes, and ._this and ._that These can all die in a fire too, as far as I am concerned. macOS loves to treat the user's filesystem as its own personal garbage dump.
- gerdesj 6mo ago
  thumbs.db and those weird MS alternative stream files for recording origination. filesystem attributes are for decorating files with meaning. Anything else that attempts to use filesystems in "interesting" ways is silly. Apple and MS really ought to consider why they do this sort of fragile, idiosyncratic nonsense.
  - Joker_vD 6mo ago
    But... thumbs.db is precisely not an "attempt to use filesystems in "interesting" ways" — it's literally a just hidden file with previews stored in it. Storing the preview in the alternative stream of the file with the picture itself would be "an interesting way".
    - kstrauser 6mo ago
      Agreed. Where else would you put that stuff? It’s gotta go somewhere, and this is the least surprising place IMO. Anywhere else would have to be a parallel store that follows filesystem mounts and unmounts, renaming directories, etc so that it alway perfectly mirrors the thing it’s configuring.
      - noisem4ker 5mo ago
        > Where else would you put that stuff? A "Centralized thumbnail cache" in the user profile folder, where it's been for a long while. https://en.wikipedia.org/wiki/Windows_thumbnail_cache https://en.wikipedia.org/wiki/Windows_thumbnail_cache > so that it alway perfectly mirrors Who cares? It's a cache.
        kstrauser 5mo ago
        That windows takes an approach does not mean it’s a good idea. And what about things like folder settings, such as whether to display is as a list or as icons, or how to sort it, etc? That’s more important than a thumbnail cache.
    - mook 6mo ago
      In the particular case of thumbs.db, storing them in NTFS alternate data streams would have been a good idea; they're essentially caches for the main data stream, so if they fail to copy to different filesystems it's totally fine. Of course, that wasn't viable because 1) IIRC that was before the widespread adoption of NTFS, and 2) they probably still need the cache somewhere for vFAT USB drives.
    - fingerlocks 5mo ago
      And .DS_Store is just your folder level preferences in Finder. If you don’t use Finder they won’t be created
      - taftster 5mo ago
        Yes. And truthfully, I try to remember to only ever navigate my project folders (particularly those under revision control) using command line and/or IDE folder views. But eventually, for whatever reason, I use Finder to go looking into a directory structure and bam, now I have .DS_Store. gitignore takes care of it, I know, but still, it's annoying.
  - noisem4ker 5mo ago
    > Thumbs.db Windows has been storing thumbnail cache in the user profile folder since Vista (2006). It's been 20 years. Time to let it go.
- emmelaich 6mo ago
  OTOH, If you want the information contained in those files, where else would you save it?
  - ajxs 6mo ago
    To me it seems more sensible to store information relevant only to this OS in a specific cache somewhere within that OS. It would even make cache-like functionality such as evicting old entries super easy.
    - Gigachad 6mo ago
      There are some tradeoffs. Like if you used a usb and set up folder colours or any of the other things stored in the file, they would not move along with the usb when used on another computer.
      - ajxs 5mo ago
        If I set a folder colour in Finder on my work MacBook, and then plug that USB drive into my personal computer which uses Thunar as a file browser on Debian, nothing would happen.
        Someone 5mo ago
        And? If you mount a Unix file system on another system, you may see ‘invisible’ fuels whose name starts with a period, may even see weird files named “.” or “..”, may not see ACLs, and may not see any file attributes such as user and group information. In 1970 it already was not true that one could treat all filesystems the way Unix did, but it certainly isn’t true anymore today.
    - Someone 5mo ago
      > sensible to store information relevant only to this OS in a specific cache somewhere within that OS. For most of these files, this isn’t information that can be reconstructed, so caching isn’t an option. Also, the information has to move with the disk, if it is moved to or mounted on another system.

JoshTriplett 6mo ago

> Arguably, principle of least surprise is very Apple. Principle of least surprise is good engineering practice. The question is always whose surprise. Someone who expects tar to behave like other UNIX systems is going to be surprised by this. Someone who expects tar on Apple to have perfect fidelity would be surprised by not-this. I increasingly feel like build systems should never be relying on any "native" utilities from the host system, and should instead be bringing them in via dependencies. You can't have this problem if your packaging system pulls in a specific portable `tar` library.

amarant 6mo ago
Nixos has a pretty solid solution to this issue: key your dependencies with checksums of the content. That way you get the best of both worlds: you always get the exact version you want, and you can share a copy of that exact version with other software that wants to use that exact version too!
- JoshTriplett 6mo ago
  Yeah, Nix-like distributions (e.g. guix, lix) do for Linux systems what some language package managers (e.g. cargo) do for individual projects.
- altairprime 6mo ago
  Are the xattr / chattr / umask checksums rolled into the main data fork content or are they hashed separately (or not at all)?
  - a_t48 6mo ago
    IIRC Nix is checksummed in the hash of the source of the content, not the results.
    - microtonal 6mo ago
      Hash of a normalization of the derivation, so this roughly means source, dependencies and the ‘build recipe’. The exception are fixed-output derivations, which are typically content-hashed. That said, a lot of work is done in content-addressed hashing, but AFAIK it’s not the default yet.

saghm 6mo ago

If you think that most people who run the tar command are assuming it will work like a tape archive, you'll probably be the one surprised

adrian_b 5mo ago

> I want that file system archived to tape. And so, tar does. The traditional UNIX tar and cpio utilities cannot archive the modern Linux file systems without loss of metadata. Most modern tar programs implement various file format extensions as a workaround for this, but the extensions may be incompatible between distinct tar programs and frequently they are very poorly documented. Some years in the past, libarchive was the only archiver available on Linux that guaranteed lossless backups for the Linux file systems, e.g. xfs or ext4 (and also lossless file transfers between Linux file systems and FreeBSD file systems). Therefore that is what I have been using on Linux since then. Presumably since then GNU tar and other tar programs should have caught up with it, but I have not verified this. Whichever tar program was used in TFA, it was an obsolete tar program, and that was the real problem, not that the archives had been created on an Apple computer.