6 ms·

> The article speculates on why Apple integrates the SSD controller onto the SOC for their A and M series chips, but misses one big reason, data integrity. If

> The article speculates on why Apple integrates the SSD controller onto the SOC for their A and M series chips, but misses one big reason, data integrity. If they're really interested with data integrity they should add checksums to APFS. If you don't have RAID you can't rebuild corrupted data, but at least you know there's a problem and perhaps restore from Time Machine. For metadata, you may have multiple copies, so can use a known-good one (this is how ZFS works: some things have multiple copies 'inherently' because they're so important). Edit: > Apple File System uses checksums to ensure data integrity for metadata but not for the actual user data, relying instead on error-correcting code (ECC) mechanisms in the storage hardware.[18] * https://en.wikipedia.org/wiki/Apple_File_System#Data_integrity https://en.wikipedia.org/wiki/Apple_File_System#Data_integri...

sneak 1y ago
You can do this yourself in userspace if you really want it: https://git.eeqj.de/sneak/attrsum https://git.eeqj.de/sneak/attrsum I use zfs where I can (it has content checksums) but it sucks bad on macOS, so I wrote attrsum. It keeps the file content checksum in an xattr (which APFS (and ext3/4) supports). I use it to protect my photo library on a huge external SSD formatted with APFS (encrypted, natch) because I need to mount it on a mac laptop for Lightroom.
- arm 1y ago
  A similar alternative is Howard Oakley’s Dintch/Fintch/cintch: https://eclecticlight.co/dintch/ https://eclecticlight.co/dintch/
GeekyBear 1y ago
> If they're really interested with data integrity they should add checksums to APFS. Or you can spend half a billion dollars to solve the issue in hardware. As one of the creators of ZFS wrote when APFS was announced: > Explicitly not checksumming user data is a little more interesting. The APFS engineers I talked to cited strong ECC protection within Apple storage devices. Both NAND flash SSDs and magnetic media HDDs use redundant data to detect and correct errors. The Apple engineers contend that Apple devices basically don't return bogus data. https://arstechnica.com/gadgets/2016/06/a-zfs-developers-analysis-of-the-good-and-bad-in-apples-new-apfs-file-system/ https://arstechnica.com/gadgets/2016/06/a-zfs-developers-ana... APFS keeps redundant copies and checksums for metadata, but doesn't constantly checksum files looking for changes any more than NTFS does.
- sitkack 1y ago
  That is a weak excuse to rely on data integrity in the hardware. They most likely had that feature and removed it so they wouldn't be liable for a class action lawsuit when it turns out the NAND ages out due to bug in the retention algorithm. NTFS is what, 35 years old at this point? Odd comparison.
  - GeekyBear 1y ago
    The point is that NTFS makes zero effort to maintain file integrity at any level. Handling file integrity at the hardware level is a big step up.
    - sitkack 1y ago
      NTFS is an ancient but well designed file system, a more apt comparison would be ZFS or BTRFS both of which checksum metadata and data.
- protimewaster 1y ago
  That solution doesn't help anyone who's using external storage, though, so it kinda feels like a half billion dollars spent on a limited solution.
  - 1over137 1y ago
    Apple does not care about external storage at all, as in external disks. They offer iCloud for external storage. They don't sell external disks. They don't like cables. They make lots of money selling you a bigger internal disk.
    - realityking 1y ago
      Their store has a whole section dedicated to storage, most of it external. https://www.apple.com/shop/mac/accessories/storage https://www.apple.com/shop/mac/accessories/storage
  - 1y ago
bell-cot 1y ago
Worth noting, for ZFS - you can use the "copies" property of the dataset to save 2 or (usually) 3 separate copies of your data to the drive(s).
brookst 1y ago
Believing that giant companies are monolithic “theys” leads to all sorts of fallacies. Odds are very good that totally different people work on the architecture of AFS and SoC design.
- dylan604 1y ago
  Even still, those people report to people that report to people until you eventually get to the person in charge of the full product.

sneak 1y ago

You can do this yourself in userspace if you really want it: https://git.eeqj.de/sneak/attrsum https://git.eeqj.de/sneak/attrsum I use zfs where I can (it has content checksums) but it sucks bad on macOS, so I wrote attrsum. It keeps the file content checksum in an xattr (which APFS (and ext3/4) supports). I use it to protect my photo library on a huge external SSD formatted with APFS (encrypted, natch) because I need to mount it on a mac laptop for Lightroom.

arm 1y ago
A similar alternative is Howard Oakley’s Dintch/Fintch/cintch: https://eclecticlight.co/dintch/ https://eclecticlight.co/dintch/

GeekyBear 1y ago

> If they're really interested with data integrity they should add checksums to APFS. Or you can spend half a billion dollars to solve the issue in hardware. As one of the creators of ZFS wrote when APFS was announced: > Explicitly not checksumming user data is a little more interesting. The APFS engineers I talked to cited strong ECC protection within Apple storage devices. Both NAND flash SSDs and magnetic media HDDs use redundant data to detect and correct errors. The Apple engineers contend that Apple devices basically don't return bogus data. https://arstechnica.com/gadgets/2016/06/a-zfs-developers-analysis-of-the-good-and-bad-in-apples-new-apfs-file-system/ https://arstechnica.com/gadgets/2016/06/a-zfs-developers-ana... APFS keeps redundant copies and checksums for metadata, but doesn't constantly checksum files looking for changes any more than NTFS does.

sitkack 1y ago
That is a weak excuse to rely on data integrity in the hardware. They most likely had that feature and removed it so they wouldn't be liable for a class action lawsuit when it turns out the NAND ages out due to bug in the retention algorithm. NTFS is what, 35 years old at this point? Odd comparison.
- GeekyBear 1y ago
  The point is that NTFS makes zero effort to maintain file integrity at any level. Handling file integrity at the hardware level is a big step up.
  - sitkack 1y ago
    NTFS is an ancient but well designed file system, a more apt comparison would be ZFS or BTRFS both of which checksum metadata and data.
protimewaster 1y ago
That solution doesn't help anyone who's using external storage, though, so it kinda feels like a half billion dollars spent on a limited solution.
- 1over137 1y ago
  Apple does not care about external storage at all, as in external disks. They offer iCloud for external storage. They don't sell external disks. They don't like cables. They make lots of money selling you a bigger internal disk.
  - realityking 1y ago
    Their store has a whole section dedicated to storage, most of it external. https://www.apple.com/shop/mac/accessories/storage https://www.apple.com/shop/mac/accessories/storage
- 1y ago

bell-cot 1y ago

Worth noting, for ZFS - you can use the "copies" property of the dataset to save 2 or (usually) 3 separate copies of your data to the drive(s).

brookst 1y ago

Believing that giant companies are monolithic “theys” leads to all sorts of fallacies. Odds are very good that totally different people work on the architecture of AFS and SoC design.

dylan604 1y ago
Even still, those people report to people that report to people until you eventually get to the person in charge of the full product.