4 ms·

I don't accept the premise that filesystems are "unreliable" because you experienced a case of data loss. Hard drives can and regularly do experience hardware f

by abalone 12y ago

I don't accept the premise that filesystems are "unreliable" because you experienced a case of data loss. Hard drives can and regularly do experience hardware failures even with the best possible filesystem software. Camera memory cards can experience data loss before you get around to uploading. Reliability is like security; it's a matter of degrees not an absolute measure. That is why a good, well-integrated offsite backup solution that "reduces the window of a chance for corruption" is a much more significant impact on data security than anything we could hope to incrementally improve in the filesystem software layer. And Apple's solution is even better than you describe. Photos taken from Apple cameras (which are the vast bulk of photos for most customers) are automatically uploaded directly to the cloud as soon as the device joins wifi.

binarycrusader 12y ago
And you seem to be completely ignoring the fact that a filesystem that can't even guarantee that it wrote the data to an intermediate storage device correctly is fundamentally unreliable. That is the core issue. Yes, well-integrated offline backup solutions mitigate some aspects of the problem, but when your filesystem can't even guarantee that it did its job correctly, it doesn't matter how well-integrated your "backup" solutions are because the data may already be corrupt. Nevermind that the "well-integrated" backup solution you are describing here, because of the unreliable filesystem, can't even guarantee that it read the data from disk correctly to a reasonable degree. This is not some anecdotal "Mac OS X burned me" story -- this is a fundamental, architectural issue. Data loss is real -- and so are my concerns about HFS+.
- millstone 12y ago
  Why do you think HFS+ can't guarantee that it wrote data to a storage device? From what I know (F_FULLFSYNC, etc) it does have that capability.
  - MCRed 12y ago
    And why isn't he giving more details? He's being kinda broad without being specific enough. At this point my guess is he's been running HFS+ without journaling turned on (you can format drives this way) and experienced data loss as a result because the filesystems ability to correct itself was disabled. This would look like what he's describing.
    - binarycrusader 12y ago
      No, I'm using the defaults for what OS X specifies when you first install. Which means journaling is turned on. See John Siracusa's article: http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/ http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/
  - binarycrusader 12y ago
    Guarantee that it wrote the data correctly. Think checksums, like what ZFS/Btrfs have: http://en.wikipedia.org/wiki/Data_corruption#SILENT http://en.wikipedia.org/wiki/Data_corruption#SILENT Same goes for when it reads back the data.
    - millstone 12y ago
      I'm sorry to hear you lost data due to that issue. NTFS, ext4, and other consumer-targeted filesystems do not have this feature either, so it's unfair to imply this is somehow specific to HFS+ or Macs. You'd have the same issue on a Windows or Ubuntu install.
      - binarycrusader 12y ago
        Actually, as of Windows 8.1 (64-bit only), ReFS is now supported which does have those features: https://en.wikipedia.org/wiki/ReFS https://en.wikipedia.org/wiki/ReFS So in the near future, Windows will have an answer for the general mainstream, while OS X will be left far behind. Even ignoring the checksumming issue, HFS+ has known pathologies not only in reliability but in performance compared to NTFS or even ext4 on Linux. HFS+ is long overdue for replacement.

by abalone 12y ago

binarycrusader 12y ago

And you seem to be completely ignoring the fact that a filesystem that can't even guarantee that it wrote the data to an intermediate storage device correctly is fundamentally unreliable. That is the core issue. Yes, well-integrated offline backup solutions mitigate some aspects of the problem, but when your filesystem can't even guarantee that it did its job correctly, it doesn't matter how well-integrated your "backup" solutions are because the data may already be corrupt. Nevermind that the "well-integrated" backup solution you are describing here, because of the unreliable filesystem, can't even guarantee that it read the data from disk correctly to a reasonable degree. This is not some anecdotal "Mac OS X burned me" story -- this is a fundamental, architectural issue. Data loss is real -- and so are my concerns about HFS+.

millstone 12y ago
Why do you think HFS+ can't guarantee that it wrote data to a storage device? From what I know (F_FULLFSYNC, etc) it does have that capability.
- MCRed 12y ago
  And why isn't he giving more details? He's being kinda broad without being specific enough. At this point my guess is he's been running HFS+ without journaling turned on (you can format drives this way) and experienced data loss as a result because the filesystems ability to correct itself was disabled. This would look like what he's describing.
  - binarycrusader 12y ago
    No, I'm using the defaults for what OS X specifies when you first install. Which means journaling is turned on. See John Siracusa's article: http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/ http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/
- binarycrusader 12y ago
  Guarantee that it wrote the data correctly. Think checksums, like what ZFS/Btrfs have: http://en.wikipedia.org/wiki/Data_corruption#SILENT http://en.wikipedia.org/wiki/Data_corruption#SILENT Same goes for when it reads back the data.
  - millstone 12y ago
    I'm sorry to hear you lost data due to that issue. NTFS, ext4, and other consumer-targeted filesystems do not have this feature either, so it's unfair to imply this is somehow specific to HFS+ or Macs. You'd have the same issue on a Windows or Ubuntu install.
    - binarycrusader 12y ago
      Actually, as of Windows 8.1 (64-bit only), ReFS is now supported which does have those features: https://en.wikipedia.org/wiki/ReFS https://en.wikipedia.org/wiki/ReFS So in the near future, Windows will have an answer for the general mainstream, while OS X will be left far behind. Even ignoring the checksumming issue, HFS+ has known pathologies not only in reliability but in performance compared to NTFS or even ext4 on Linux. HFS+ is long overdue for replacement.