6 ms·

> OS X has a fundamentally unreliable filesystem Unfortunately so is every other filesystem. Which is also why Apple heavily emphasizes making backups.

dman 12y ago
Have you looked at ZFS?
- coldtea 12y ago
  Have YOU looked at ZFS, and how it's unfit for desktop use?
  - laumars 12y ago
    That's unsubstantiated FUD. You only have to glance through this thread to see that there are people (myself included) who run ZFS on their desktop machine (laptop in my case). People love to over exaggerate the RAM requirements for ZFS because they see build logs of enterprise-grade storage servers. But you wouldn't expect that kind of throughput on your desktop systems even with lighter file systems. And most desktops have their 64bit CPU sat idle for most of the time, so there's not even an argument for the additional instruction overhead. Obviously the real crux of argument is "what do you primarily use your desktop for"? If it's gaming, then there's little point running ZFS since you're going to be disinterested in the benefits of ZFS anyway, so it's not worth the marginal overhead. However if your desktop is a development machine, used for multimedia authoring or even just an internet terminal (like most PCs are these days), then ZFS is a viable option.
    - coldtea 12y ago
      >And most desktops have their 64bit CPU sat idle for most of the time, so there's not even an argument for the additional instruction overhead. Most "desktops" today are in fact laptops, and the reason you want to have the CPU sit idle is battery time. >However if your desktop is a development machine, used for multimedia authoring or even just an internet terminal (like most PCs are these days), then ZFS is a viable option. If you're not pooling disks together but just have the built in disk, and usb hard disks that you conenct occasionally you don't get much if anything from ZFS, but you get the penalties it incurs... And it's not a substitute for backing up your disks either...
      - laumars 12y ago
        > Most "desktops" today are in fact laptops, and the reason you want to have the CPU sit idle is battery time. That's a fair point. I'd argue that ZFS doesn't chuck that much extra CPU time that most people would notice. However if you're one of those guys that likes to tweak your power settings to the nth degree, then ZFS probably isn't best. But those kinds of people are more likely to run a tiling WM instead of a full fat compositing desktop environment; and such like. So they're definitely not your regular desktop users > If you're not pooling disks together but just have the built in disk, and usb hard disks that you conenct occasionally you don't get much if anything from ZFS, but you get the penalties it incurs... You seem to have missed quite a number of useful ZFS features off your list. Ignoring deduplication (as that is extremely memory and CPU hungry), ZFS has snapshotting, checksumming, superblock recovery (even if your journal gets trashed, you can recover it), online data integrity checks (no annoying wait screens for fsck to run through), better compression algorithms than is supported in most other file systems... I could go on, but suffice to say that ZFS is an improvement over many other file systems in other areas than just software RAIDing. That all said, in an ideal world desktops would be running HAMMER (DragonflyBSD's file system) over ZFS. I do love ZFS, but HAMMER seems a slightly better fit for desktops / laptops as it has the aforementioned features but without many of the data centre stuff that home users wouldn't need. A better compromise. However I don't see much interest for HAMMER outside of Dragonfly. > And it's not a substitute for backing up your disks either... Nobody suggested it was. However ZFS does also have backup features built into it too.
santaclaus 12y ago
Yea but there is a continuum of reliability -- ZFS is way better than HFS+ in this regard. Also, Btrfs.
- 286c8cb04bda 12y ago
  I've used btrfs (at several-petabytes-of-data scale) and wouldn't trust it any more than HFS+. Use ZFS if you like, or XFS, or even ext4 instead.
  - yungchin 12y ago
    Would love to hear more about your findings with btrfs.
    - 286c8cb04bda 12y ago
      The biggest recurring issue was deadlocks in the garbage collector. It would start cleanup in a subvolume and trip all over itself. After that, any I/O to that specific directory would never return. The only solutions was to reboot the server and fsck for a few hours. Second frequent problem: Hitting 90% capacity in a filesystem has a non-trivial chance to ruin it forever. Hit the wrong code path, and, even if you immediately delete a bunch of things, I/O to that filesystem would be forever 3000% slower.
  - diroussel 12y ago
    I would also like to learn more about your view of btrfs. Is it a fundamental problem? Or an implementation, or community issue you have?
nodata 12y ago
What are you talking about? HFS is creaky and ancient and doesn't have checksums, ZFS and btrfs do.
- Someone 12y ago
  The way I see it: HFS+ is creaky, ancient, but battle-hardened. ZFS is solid and battle-hardened, but not that suitable for many of Apple's customers, who want the ability to unmount disks they added, and who will not understand that they need huge amounts of RAM (they their Macoboks don't have) to keep their disks speedy, btrfs is promising but not battle-hardened yet, and it is very unlikely Apple will use it as long as it has a GPL license (I think it is more likely that Apple will add checksums to HFS+ in some somewhat clumsy, but working, way than that it starts supporting btrfs)
  - rodgerd 12y ago
    > HFS+ is creaky, ancient, but battle-hardened. If you think HFS and HFS+ are battle-hardened I don't know what to say to you. "Shell-shocked" would be a better description.
    - threeseed 12y ago
      Tens of millions of users have been using HFS for decades mostly with zero issues. I personally have used a Mac since the Mac+ and not one had corruption due to the filesystem. I would describe it as battle hardened for sure.
      - rodgerd 12y ago
        > I personally have used a Mac since the Mac+ and not one had corruption due to the filesystem. You're a very lucky man, then. A large part of my time, at one point in my life, was spent fixing up corrupted HFS filesystems.
      - acdha 12y ago
        “mostly with zero issues” is the same as saying “with issues”. I would generally agree that it's rare for e.g. HFS to corrupt its own metadata due to bugs but bit-rot happens. If you think you haven't encountered it, you probably haven't actually checked your files closely enough – this is particularly easy to miss with things like video or image formats which were designed to tolerate a few bit-flips. The reason why so many Mac users were interested in ZFS was that a modern filesystem incorporates strong integrity checks so you are guaranteed to either read back the bytes you originally wrote or get an error. That's a much better way to work than requiring you to use a tool which does strong hashes periodically to see if anything has broken.
ChrisLTD 12y ago
A backup doesn't help you if the photo is corrupted, and then that corrupted photo is propagated to your backups. You need a way to catch file corruption, like what ZFS and Btrfs have.
- vacri 12y ago
  btrfs still isn't 'production ready', unfortunately.
  - samf 12y ago
    ZFS is, and Apple almost had it. Damn.
    - coldtea 12y ago
      ZFS is totally broken for regular desktop use. It's use case is a different thing altogether. Do people just read the "pros" and not the "cons"?
      - nroach 12y ago
        Have you tried https://openzfsonosx.org https://openzfsonosx.org ? I use it daily on my primary desktop for my home directory and with an external time machine backup drive and have had zero problems.
      - laumars 12y ago
        ZFS doesn't have a specific "use case". In fact a large amount of effort went into ensuring that ZFS was tolerant of consumer hardware, so that makes it as sensible option for desktop use as ext4 or HFS+ (when looking strictly at "use cases"). You've also failed to quantify why ZFS is "totally broken for regular desktop use" (likely because you've never actually used ZFS so can only blow smoke about it's issues). People love to over exaggerate the RAM requirements for ZFS because they see build logs of enterprise-grade storage servers. But you wouldn't expect that kind of throughput on your desktop systems even with lighter file systems. And most desktops have their 64bit CPU sat idle for most of the time, so there's not even an argument for the additional instruction overhead. Obviously the real crux of argument is "what do you primarily use your desktop for"? If it's gaming, then there's little point running ZFS since you're going to be disinterested in the benefits of ZFS (plus likely running Windows anyway). However if your desktop is a development machine, used for multimedia authoring or even just an internet terminal (like most PCs are these days), then ZFS is a viable option.
galago 12y ago
I've head John Siracusa discuss this, but I can't find a good link at the moment. Yes, with cloud storage systems, there is a chance that with HFS+ you're just backing up garbage and possibly won't notice until you use the file. HFS+ is not really equivalent to its peers.
- 72deluxe 12y ago
  Here you are, it's from the Lion review I think. Pretty scary reading, and particularly since we've had numerous releases since then with not many changes: http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/ http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/

dman 12y ago

Have you looked at ZFS?

coldtea 12y ago
Have YOU looked at ZFS, and how it's unfit for desktop use?
- laumars 12y ago
  That's unsubstantiated FUD. You only have to glance through this thread to see that there are people (myself included) who run ZFS on their desktop machine (laptop in my case). People love to over exaggerate the RAM requirements for ZFS because they see build logs of enterprise-grade storage servers. But you wouldn't expect that kind of throughput on your desktop systems even with lighter file systems. And most desktops have their 64bit CPU sat idle for most of the time, so there's not even an argument for the additional instruction overhead. Obviously the real crux of argument is "what do you primarily use your desktop for"? If it's gaming, then there's little point running ZFS since you're going to be disinterested in the benefits of ZFS anyway, so it's not worth the marginal overhead. However if your desktop is a development machine, used for multimedia authoring or even just an internet terminal (like most PCs are these days), then ZFS is a viable option.
  - coldtea 12y ago
    >And most desktops have their 64bit CPU sat idle for most of the time, so there's not even an argument for the additional instruction overhead. Most "desktops" today are in fact laptops, and the reason you want to have the CPU sit idle is battery time. >However if your desktop is a development machine, used for multimedia authoring or even just an internet terminal (like most PCs are these days), then ZFS is a viable option. If you're not pooling disks together but just have the built in disk, and usb hard disks that you conenct occasionally you don't get much if anything from ZFS, but you get the penalties it incurs... And it's not a substitute for backing up your disks either...
    - laumars 12y ago
      > Most "desktops" today are in fact laptops, and the reason you want to have the CPU sit idle is battery time. That's a fair point. I'd argue that ZFS doesn't chuck that much extra CPU time that most people would notice. However if you're one of those guys that likes to tweak your power settings to the nth degree, then ZFS probably isn't best. But those kinds of people are more likely to run a tiling WM instead of a full fat compositing desktop environment; and such like. So they're definitely not your regular desktop users > If you're not pooling disks together but just have the built in disk, and usb hard disks that you conenct occasionally you don't get much if anything from ZFS, but you get the penalties it incurs... You seem to have missed quite a number of useful ZFS features off your list. Ignoring deduplication (as that is extremely memory and CPU hungry), ZFS has snapshotting, checksumming, superblock recovery (even if your journal gets trashed, you can recover it), online data integrity checks (no annoying wait screens for fsck to run through), better compression algorithms than is supported in most other file systems... I could go on, but suffice to say that ZFS is an improvement over many other file systems in other areas than just software RAIDing. That all said, in an ideal world desktops would be running HAMMER (DragonflyBSD's file system) over ZFS. I do love ZFS, but HAMMER seems a slightly better fit for desktops / laptops as it has the aforementioned features but without many of the data centre stuff that home users wouldn't need. A better compromise. However I don't see much interest for HAMMER outside of Dragonfly. > And it's not a substitute for backing up your disks either... Nobody suggested it was. However ZFS does also have backup features built into it too.

santaclaus 12y ago

Yea but there is a continuum of reliability -- ZFS is way better than HFS+ in this regard. Also, Btrfs.

286c8cb04bda 12y ago
I've used btrfs (at several-petabytes-of-data scale) and wouldn't trust it any more than HFS+. Use ZFS if you like, or XFS, or even ext4 instead.
- yungchin 12y ago
  Would love to hear more about your findings with btrfs.
  - 286c8cb04bda 12y ago
    The biggest recurring issue was deadlocks in the garbage collector. It would start cleanup in a subvolume and trip all over itself. After that, any I/O to that specific directory would never return. The only solutions was to reboot the server and fsck for a few hours. Second frequent problem: Hitting 90% capacity in a filesystem has a non-trivial chance to ruin it forever. Hit the wrong code path, and, even if you immediately delete a bunch of things, I/O to that filesystem would be forever 3000% slower.
- diroussel 12y ago
  I would also like to learn more about your view of btrfs. Is it a fundamental problem? Or an implementation, or community issue you have?

nodata 12y ago

What are you talking about? HFS is creaky and ancient and doesn't have checksums, ZFS and btrfs do.

Someone 12y ago
The way I see it: HFS+ is creaky, ancient, but battle-hardened. ZFS is solid and battle-hardened, but not that suitable for many of Apple's customers, who want the ability to unmount disks they added, and who will not understand that they need huge amounts of RAM (they their Macoboks don't have) to keep their disks speedy, btrfs is promising but not battle-hardened yet, and it is very unlikely Apple will use it as long as it has a GPL license (I think it is more likely that Apple will add checksums to HFS+ in some somewhat clumsy, but working, way than that it starts supporting btrfs)
- rodgerd 12y ago
  > HFS+ is creaky, ancient, but battle-hardened. If you think HFS and HFS+ are battle-hardened I don't know what to say to you. "Shell-shocked" would be a better description.
  - threeseed 12y ago
    Tens of millions of users have been using HFS for decades mostly with zero issues. I personally have used a Mac since the Mac+ and not one had corruption due to the filesystem. I would describe it as battle hardened for sure.
    - rodgerd 12y ago
      > I personally have used a Mac since the Mac+ and not one had corruption due to the filesystem. You're a very lucky man, then. A large part of my time, at one point in my life, was spent fixing up corrupted HFS filesystems.
    - acdha 12y ago
      “mostly with zero issues” is the same as saying “with issues”. I would generally agree that it's rare for e.g. HFS to corrupt its own metadata due to bugs but bit-rot happens. If you think you haven't encountered it, you probably haven't actually checked your files closely enough – this is particularly easy to miss with things like video or image formats which were designed to tolerate a few bit-flips. The reason why so many Mac users were interested in ZFS was that a modern filesystem incorporates strong integrity checks so you are guaranteed to either read back the bytes you originally wrote or get an error. That's a much better way to work than requiring you to use a tool which does strong hashes periodically to see if anything has broken.

ChrisLTD 12y ago

A backup doesn't help you if the photo is corrupted, and then that corrupted photo is propagated to your backups. You need a way to catch file corruption, like what ZFS and Btrfs have.

vacri 12y ago
btrfs still isn't 'production ready', unfortunately.
- samf 12y ago
  ZFS is, and Apple almost had it. Damn.
  - coldtea 12y ago
    ZFS is totally broken for regular desktop use. It's use case is a different thing altogether. Do people just read the "pros" and not the "cons"?
    - nroach 12y ago
      Have you tried https://openzfsonosx.org https://openzfsonosx.org ? I use it daily on my primary desktop for my home directory and with an external time machine backup drive and have had zero problems.
    - laumars 12y ago
      ZFS doesn't have a specific "use case". In fact a large amount of effort went into ensuring that ZFS was tolerant of consumer hardware, so that makes it as sensible option for desktop use as ext4 or HFS+ (when looking strictly at "use cases"). You've also failed to quantify why ZFS is "totally broken for regular desktop use" (likely because you've never actually used ZFS so can only blow smoke about it's issues). People love to over exaggerate the RAM requirements for ZFS because they see build logs of enterprise-grade storage servers. But you wouldn't expect that kind of throughput on your desktop systems even with lighter file systems. And most desktops have their 64bit CPU sat idle for most of the time, so there's not even an argument for the additional instruction overhead. Obviously the real crux of argument is "what do you primarily use your desktop for"? If it's gaming, then there's little point running ZFS since you're going to be disinterested in the benefits of ZFS (plus likely running Windows anyway). However if your desktop is a development machine, used for multimedia authoring or even just an internet terminal (like most PCs are these days), then ZFS is a viable option.

galago 12y ago

I've head John Siracusa discuss this, but I can't find a good link at the moment. Yes, with cloud storage systems, there is a chance that with HFS+ you're just backing up garbage and possibly won't notice until you use the file. HFS+ is not really equivalent to its peers.

72deluxe 12y ago
Here you are, it's from the Lion review I think. Pretty scary reading, and particularly since we've had numerous releases since then with not many changes: http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/ http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/