10 ms·

Linus Torvalds on HFS+

177 points by kannonboy 12y ago

kannonboy 12y ago
Scroll down for Linus' commentary. > "The true horrors of HFS+ are not in how it's not a great > filesystem, but in how it's actively designed to be a bad > filesystem by people who thought they had good ideas." There doesn't seem to be a way to deep link to comments in G+?
- astrodust 12y ago
  G+ is a train wreck for a number of reasons, this included. Yeah, he's not a fan of HFS+ at all. Wasn't the plan to move to ZFS prior to the Oracle acquisition of Sun? Hopefully that ends up back on track somehow.
  - iwwr 12y ago
    Isn't there a problem with ZFS' memory footprint to be usable for desktop systems?
    - Malic 12y ago
      It seems there was a licensing problem using ZFS on OSX: http://arstechnica.com/apple/2009/10/apple-abandons-zfs-on-mac-os-x-project-over-licensing-issues/ http://arstechnica.com/apple/2009/10/apple-abandons-zfs-on-m...
    - mindajar 12y ago
      ZFS works a lot better if you dedicate lots of memory to it, yes. Apple sells far more iOS devices than Macs these days, and the iOS filesystem is case-sensitive HFS+. It's hard to imagine Apple doing something new in filesystems that doesn't improve the platform they make the most money from. In a way, the same logic applies to Macs. The number of Macs that support multiple internal disks will soon be zero if it isn't already, so some of the best features of ZFS are irrelevant on new Apple computers.
      - stephenr 12y ago
        With thunderbolt ports on very Mac why should storage be limited to internal devices?
        kstenerud 12y ago
        Because they want you on iCloud. You can't even install the OS without an Apple ID and an active internet connection anymore.
        nizmow 12y ago
        This is extremely false.
        kstenerud 12y ago
        Which part? The iCloud part or the OS install part? I just finished reinstalling Yosemite an hour ago after wiping the HD of a friend's new laptop. It would not allow me to get past the eligibility check until I logged in with a valid Apple ID.
        X-Istence 12y ago
        Your friend may have had Find my Mac turned on. At that point the only way to get an OS installed on it is to have the original apple ID it was turned on under.
        kstenerud 12y ago
        Nope. She never used it. It was the initial "determining the eligibility of this machine" phase where it asked me to log in (originally, this was to check that you'd indeed purchased OSX Lion, but the OS is "free" now). If "Find my Mac" is turned on, it won't even let you wipe the hard drive.
        12y ago
  - exelius 12y ago
    ZFS on OS X was abandoned several years ago. Unless Apple has a new filesystem under development as a skunkworks project, HFS+ is the filesystem on OS X for the foreseeable future. And even if they did build a new FS, there are some assumptions like case insensitivity that likely couldn't be abandoned without potentially breaking backwards compatibility.
    - cmurf 12y ago
      And in effect ZFS is forked. There's openzfs and Oracle ZFS, so if Apple were to go with ZFS, which one would they pick? The smart choice would be openzfs which almost certainly means they end up licensing Oracle's ZFS at which point it's like, classic Apple - let's try whenever possible to make interoperability either difficult or hilarious. Apple's Core Storage does have some COW properties, so they could probably do something like linux LVM thin provisioning style snapshots. They could maybe add a tree for checksums, which would be a good idea for the journal at least but seeing as this hasn't been a priority for even just the journal, the most unencumbered and recent add-on, it seems obviously not a priority. The COW stuff was necessary to automagically migrate data from the SSD to the HDD in their so called "fusion drive", so maybe they leverage that for an updated version of Time Machine and could also do it for atomic online OS/software update with an optional later reboot rather than a mandatory one. One rather cool thing it does is on-the-fly encryption and decryption of an LV. You can even reboot in the middle of either conversion and it resumes where it left off. And those encrypted volumes can be resized (that's not well documented but is supported and is even used by their own Boot Camp Assistant). So about the limit of the new fs is an LVM like thing under journaled-HFS+/X.
  - fubarred 12y ago
    Only if AAPL acquires/merges with ORCL. 3rd-party: there's the non-FUSE port https://openzfsonosx.org/ https://openzfsonosx.org/
  - donavanm 12y ago
    Project predated oracle, back in the osx 10.5-10.6 days when there was the dtrace integration as well. There were internal builds of the zfs kext and at one point I saw leaked source on the Internet. Project was abandoned because of legals disagreements over cddl licensing, as I recall.
    - nobleach 12y ago
      It was in the dropdown list in Disk Utilities in Leopard developer betas. I never actually tried using it.
    - the_why_of_y 12y ago
      CDDL licensing? more likely NetApp scared Apple off with patent aggression http://en.swpat.org/wiki/NetApp%27s_filesystem_patents http://en.swpat.org/wiki/NetApp%27s_filesystem_patents
deleted 12y ago
[deleted]
wiremine 12y ago
"Quite frankly, HFS+ is probably the worst filesystem ever." Can anyone summarize why he thinks this?
- kannonboy 12y ago
  You might've missed Linus' second comment on the page, wherein he talks about case-insensitivity and NFD normalization.
- wazoox 12y ago
  HFS+ has been patched with duct tape and pieces of cardboard for 20 years; receiving journaling, support for Unix attributes, extended attributes, 64 bits sizing, multi-processing, multi-users, hard links, etc. over the years. Really it's a sort of monument to kludge. It should have been ditched like 10 years ago.
  - mikeash 12y ago
    HFS+ is basically the Apple equivalent of an alternate universe where Microsoft just keeps gluing new features on to FAT forever and never creates NTFS.
  - verbatim 12y ago
    I'm not sure that's terribly different from ext4's lineage, which is generally accepted (AFAIK) as a pretty good file system, even if not cutting-edge.
- yarrel 12y ago
  Because he hasn't used MFS. ;-)
deleted 12y ago
[deleted]
Someone1234 12y ago
Can someone explain to me why case insensitivity is a bad thing? Clearly Linus believes so but didn't explain why he believes so. Most UNIX and Linux systems seem to have an "all lowercase" or "all uppercase" convention, so the fact that they have case sensitivity is often not utilised. In fact the biggest reason you'd want case sensitivity off the top of my head is legacy support but that's just a circular argument (since you never really reach WHY it was that way originally, just that it was). I guess based on what he talks about next he is worried about how case insensitivity interacts with other character sets (i.e. does it correctly change their case), but for most sets isn't the lower and upper case defined in the UNICODE language spec itself?
- bigdubs 12y ago
  I've run into issues with it from a programming point of view, you have to make sure to do a (slightly) more expensive case insensitive comparison when dealing with filenames.
  - Someone1234 12y ago
    This is only an issue when it is fake case insensitive. In the sense that it claims it is case insensitive but in fact is case sensitive (and returns case sensitive filenames). NTFS is a perfect example of this, it claims to be case insensitive but in fact stores everything in a case sensitive way (so you wind up doing things like ToUpper() on every filename). On a legitimately case insensitive filesystem both "etc" and "ETC" would return identical filenames as they would be stored identically in the file system's database (e.g. would return "etc" for both). So what you're really giving is not an example of why case insensitivity is bad, but why half measures are bad.
    - Arnavion 12y ago
      >NTFS is a perfect example of this, it claims to be case insensitive Where does it claim this? In any case this is not true because... >but in fact stores everything in a case sensitive way This does not have to be true. NTFS can be case-sensitive or case-insensitive depending on which subsystem is accessing it. For example, while the POSIX subsystem for Windows was a thing, installing it had the side-effect of turning NTFS case-sensitive.
terminus 12y ago
Also a longer critique by John Siracusa here: http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/#hfs-problems http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/#hfs-p... I wonder how this ties in with the whole Apple philosophy of "Design is how it works." Clearly the innards look nothing like the facade.
- yuhong 12y ago
  Wonder why the "HFS+ Private Data" hack was chosen when Rhapsody existed even back in 1997.
- Someone1234 12y ago
  > File system metadata structures in HFS+ have global locks. Only one process can update the file system at a time. Holy heck! How does that work in practice? Do operations get queued and then a single kernel process who can take the lock do the atomic updates? I have to imagine that is going to cause a bottleneck however, as all non-read operations need to update the metadata (e.g. timestamp, maybe size if it is stored). That all being said I haven't noticed OS X being particularly slower to do things than e.g. Windows. So if that is the case they're hiding it well.
  - mindajar 12y ago
    HFS+ is an oddball filesystem among FSes that anyone actually uses. Since the earliest days of the Mac, it's been able to e.g. track file identity even as a file is moved around, and the metadata to support this lives in a volume-wide tree called the Catalog File. So, there's your global lock. (Add an SSD as necessary for better performance.) OS X already has some pretty high-level file and metadata APIs not found on other systems, so maybe Apple's future plans don't look like a traditional Unix file system at all. They've already demonstrated they know how to make a very weird, non-Unix filesystem look like one. ;)
AceJohnny2 12y ago
I'm rather amused (and, as always, a bit saddened) by the comment of Terry A. Davis, of TempleOS fame/notoriety. "linux doesn't have to search parent directories for file-not-found, but I do." wat Edit: further parsing reveals he implemented a (read-only) overlay system in his FS. Interesting, I wonder what the side-effects (vulns) could be?
- iso8859-1 12y ago
  The word "vulnerability" doesn't make sense when talking about TempleOS, since it does not even attempt to offer any kind of security.
yuhong 12y ago
BTW, one other stupidity is that Linux's HFS+ implementation refuses to mount journaled volumes when Apple designed it to be backward compatible.
- ajross 12y ago
  How could that possibly work? I presume by "backward compatible" you mean that the data outside the journal remains consistent, and the journal layer is capable of detecting modifications made by non-journaled mounts. That's... fine, I guess. It prevents the obvious corruption cases. But the only plausible recovery mechanism after such a mount is to throw out the journal! That's not likely to be acceptable to most users ("I booted to linux and back, and now a bunch of new files disappeared!"). That's stretching the meaning of "compatible" too far.
  - yuhong 12y ago
    Pretty much. Look for lastModifiedVersion in: http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.html http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.ht...
    - ajross 12y ago
      Then I fail to see how this is a stupidity. Linux is doing the right thing: there's no way for it to mount the filesystem without damage.
- cmurf 12y ago
  That's not correct. It won't rw mount journaled HFS+ volumes, they are mounted ro by default. You can force them to mount rw and it'll ignore the journal possibly at the immediate or near future peril of the filesystem, depending on what state its in. I don't know why linux is so far behind on supporting it and doesn't bode well for supporting Core Storage volumes. Near as I can tell while HFS+/X are part of Darwin and thus at least sorta open sourced, I'm not finding any of the Core Storage stuff open sourced meaning it'd have to be completely reverse engineered.
gchpaco 12y ago
HFS+ is probably the worst filesystem in common use right now; even FAT has the benefit of simplicity. Most of its issues, however, are with its horrific implementation; the Unicode naming is kind of bad but Linus manages to be wrong about several things. Regarding case sensitivity: it is generally accepted among the user interface crowd that (Western) users don't really understand that 'C' and 'c' are different things; they're "both" 'c'. Case-preserving is thus the accepted practice. However case manipulation is not an operation that can be done absent a locale; my go to example here is that 'i' upcases to 'I' unless you're a Turk in which case it upcases to 'İ'. Similar although not quite as bad is the fact that 'ß' upcases to 'ẞ' U+1E9E in some exotic circumstances; see http://en.wikipedia.org/wiki/Capital_ẞ http://en.wikipedia.org/wiki/Capital_ẞ for details. Similar limitations apply to sorting, which users also expect. Regarding Unicode: NFD is a normalization format; it converts 'é' U+00E9 and 'é' U+0065 U+0301--which are semantically identical--into the same coding. As it happens NFD picks U+0065 U+0301 for that string; NFC picks U+00E9. Any time there is ambiguity, NF[CD] will retain the original ordering. Calling it "destroying user data" is meaningless histrionics. Most of the time we tend to use NFC. I am told that NFD has certain advantages for sorting, where one might want to match the French word 'être' with the search string 'etre'; in NFC this requires large equivalence tables but in NFD the root character is the same in both cases. Linus's claim 'Even the people who think normalization is a good thing admit that NFD is a bad format, and certainly not for data exchange.' has a big [citation needed] tag attached. As it happens, my personal belief is the following: Given that users expect case sensitivity and locale specific ordering, which complicate filesystem design tremendously. Given that users mostly interact with the system through GUI dialogs, which already hide system files (files with the hidden bit in HFS+, or files starting with '.'). Therefore, extract the case sensitivity to a layer, used by the GUI, which can understand the user's locale and so fold case properly. This layer should be available to command line applications so that they can use the same rules if they so choose. The underlying filesystem will then be case insensitive, but is still used to encode Unicode data; the right thing to do here is to normalize. Either NFC or NFD is fine, really. For pedants: the related NFKC/NFKD forms add a canonicalization step and are absolutely not semantically safe in any way, for all that they're useful for sorting.
- yuhong 12y ago
niels_olson 12y ago
1) What are the odds Tim Cook or Craig Federighi will here about this? 2) What are good filesystems for OS X to adopt? OS X supports other filesystems. Is there a way to force it to install the OS onto a different filesystem, like ext4?
- duskwuff 12y ago
  OS X doesn't have full native support for any filesystem other than HFS+. It supported UFS up until 10.9, but that was ancient and awful. It can read and write FAT, but that's even worse. (It doesn't even support permissions, so it couldn't possibly boot from a FAT volume.) It can read and write FATX. I don't think that supports permissions either, though, and it's also annoyingly patent-encumbered. It can read NTFS, but not write to it.
  - Zardoz84 12y ago
    NTFS3g not was ported to OS X ?
    - emgee3 12y ago
      ntfs-3g was indeed ported and can be easily installed, but I think the point was is writing to NTFS is not natively supported.
- mindajar 12y ago
  Re: 2), there are significant parts of OS X that seem to either check for HFS+ or rely on its implementation and bugs, and those things don't work properly on other filesystems. OpenZFS, for instance, still doesn't work with Spotlight, on which a surprising number of things depend these days. https://openzfsonosx.org/wiki/FAQ#Limitations https://openzfsonosx.org/wiki/FAQ#Limitations If you want to use all the software features of your Mac, your only option is HFS+.
  - knweiss 12y ago
    Soon it will support Spotlight. See https://openzfsonosx.org/wiki/Changelog#1.3.1-RC5 https://openzfsonosx.org/wiki/Changelog#1.3.1-RC5
kennywinker 12y ago
While I really like thristian's explanation of why case insensitivity adds massive complexity to a system: (https://news.ycombinator.com/item?id=8876873 https://news.ycombinator.com/item?id=8876873) I have to point out that case sensitivity offloads a bunch of that complexity to the user. This is almost definitely why OS X uses case-insensitive HFS+ by default. As a ultra-simple example: with a case-sensitive file system I can have two directories, "documents" and "Documents". Which one are my documents in? Half in one, half in the other, probably. I'm not saying Linus at al. are wrong that case-sensitive is the way to go, but there are some reasonable arguments for trying to take that complexity back from the user and deal with it in the file system. Source: I'm a long time mac user, and I just switched to a case-sensitive file system last year. The win of having my dev machine match my deployment environment (iOS) is bigger than any downsides I've seen yet.
- SwellJoe 12y ago
  I find case-insensitivity (as both Mac OS X and Windows do it) user-hostile. So, there's a counter-anecdote. But, I believe all that proves is that in many cases, "what I'm used to" is the same as "easy and intuitive". There is often no one true way to make something complex easy and intuitive, and sometimes the "easy way" is the way you're used to (but for someone used to another way, it won't be easy). In this case, in the absence of compelling evidence that case-insensitivity is a net win for the user, the simpler implementation is probably the right one. The number of bugs over the years caused by insensitivity and Unicode mapping and such, is probably further evidence favoring simplicity. Also, the average Mac user doesn't even know they have a command line (seriously, I can't remember ever saying to a casual Mac user, "Open terminal" and have them say anything other than, "I don't know what that is" or "I don't have that"). There's no reason a higher layer can't make case-insensitive decisions for the user (say, searching for "the clash" will find music by "The Clash"), which doesn't require the freaking filesystem to make those kinds of guesses. Having this happen at the filesystem layer has always seemed utterly mad, to me. And, having it happen at the higher layers is how Linux and UNIX software has always handled it. Somehow we muddle through with the filesystem being entirely naive about the letter "a" being sort of the same as "A" (for humans).
  - 12y ago
jeffehobbs 12y ago
DING
makecheck 12y ago
I agree with Linus from a technical point of view but I think Apple had many considerations here. A number of games (and possibly other programs) on the App Store alone specifically mention that they will not work on Macs configured with case-sensitive file systems. My guess is that this aids programmers who may have ported something from Windows and not tested all possible file/path dependencies. This may also help users when copying files from Windows network disks or Mac legacy systems where (from their point of view) they expect things to work.
justizin 12y ago
Several years ago, when I'd first moved to SF, I got a scholarship to WWDC for working for ACM, and I took advantage of the opportunity to go to a Birds-of-Feather for people interested in "Darwin filesystems" or something like that. It was basically a little conference room with the FS team, and I asked, flat-out, "Why case insensitive?" And they answered, flat-out: "Microsoft Office" Even this past week, I was talking with coworkers about having trouble with nothing but Valve Steam on my Macs that have a case-sensitive filesystem[0]. That's particularly odd, since it works on Linux now, but that's another matter. What I found most notable about this thread, is this quote from Linus: "And Apple let these monkeys work on their filesystem? Seriously?" I'm pretty sure Apple actually _fired_ anyone who wanted any of the things done anything close to any way that Linus Torvalds would agree with. ext* not being particularly perfect, I'm happy to have both. I mean, ext2 is hard to complain about, but it comes from an era where basically all filesystems were terrible, literally the era when SGI started installing backup batteries to race with fsync(). ext4 has an alarming number of corruption bugs, but I'm sure it's not because of insane unicode handling, though I take Linus' description of how the OSX filesystem works with a grain of salt. He can't possibly _care_ to know as much about it as he knows about Linux's. [0] achievable by formatting HFS+X in Disk Utility in Recovery Mode, then installing onto that drive
- thehacker005263 12y ago
  Why wouldn't Microsoft office not work properly if the file system is case-sensitive ?
  - twoodfin 12y ago
    Just a guess, but multiple hard-coded references to the same file with inconsistent case?
- phaemon 12y ago
  > ext4 has an alarming number of corruption bugs Which bugs are these? > though I take Linus' description of how the OSX filesystem works with a grain of salt. He can't possibly _care_ to know as much about it as he knows about Linux's. http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/fs/hfsplus http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...
datashovel 12y ago
Thank God we have someone like Linus to keep BIG TECH in line. It's not all about marketing Apple, Microsoft. It's about good design, and openness.
- datashovel 12y ago
  I would expect far more downvotes than that! Come on what's wrong with you "Hacker News". Only 1 downvote when I'm slammin' Apple for their shitty strategy? Surely some of you have at least some downvotes available to use against that!!!
  - datashovel 12y ago
    Even after a good night's sleep, I'm happy I decided to post this comment. The people hiding behind their karma scores to downvote perfectly legitimate and contextually relevant comments are one of the most frustrating and annoying things about Hacker News.
jen729w 12y ago
Ding!
lispm 12y ago
I agree that HFS+ and its API is outdated. Case insensitivity I find useful, OTOH.

177 points by kannonboy 12y ago

Scroll down for Linus' commentary. > "The true horrors of HFS+ are not in how it's not a great > filesystem, but in how it's actively designed to be a bad > filesystem by people who thought they had good ideas." There doesn't seem to be a way to deep link to comments in G+?

astrodust 12y ago
G+ is a train wreck for a number of reasons, this included. Yeah, he's not a fan of HFS+ at all. Wasn't the plan to move to ZFS prior to the Oracle acquisition of Sun? Hopefully that ends up back on track somehow.
- iwwr 12y ago
  Isn't there a problem with ZFS' memory footprint to be usable for desktop systems?
  - Malic 12y ago
    It seems there was a licensing problem using ZFS on OSX: http://arstechnica.com/apple/2009/10/apple-abandons-zfs-on-mac-os-x-project-over-licensing-issues/ http://arstechnica.com/apple/2009/10/apple-abandons-zfs-on-m...
  - mindajar 12y ago
    ZFS works a lot better if you dedicate lots of memory to it, yes. Apple sells far more iOS devices than Macs these days, and the iOS filesystem is case-sensitive HFS+. It's hard to imagine Apple doing something new in filesystems that doesn't improve the platform they make the most money from. In a way, the same logic applies to Macs. The number of Macs that support multiple internal disks will soon be zero if it isn't already, so some of the best features of ZFS are irrelevant on new Apple computers.
    - stephenr 12y ago
      With thunderbolt ports on very Mac why should storage be limited to internal devices?
      - kstenerud 12y ago
        Because they want you on iCloud. You can't even install the OS without an Apple ID and an active internet connection anymore.
        nizmow 12y ago
        This is extremely false.
        kstenerud 12y ago
        Which part? The iCloud part or the OS install part? I just finished reinstalling Yosemite an hour ago after wiping the HD of a friend's new laptop. It would not allow me to get past the eligibility check until I logged in with a valid Apple ID.
        X-Istence 12y ago
        Your friend may have had Find my Mac turned on. At that point the only way to get an OS installed on it is to have the original apple ID it was turned on under.
        kstenerud 12y ago
        Nope. She never used it. It was the initial "determining the eligibility of this machine" phase where it asked me to log in (originally, this was to check that you'd indeed purchased OSX Lion, but the OS is "free" now). If "Find my Mac" is turned on, it won't even let you wipe the hard drive.
        12y ago
- exelius 12y ago
  ZFS on OS X was abandoned several years ago. Unless Apple has a new filesystem under development as a skunkworks project, HFS+ is the filesystem on OS X for the foreseeable future. And even if they did build a new FS, there are some assumptions like case insensitivity that likely couldn't be abandoned without potentially breaking backwards compatibility.
  - cmurf 12y ago
    And in effect ZFS is forked. There's openzfs and Oracle ZFS, so if Apple were to go with ZFS, which one would they pick? The smart choice would be openzfs which almost certainly means they end up licensing Oracle's ZFS at which point it's like, classic Apple - let's try whenever possible to make interoperability either difficult or hilarious. Apple's Core Storage does have some COW properties, so they could probably do something like linux LVM thin provisioning style snapshots. They could maybe add a tree for checksums, which would be a good idea for the journal at least but seeing as this hasn't been a priority for even just the journal, the most unencumbered and recent add-on, it seems obviously not a priority. The COW stuff was necessary to automagically migrate data from the SSD to the HDD in their so called "fusion drive", so maybe they leverage that for an updated version of Time Machine and could also do it for atomic online OS/software update with an optional later reboot rather than a mandatory one. One rather cool thing it does is on-the-fly encryption and decryption of an LV. You can even reboot in the middle of either conversion and it resumes where it left off. And those encrypted volumes can be resized (that's not well documented but is supported and is even used by their own Boot Camp Assistant). So about the limit of the new fs is an LVM like thing under journaled-HFS+/X.
- fubarred 12y ago
  Only if AAPL acquires/merges with ORCL. 3rd-party: there's the non-FUSE port https://openzfsonosx.org/ https://openzfsonosx.org/
- donavanm 12y ago
  Project predated oracle, back in the osx 10.5-10.6 days when there was the dtrace integration as well. There were internal builds of the zfs kext and at one point I saw leaked source on the Internet. Project was abandoned because of legals disagreements over cddl licensing, as I recall.
  - nobleach 12y ago
    It was in the dropdown list in Disk Utilities in Leopard developer betas. I never actually tried using it.
  - the_why_of_y 12y ago
    CDDL licensing? more likely NetApp scared Apple off with patent aggression http://en.swpat.org/wiki/NetApp%27s_filesystem_patents http://en.swpat.org/wiki/NetApp%27s_filesystem_patents

deleted 12y ago

[deleted]

wiremine 12y ago

"Quite frankly, HFS+ is probably the worst filesystem ever." Can anyone summarize why he thinks this?

kannonboy 12y ago
You might've missed Linus' second comment on the page, wherein he talks about case-insensitivity and NFD normalization.
wazoox 12y ago
HFS+ has been patched with duct tape and pieces of cardboard for 20 years; receiving journaling, support for Unix attributes, extended attributes, 64 bits sizing, multi-processing, multi-users, hard links, etc. over the years. Really it's a sort of monument to kludge. It should have been ditched like 10 years ago.
- mikeash 12y ago
  HFS+ is basically the Apple equivalent of an alternate universe where Microsoft just keeps gluing new features on to FAT forever and never creates NTFS.
- verbatim 12y ago
  I'm not sure that's terribly different from ext4's lineage, which is generally accepted (AFAIK) as a pretty good file system, even if not cutting-edge.
yarrel 12y ago
Because he hasn't used MFS. ;-)

deleted 12y ago

[deleted]

Someone1234 12y ago

Can someone explain to me why case insensitivity is a bad thing? Clearly Linus believes so but didn't explain why he believes so. Most UNIX and Linux systems seem to have an "all lowercase" or "all uppercase" convention, so the fact that they have case sensitivity is often not utilised. In fact the biggest reason you'd want case sensitivity off the top of my head is legacy support but that's just a circular argument (since you never really reach WHY it was that way originally, just that it was). I guess based on what he talks about next he is worried about how case insensitivity interacts with other character sets (i.e. does it correctly change their case), but for most sets isn't the lower and upper case defined in the UNICODE language spec itself?

bigdubs 12y ago
I've run into issues with it from a programming point of view, you have to make sure to do a (slightly) more expensive case insensitive comparison when dealing with filenames.
- Someone1234 12y ago
  This is only an issue when it is fake case insensitive. In the sense that it claims it is case insensitive but in fact is case sensitive (and returns case sensitive filenames). NTFS is a perfect example of this, it claims to be case insensitive but in fact stores everything in a case sensitive way (so you wind up doing things like ToUpper() on every filename). On a legitimately case insensitive filesystem both "etc" and "ETC" would return identical filenames as they would be stored identically in the file system's database (e.g. would return "etc" for both). So what you're really giving is not an example of why case insensitivity is bad, but why half measures are bad.
  - Arnavion 12y ago
    >NTFS is a perfect example of this, it claims to be case insensitive Where does it claim this? In any case this is not true because... >but in fact stores everything in a case sensitive way This does not have to be true. NTFS can be case-sensitive or case-insensitive depending on which subsystem is accessing it. For example, while the POSIX subsystem for Windows was a thing, installing it had the side-effect of turning NTFS case-sensitive.

terminus 12y ago

Also a longer critique by John Siracusa here: http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/#hfs-problems http://arstechnica.com/apple/2011/07/mac-os-x-10-7/12/#hfs-p... I wonder how this ties in with the whole Apple philosophy of "Design is how it works." Clearly the innards look nothing like the facade.

yuhong 12y ago
Wonder why the "HFS+ Private Data" hack was chosen when Rhapsody existed even back in 1997.
Someone1234 12y ago
> File system metadata structures in HFS+ have global locks. Only one process can update the file system at a time. Holy heck! How does that work in practice? Do operations get queued and then a single kernel process who can take the lock do the atomic updates? I have to imagine that is going to cause a bottleneck however, as all non-read operations need to update the metadata (e.g. timestamp, maybe size if it is stored). That all being said I haven't noticed OS X being particularly slower to do things than e.g. Windows. So if that is the case they're hiding it well.
- mindajar 12y ago
  HFS+ is an oddball filesystem among FSes that anyone actually uses. Since the earliest days of the Mac, it's been able to e.g. track file identity even as a file is moved around, and the metadata to support this lives in a volume-wide tree called the Catalog File. So, there's your global lock. (Add an SSD as necessary for better performance.) OS X already has some pretty high-level file and metadata APIs not found on other systems, so maybe Apple's future plans don't look like a traditional Unix file system at all. They've already demonstrated they know how to make a very weird, non-Unix filesystem look like one. ;)

AceJohnny2 12y ago

I'm rather amused (and, as always, a bit saddened) by the comment of Terry A. Davis, of TempleOS fame/notoriety. "linux doesn't have to search parent directories for file-not-found, but I do." wat Edit: further parsing reveals he implemented a (read-only) overlay system in his FS. Interesting, I wonder what the side-effects (vulns) could be?

iso8859-1 12y ago
The word "vulnerability" doesn't make sense when talking about TempleOS, since it does not even attempt to offer any kind of security.

yuhong 12y ago

BTW, one other stupidity is that Linux's HFS+ implementation refuses to mount journaled volumes when Apple designed it to be backward compatible.

ajross 12y ago
How could that possibly work? I presume by "backward compatible" you mean that the data outside the journal remains consistent, and the journal layer is capable of detecting modifications made by non-journaled mounts. That's... fine, I guess. It prevents the obvious corruption cases. But the only plausible recovery mechanism after such a mount is to throw out the journal! That's not likely to be acceptable to most users ("I booted to linux and back, and now a bunch of new files disappeared!"). That's stretching the meaning of "compatible" too far.
- yuhong 12y ago
  Pretty much. Look for lastModifiedVersion in: http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.html http://dubeiko.com/development/FileSystems/HFSPLUS/tn1150.ht...
  - ajross 12y ago
    Then I fail to see how this is a stupidity. Linux is doing the right thing: there's no way for it to mount the filesystem without damage.
cmurf 12y ago
That's not correct. It won't rw mount journaled HFS+ volumes, they are mounted ro by default. You can force them to mount rw and it'll ignore the journal possibly at the immediate or near future peril of the filesystem, depending on what state its in. I don't know why linux is so far behind on supporting it and doesn't bode well for supporting Core Storage volumes. Near as I can tell while HFS+/X are part of Darwin and thus at least sorta open sourced, I'm not finding any of the Core Storage stuff open sourced meaning it'd have to be completely reverse engineered.

gchpaco 12y ago

HFS+ is probably the worst filesystem in common use right now; even FAT has the benefit of simplicity. Most of its issues, however, are with its horrific implementation; the Unicode naming is kind of bad but Linus manages to be wrong about several things. Regarding case sensitivity: it is generally accepted among the user interface crowd that (Western) users don't really understand that 'C' and 'c' are different things; they're "both" 'c'. Case-preserving is thus the accepted practice. However case manipulation is not an operation that can be done absent a locale; my go to example here is that 'i' upcases to 'I' unless you're a Turk in which case it upcases to 'İ'. Similar although not quite as bad is the fact that 'ß' upcases to 'ẞ' U+1E9E in some exotic circumstances; see http://en.wikipedia.org/wiki/Capital_ẞ http://en.wikipedia.org/wiki/Capital_ẞ for details. Similar limitations apply to sorting, which users also expect. Regarding Unicode: NFD is a normalization format; it converts 'é' U+00E9 and 'é' U+0065 U+0301--which are semantically identical--into the same coding. As it happens NFD picks U+0065 U+0301 for that string; NFC picks U+00E9. Any time there is ambiguity, NF[CD] will retain the original ordering. Calling it "destroying user data" is meaningless histrionics. Most of the time we tend to use NFC. I am told that NFD has certain advantages for sorting, where one might want to match the French word 'être' with the search string 'etre'; in NFC this requires large equivalence tables but in NFD the root character is the same in both cases. Linus's claim 'Even the people who think normalization is a good thing admit that NFD is a bad format, and certainly not for data exchange.' has a big [citation needed] tag attached. As it happens, my personal belief is the following: Given that users expect case sensitivity and locale specific ordering, which complicate filesystem design tremendously. Given that users mostly interact with the system through GUI dialogs, which already hide system files (files with the hidden bit in HFS+, or files starting with '.'). Therefore, extract the case sensitivity to a layer, used by the GUI, which can understand the user's locale and so fold case properly. This layer should be available to command line applications so that they can use the same rules if they so choose. The underlying filesystem will then be case insensitive, but is still used to encode Unicode data; the right thing to do here is to normalize. Either NFC or NFD is fine, really. For pedants: the related NFKC/NFKD forms add a canonicalization step and are absolutely not semantically safe in any way, for all that they're useful for sorting.

yuhong 12y ago

niels_olson 12y ago

1) What are the odds Tim Cook or Craig Federighi will here about this? 2) What are good filesystems for OS X to adopt? OS X supports other filesystems. Is there a way to force it to install the OS onto a different filesystem, like ext4?

duskwuff 12y ago
OS X doesn't have full native support for any filesystem other than HFS+. It supported UFS up until 10.9, but that was ancient and awful. It can read and write FAT, but that's even worse. (It doesn't even support permissions, so it couldn't possibly boot from a FAT volume.) It can read and write FATX. I don't think that supports permissions either, though, and it's also annoyingly patent-encumbered. It can read NTFS, but not write to it.
- Zardoz84 12y ago
  NTFS3g not was ported to OS X ?
  - emgee3 12y ago
    ntfs-3g was indeed ported and can be easily installed, but I think the point was is writing to NTFS is not natively supported.
mindajar 12y ago
Re: 2), there are significant parts of OS X that seem to either check for HFS+ or rely on its implementation and bugs, and those things don't work properly on other filesystems. OpenZFS, for instance, still doesn't work with Spotlight, on which a surprising number of things depend these days. https://openzfsonosx.org/wiki/FAQ#Limitations https://openzfsonosx.org/wiki/FAQ#Limitations If you want to use all the software features of your Mac, your only option is HFS+.
- knweiss 12y ago
  Soon it will support Spotlight. See https://openzfsonosx.org/wiki/Changelog#1.3.1-RC5 https://openzfsonosx.org/wiki/Changelog#1.3.1-RC5

kennywinker 12y ago

While I really like thristian's explanation of why case insensitivity adds massive complexity to a system: (https://news.ycombinator.com/item?id=8876873 https://news.ycombinator.com/item?id=8876873) I have to point out that case sensitivity offloads a bunch of that complexity to the user. This is almost definitely why OS X uses case-insensitive HFS+ by default. As a ultra-simple example: with a case-sensitive file system I can have two directories, "documents" and "Documents". Which one are my documents in? Half in one, half in the other, probably. I'm not saying Linus at al. are wrong that case-sensitive is the way to go, but there are some reasonable arguments for trying to take that complexity back from the user and deal with it in the file system. Source: I'm a long time mac user, and I just switched to a case-sensitive file system last year. The win of having my dev machine match my deployment environment (iOS) is bigger than any downsides I've seen yet.

SwellJoe 12y ago
I find case-insensitivity (as both Mac OS X and Windows do it) user-hostile. So, there's a counter-anecdote. But, I believe all that proves is that in many cases, "what I'm used to" is the same as "easy and intuitive". There is often no one true way to make something complex easy and intuitive, and sometimes the "easy way" is the way you're used to (but for someone used to another way, it won't be easy). In this case, in the absence of compelling evidence that case-insensitivity is a net win for the user, the simpler implementation is probably the right one. The number of bugs over the years caused by insensitivity and Unicode mapping and such, is probably further evidence favoring simplicity. Also, the average Mac user doesn't even know they have a command line (seriously, I can't remember ever saying to a casual Mac user, "Open terminal" and have them say anything other than, "I don't know what that is" or "I don't have that"). There's no reason a higher layer can't make case-insensitive decisions for the user (say, searching for "the clash" will find music by "The Clash"), which doesn't require the freaking filesystem to make those kinds of guesses. Having this happen at the filesystem layer has always seemed utterly mad, to me. And, having it happen at the higher layers is how Linux and UNIX software has always handled it. Somehow we muddle through with the filesystem being entirely naive about the letter "a" being sort of the same as "A" (for humans).
- 12y ago

jeffehobbs 12y ago

DING

makecheck 12y ago

I agree with Linus from a technical point of view but I think Apple had many considerations here. A number of games (and possibly other programs) on the App Store alone specifically mention that they will not work on Macs configured with case-sensitive file systems. My guess is that this aids programmers who may have ported something from Windows and not tested all possible file/path dependencies. This may also help users when copying files from Windows network disks or Mac legacy systems where (from their point of view) they expect things to work.

justizin 12y ago

Several years ago, when I'd first moved to SF, I got a scholarship to WWDC for working for ACM, and I took advantage of the opportunity to go to a Birds-of-Feather for people interested in "Darwin filesystems" or something like that. It was basically a little conference room with the FS team, and I asked, flat-out, "Why case insensitive?" And they answered, flat-out: "Microsoft Office" Even this past week, I was talking with coworkers about having trouble with nothing but Valve Steam on my Macs that have a case-sensitive filesystem[0]. That's particularly odd, since it works on Linux now, but that's another matter. What I found most notable about this thread, is this quote from Linus: "And Apple let these monkeys work on their filesystem? Seriously?" I'm pretty sure Apple actually _fired_ anyone who wanted any of the things done anything close to any way that Linus Torvalds would agree with. ext* not being particularly perfect, I'm happy to have both. I mean, ext2 is hard to complain about, but it comes from an era where basically all filesystems were terrible, literally the era when SGI started installing backup batteries to race with fsync(). ext4 has an alarming number of corruption bugs, but I'm sure it's not because of insane unicode handling, though I take Linus' description of how the OSX filesystem works with a grain of salt. He can't possibly _care_ to know as much about it as he knows about Linux's. [0] achievable by formatting HFS+X in Disk Utility in Recovery Mode, then installing onto that drive

thehacker005263 12y ago
Why wouldn't Microsoft office not work properly if the file system is case-sensitive ?
- twoodfin 12y ago
  Just a guess, but multiple hard-coded references to the same file with inconsistent case?
phaemon 12y ago
> ext4 has an alarming number of corruption bugs Which bugs are these? > though I take Linus' description of how the OSX filesystem works with a grain of salt. He can't possibly _care_ to know as much about it as he knows about Linux's. http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/fs/hfsplus http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...

datashovel 12y ago

Thank God we have someone like Linus to keep BIG TECH in line. It's not all about marketing Apple, Microsoft. It's about good design, and openness.

datashovel 12y ago
I would expect far more downvotes than that! Come on what's wrong with you "Hacker News". Only 1 downvote when I'm slammin' Apple for their shitty strategy? Surely some of you have at least some downvotes available to use against that!!!
- datashovel 12y ago
  Even after a good night's sleep, I'm happy I decided to post this comment. The people hiding behind their karma scores to downvote perfectly legitimate and contextually relevant comments are one of the most frustrating and annoying things about Hacker News.

jen729w 12y ago

Ding!

lispm 12y ago

I agree that HFS+ and its API is outdated. Case insensitivity I find useful, OTOH.