10 ms·
Apple File System Reference [pdf]
- plg 8y agofonts?
- sawaali 8y agoShould be San Francisco.
- saagarjha 8y agoAnd San Francisco Mono for the monospaced font.
- sjwright 8y agoEvery time I see the San Francisco font mentioned anywhere, I don't think of Apple's rather lovely neo-grotesque, but rather Apple's original San Francisco. (Which is also lovely, in an oh cool, this 1984 era computer has a variety of fonts kind of way.) http://luc.devroye.org/SusanKare--SanFrancisco-1984.png http://luc.devroye.org/SusanKare--SanFrancisco-1984.png
- chungy 8y agoThis seems like it's probably enough to re-implement APFS on non-Mac platforms; in fact, the about page (page 6) says as much. Kudos to Apple for providing the information. It's a hell of a lot better than reverse engineering the thing (see how many years it took to get NTFS down...)
- Rondom 8y agoThere is already one third-party-implementation in the works. They sure find this helpful in their efforts. https://github.com/sgan81/apfs-fuse https://github.com/sgan81/apfs-fuse
- atonse 8y agoThis is user space – hopefully a kernel level one will come out as a result of this. Or does it matter, performance-wise? People that know more can chime in.
- Cyph0n 8y agoAgreed. I was honestly expecting a more hand-wavy explanation, but I was surprised that they dived into so much detail on, e.g., the structs used to represent various objects in APFS.
- deleted 8y ago[deleted]
- swingline-747 8y agoLikely enough to create third-party disk recovery utilities.
- sjwright 8y agoFrom the second paragraph of the PDF: "This document is for developers of software that interacts with the file system directly, without using any frameworks or the operating system—for example, a disk recovery utility or an implementation of Apple File System on another platform."
- monocasa 8y agoIt's a little light. It reminds me of some vendor GPU documentation that explains the names of constants and layouts, but not as much how the pieces fit together, and the gotchas of the interrelated data structures. And that's what tends to be the hard part anyway.
- israrkhan 8y agoIn past, I implemented HFS+ on an embedded rtos platform,. Technical Note TN1150 [1] proved to be extremely useful asset. From a cursory look, I feel TN1150 was much more detailed, and perhaps can be treated as a pre-req to this document. At least it should have been mentioned in this document. [1] https://developer.apple.com/library/archive/technotes/tn/tn1150.html https://developer.apple.com/library/archive/technotes/tn/tn1...
- tambourine_man 8y agoYou got me curious, can you explain why you needed to implement HFS+? Was it read only?
- israrkhan 8y agoI was working on a video recorder product that could record videos directly to iPods, iPhones and other media players. iPods were formatted to HFS+, if you connected them to a Mac out-of-box, and were formatted to FAT32, if you connected them to a PC. The platform i was working on was an RTOS, and we had to develop both FAT32, and HFS+ from scratch. It was not read-only, it supported both read/write.
- tambourine_man 8y agoNice, that sounds fun, thanks.
- saagarjha 8y agoIt's nice to see that this is finally up; I know a lot of people have been clamoring for a more detailed reference for a while and this should hopefully make it easier for them to interact with APFS.
- sneak 8y agoI’m still sad that this filesystem does not contain file data checksums. It looks like we will be stuck with it for some years to come.
- nicky0 8y agoCare to enlighten ignorant me why we would want that?
- scienceman 8y agoNot OP, but probably to make sure the contents of a file are not changed by hardware errors.
- jandrese 8y agoYep, otherwise you aren't able to detect bit level errors unless they impact the metadata, and the metadata is a tiny fraction of your total storage. That said, your hard drive already does block level checksumming so doing it at the FS layer is mostly redundant unless the errors are being introduced in your SATA controller or on the PCI bus.
- aaaaaaaaaab 8y agoYou would still need end-to-end integrity checking, unless your Mac came with ECC memory (which it probably didn't).
- mcpherrinm 8y agoMemory errors are still a concern, however, RAM is not used for persistent storage. If a bit flip occurs during the path to storing data, that could get persisted. That's a moment in time, though. Maybe you'll notice the document you just wrote seems corrupted, or just has a typo. But if you write successfully to disk, you are trusting that data to stay there long-term. If years later your drive corrupts a bit, you may have a very hard time noticing. Bad RAM manifests as computer instability and you can just replace RAM without data loss, as nobody is permanently storing data in RAM Because the data spends so much longer on disk than in RAM, the chance of a bit flip affecting stored data.
- petecooper 8y agoOne of my long-time favourite macOS applications -- iDefrag -- had support withdrawn shortly after APFS appeared. Reasons cited were lack of an APFS spec and increased System Integrity Protection. I fear this is too little, too late to have iDefrag make a comeback. I understand defragmenting an SSD typically does more harm than good [edit: and I only defrag spinning drives), but nothing touched it for effectiveness on spinning drives. https://coriolis-systems.com/iDefrag/ https://coriolis-systems.com/iDefrag/ https://coriolis-systems.com/blog/2017/9/what-works-macos-1013-high-sierra-what-problems-ex https://coriolis-systems.com/blog/2017/9/what-works-macos-10...
- r00fus 8y agoI haven't defragged a disk in almost a decade. When is defragging an SSD ever a good thing? I only have spinning rust in my (older) NAS or as backup drives.
- tinus_hn 8y agoProbably often but there is no way to tell really because the controller has a mind of its own and they are all different.
- forapurpose 8y ago> Probably often Defragging an SSD is often a good thing? Why? It would seem to greatly increase wear for no benefit.
- akvadrako 8y agoIf your fragments are really small like 16kb then you could see a significant performance improvement, due to better predictive loading and packet overhead. This is clear from SSD benchmarks. But I don’t see how this would ever realistically happen.
- forapurpose 8y ago
- mrpippy 8y agoThe 'Fusion' section at the end is interesting. macOS 10.13 didn't convert Fusion drives to APFS, but 10.14 will. Presumably this specific support for Fusion drives is new in 10.14. HFS+ had no knowledge about Fusion drives, the caching was handled entirely at block-level by the lower CoreStorage layer (although later versions did add some flags so CoreStorage could pin metadata/swap blocks to the SSD). Now what I'm really interested to see is if they open-source the filesystem driver along with the macOS 10.14 code drop. HFS+ (and its utilities) has always been open-source, last year APFS was not.
- arthurfm 8y agoThe only bad thing about Apple supporting APFS on Fusion Drives is that it gives them an incentive to continue selling future iMacs and Mac minis with Fusion Drives. Having had to replace failed HDDs in Fusion Drive iMacs at work, it's certainly no fun. For all new Mac purchases I ensure they are SSD only now.
- mrpippy 8y agoOn that note, I am surprised that they added Fusion-awareness to APFS, rather than just putting APFS on top of CoreStorage. It certainly is better to have the filesystem aware of the Fusion situation, but...measurably, significantly better? Would the experience have been significantly worse without it? 10.13 betas allowed APFS use on Fusion drives, presumably without any Fusion-awareness in the FS. I'm surprised, but happy to see they did it.
- swingline-747 8y agoGood find. Wonder if there's as much doc of ZFS/OpenZFS.
- chungy 8y agoIndeed there is: http://www.giis.co.in/Zfs_ondiskformat.pdf http://www.giis.co.in/Zfs_ondiskformat.pdf It glosses over and assumes knowledge of XDR from an external source. That is documented here: https://tools.ietf.org/html/rfc1014.html https://tools.ietf.org/html/rfc1014.html
- ken 8y agoAt last! Apple's old APFS docs always had this mysterious note about Fast Directory Sizing: "You cannot enable Fast Directory Sizing on directories containing files or other directories directly; you must instead first create a new directory, enable fast directory sizing on it, and then move the contents of the existing directory to the new directory." but there was never any documentation on how to do this, and no Apple engineer would say. The most common internet theory seemed to be that this feature was purely automatic, and all mentions (like this) in the docs were just incredibly misleading. Now it seems we have an answer, in this flag: "INODE_MAINTAIN_DIR_STATS: The inode tracks the size of all of its children."
- deleted 8y ago[deleted]
- cmurf 8y agoThe EFI jumpstart is particularly clever. A straightforward recipe for locating and verifying the file system driver, and then once executed the UEFI pre-boot environment can fully navigate an APFS volume.
- aasasd 8y agoUhhh, personally I'd prefer that UEFI stay away from the OS particulars. But anyway, afaik Windows' boot code had about the same feature―at least it certainly did in regard to the chipset drivers, the result being IIRC that the OS wouldn't boot if you moved the partitions a bit.
- cmurf 8y agoThe pre-boot environment needs to find the kernel and initramfs somehow. My guess for how Apple is booting from APFS, now that it's all APFS, without a separate recovery partition? They've got this minimalist EFI jumpstart code in the firmware, it loads the EFI file system driver for APFS, and now it can locate the bootloader, kernel, and kext cache. For a long time Apple has had an HFS+ driver baked into the firmware. The way APFS is implemented with EFI jumpstart, they've got much less filesystem code in firmware.
- Someone 8y agoThere are several cases where the file system has monotonically increasing integers that, when overflowing, are unrecoverable errors. Those counters always are 64 bits, and won’t overflow in normal use (for example, the text says: ”if you created 1,000,000 transactions per second, it would take more than 5,000 centuries to exhaust the available transaction identifiers.”), but I can see people making ‘interesting’ disk images, for example ones where writing to a specific directory is impossible or, depending on how the implementation handles it, even panics the OS.