7 ms·
macOS Tahoe brings a new disk image format
- bigyabai 1y agoMy kingdom for a documented disk image format.
- aaronmdjones 1y agohttps://docs.kernel.org/filesystems/ext4/index.html https://docs.kernel.org/filesystems/ext4/index.html
- zymhan 1y agoA filesystem is not a disk image.
- n_plus_1_acc 1y agoIf it quaks like a disk image, it is a disk image
- deleted 1y ago[deleted]
- duskwuff 1y agoA filesystem is not a file. (Yes, you can store a filesystem in a file - and that's a trivial sort of disk image, but one with some serious drawbacks like "you have to allocate all of the space up front". We can do better.)
- Dylan16807 1y agoSome of the most popular disk image formats are basically a sparse file abstraction for non-sparse files and nothing more. You have a bunch of blocks, a table mapping each block to its virtual location, and a couple convenience headers. If those count as a disk image when you put a filesystem inside, then I say a normal file is also a disk image when you put a filesystem inside. Especially because the sparse mapping is optional. For example, lots of VHDs are a raw file plus a 512 byte footer.
- zymhan 1y agoWhat?
- shakna 1y agoAbout the only decently documented disk image format I've found is qcow2. [0] Which is usually not the best tool for the job. So very many of them are just header details + "only works with our tools". [0] https://www.qemu.org/docs/master/interop/qcow2.html https://www.qemu.org/docs/master/interop/qcow2.html
- mannyv 1y agoI'll take your kingdom! ISO 9660. https://www.iso.org/iso-9660-images-for-computer-files.html https://www.iso.org/iso-9660-images-for-computer-files.html VMDK https://en.wikipedia.org/wiki/VMDK https://en.wikipedia.org/wiki/VMDK Amiga https://en.wikipedia.org/wiki/Amiga_Disk_File https://en.wikipedia.org/wiki/Amiga_Disk_File UDF https://en.wikipedia.org/wiki/Universal_Disk_Format https://en.wikipedia.org/wiki/Universal_Disk_Format Apple Disk Image https://en.wikipedia.org/wiki/Apple_Disk_Image https://en.wikipedia.org/wiki/Apple_Disk_Image
- shakna 1y agoVMDK isn't documented there, and is a container for multiple partially documented - and some only reverse engineered - subformats.
- mdaniel 1y agoNot on wikipedia, but VMDK is part of the DTMF OVF[1] format and thus I believe the .vmdk format would be implicitly made available therein However, words are words but software is better: - https://github.com/vmware/open-vmdk#specifications https://github.com/vmware/open-vmdk#specifications is Apache 2 - the link they cited is bitrotten but Internet Archive has you: https://web.archive.org/web/20210411181842/https://www.vmware.com/app/vmdk/?src=vmdk https://web.archive.org/web/20210411181842/https://www.vmwar... 1: https://www.dmtf.org/standards/ovf https://www.dmtf.org/standards/ovf
- shakna 1y agoUnfortunately, no. OVA and OVF are defined in the OVF spec, but VMDK is not. So congrats on finding the spec for it, as its no longer maintained! Its also no longer followed, either, unfortunately.
- tobias3 1y agoVHDX https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-vhdx/83e061f8-f6e2-4de1-91bd-5d518a43d477 https://learn.microsoft.com/en-us/openspecs/windows_protocol...
- bowsamic 1y agoMy favourite blog specifically for the painting articles
- baggy_trough 1y agoThey are amazing, for sure.
- chrisweekly 1y agoYes! So glad GP brought that up. Bookmarked, looking fwd to spending more time absorbing the fantastic images and accompanying texts.
- lucasoshiro 1y agoNice. But I would like it better if the effort was to support ext4, BtrFS, NTFS and other popular filesystems from the Linux and Windows world...
- gjsman-1000 1y agoext4 and Btrfs are only well supported on Linux; they are not universal standards. NTFS was only supported well on Windows until recently; but extensions like NTFS Encryption (BitLocker) are still Windows only. Mac still does not let you write to an NTFS volume. APFS and HFS+ are obviously Apple file systems. FreeBSD does not support ext4 or Btrfs well; but instead prefers UFS2 or ZFS despite also being an open-source Unix-inspired OS. The world runs on proprietary or non-universal file systems with CDFS (ISO 9660), FAT, and exFAT being the sole exceptions.
- nemomarx 1y agoif Mac also supported Linux file formats they would get closer to universal, though? we can't get to that world without these individual steps. creating a new non universal one is backing away from it
- gjsman-1000 1y agoEven FreeBSD can't be bothered to do a good job supporting Linux file systems. They're basically proprietary siloed file systems like the rest, even if code is available. Linux, meanwhile, can't be bothered to support either of FreeBSD's file systems officially. UFS2 is hardly patented anymore, but Linux doesn't care beyond read-only support. Being open-source, and even being in a popular open-source project, does not make a standard, or even imply inferiority to those who do not implement it. https://xkcd.com/927/ https://xkcd.com/927/
- bombcar 1y agoThere might be some license issues, but the dirty secret is filesystem portability isn't terribly important, and for the users for whom it is - ExFAT and friends are usually good enough.
- pbronez 1y agoAny chance this will reduce the space needed for major OS updates? Those have always been desperately inefficient.
- duskwuff 1y agoNo. Disk images aren't involved in the OS update process.
- happyopossum 1y agoThey are - in that OS updates are delivered in and extracted from disk images. Not sure how they could be any more involved…
- duskwuff 1y agoNot this kind of disk image. ASIF is a read-write disk image; an OS update would be distributed in a read-only image (if it is indeed a disk image).
- srameshc 1y agoI hope they did something to not load my hard drive with all the AI crap and steal the storage.
- platevoltage 1y agoThis ridiculous arms race can't be over soon enough.
- moondev 1y agohttps://developer.apple.com/documentation/virtualization/vzdiskimagestoragedeviceattachment/ https://developer.apple.com/documentation/virtualization/vzd... I guess this is the Apple version of qcow2 and friends
- henry700 1y agoIt's a shame that every new cool product/dataformat/cable/cpu/whatever researched by Apple has very little (or no) public documentation. Sure, there are lots of hackers who can test and reverse engineer those pretty quickly, but it's just unnecessary work. I don't know why Apple is so revered in hacker circles, to be honest. Not even Microsoft does this shit anymore, they're open sourcing a lot of research this decade, but they're still seen with extreme distrust. Whereas Apple was always secretive and used underhanded tactics, but it is still loved.
- alwillis 1y agoHere's the link to the Darwin, the underlying Unix layer of macOS, which is open source [1]. Go for it! [1]: https://github.com/apple-oss-distributions/distribution-macOS.git https://github.com/apple-oss-distributions/distribution-macO...
- henry700 1y agoYou're joking, right? You can see at a glance that this mess is only published for compliance reasons. There is no documentation at all, and most of the code consists of numerous versions of open-source libraries that have been subtly modified due to their licensing requirements, which necessitate disclosure of modifications. Good luck building any flavor of that! See: https://news.ycombinator.com/item?id=35197308 https://news.ycombinator.com/item?id=35197308
- frizlab 1y agoI have a meta question not directly related to the article but more about HN itself. I posted this exact link 9h before this submission was posted[1]. How is it possible that there is a new entry for the submission given the link is the same? [1]https://news.ycombinator.com/from?site=eclecticlight.co https://news.ycombinator.com/from?site=eclecticlight.co
- wmf 1y agoDupes are allowed after some number of hours.
- 90s_dev 1y agoIt depends. I tried posting a few links I came across over the past few days, and it just showed me dups but they were from days or weeks or sometimes months ago.
- summarity 1y agoNot sure why this comment is downvoted. Yes, I got redirected for a dupe earlier today for something posted 7 months ago. And even though the content substantially changed, it can’t be posted again.
- bombcar 1y agoI seem to remember getting that and having the option to post it again. Maybe it’s a karma related thing.
- loeg 1y agoI thought that number was significantly more than 9.
- azhenley 1y agoI think it is time * (votes+comments). In practice, if it gets any real amount of votes or comments, you have to wait a year to repost. If it doesn't get any attention, it can be reposted quickly (though I think it should be a day later).
- ChrisMarshallNY 1y agoDoesn’t sound particularly useful, if you aren’t setting up containers.
- burke 1y agoIf Apple would just ship bind mounts and FS/pid namespaces... or even just un-break chroot...
- MBCook 1y agoUnless they can migrate Time Machine to it. The performance improvements sound like they might be a real boon there.
- kccqzy 1y agoI use sparse disk images extensively as a tool to make certain directories require specific passwords. I simply create an encrypted sparse disk image and set a password on them. Seems like this will speed up my use case.
- aquafox 1y ago[flagged]
- skrtskrt 1y agoI think this is actually a human struggle - I add Conclusions to my blog posts / dev guides because it feels awkward to just end with the last step. But it also always feels like I am just awkwardly restating things I just said. It's not a thesis, an article generally not complicated enough to need a re-synthesizing of main points at the end.
- thierrydamiba 1y agoI have an unusual way of communicating sometimes that used to be quirky and fun. Now I simply get accused of being a bot. Before LLMs I never got accused of being a robot. I think the idea of good writing is soon going to morph with writing that doesn’t feel like it was made with AI.
- skrtskrt 1y agoIt drives me crazy that em-dashes are supposedly a smell for AI now. The em-dash is a great and underused bit of punctuation. I am just glad I am not in college dealing with all the AI and faulty AI detection crap happening. Ridiculous false accusations of plagiarism were already awful enough before this.
- kstrauser 1y agoSame here. It's almost as if LLMs emit em-dashes because the content they were trained on contains them. Wait until skeptics realize how often LLMs use the word "the".
- SkyeCA 1y ago> It drives me crazy that em-dashes are supposedly a smell for AI now. They're an excellent tell for comments written on places like HN, Reddit, and other forms of social media. The average person (myself included) simply does not write very well and I would wager the majority of English speakers are not able to properly use an em-dash. I expect them in books and suspect that's where the AI models are getting them from, but those are being written largely by professionals. It's at a point where the moment I see an em-dash on a social media post I cease reading it and move on to the next.
- cyberflame 1y agoOh awesome! So it's more virtualisation-focused as opposed to HFS+ --> APFS migration?
- aroman 1y agoApples and oranges. This is a new disk image format. You can create an APFS _volume_ in the ASIF _image_.
- AnonC 1y agoI skimmed through the article, but I have a question that I hope someone can answer. I have a sparse disk image created on a NAS (which runs Linux), and I use it to backup some stuff (not a VM) from the Mac in the native format (the macOS APFS file system). Would this new format, ASIF, make this faster and better whenever I switch to macOS Tahoe? I hope there wouldn’t be any gotchas with respect to storing this disk image on a NAS.
- poisonwomb 1y agoYeah I’m interested in this as well. There’s something about the way time machine works over SMB that is absolutely unfashionably dog-slow. I suspect SMB performance on mac is just not very good in general tbf
- benguillet 1y agoWould that potentially speed up Docker for Mac and others? (since it's using a vm underneath). That would address a major pain point
- helmsb 1y agoThey also announced new APIs for native Linux containerization on MacOS with a specific focus on security and performance. This seems like it may be in support of that as well. Anything they can do to improve the performance of containers is a huge win. https://youtu.be/JvQtvbhtXmo?si=3OphClGvylHggmSW https://youtu.be/JvQtvbhtXmo?si=3OphClGvylHggmSW
- crest 1y agoHas anyone found the specification for the format?
- ThePowerOfFuet 1y agoWas this not already a solved problem? Why reinvent the wheel other than for vendor lockin?
- kccqzy 1y agoIt's a solved problem in the sense that Apple solved it more than a decade ago. (The sparsebundle format was introduced in Mac OS X Leopard from 2007. The sparseimage format was even older; I saw discussions of using it with Panther.) Apple solved it again but made it faster.
- layer8 1y ago“Asif” is a fun name for a disk image format. It’s as-if it was a real disk. ;)
- whycome 1y agoAndor is a funny name for a tv show
- teaearlgraycold 1y agoWell, TV show and/or long movie depending on how you look at it.
- archagon 1y agoWhat blew my mind recently was that I could store an APFS sparsebundle on a NAS drive, then mount it over NFS and use it as a plain old APFS volume. Despite the filesystem layering, it works pretty much like a local volume, albeit with a bit of performance degradation. Seems preferable to something like iSCSI for using APFS with network storage. Perhaps this new format would work even better?
- kccqzy 1y agoNot sure why that would blow your mind, but Apple's old Time Capsule basically mounted a sparsebundle over the network too. And yes I would also guess this new format would work better.
- archagon 1y agoMy experience with NFS file management has been less than stellar, so running a full, virtualized, and performant APFS volume on top of it feels like a bit of a magic trick.
- kccqzy 1y agoInteresting. My employer uses NFS extensively and issues are very rare, and such issues can always be traced to underlying network issues not NFS itself. It's a good choice if you have a good network.
- saagarjha 1y agoMy former employer used NFS and it would lock up a bunch of stuff whenever the network dropped for a bit. In fact a lot of their software had detection for this case so that you wouldn't think it was at fault for the issues.
- tayiorrobinson 1y agoThe benchmarks are weird to me - the ASIF tests were done on M3/4, but for everything else it was done on an M1?
- bradknowles 1y agoYeah, that jumped out at me, too. If they can’t re-run the benchmarks on the same hardware, it’s hard to compare the numbers.
- deleted 1y ago[deleted]
- littlecranky67 1y agoI wonder with the recent changes regarding containers and now disk images, if Apple plans to enter the server or cloud market.