5 ms·

> APFS volumes cannot be reverted to HFS Is that actually true? I’d heard that Apple had reversion instructions for those running the High Sierra beats on a sy

> APFS volumes cannot be reverted to HFS Is that actually true? I’d heard that Apple had reversion instructions for those running the High Sierra beats on a system with a Fusion Drive, which they dropped support for in the final release.

LeoPanthera 9y ago
The instructions are backup, reformat, restore.
- bgentry 9y ago
  Hah, good to know. I guess that’s what you get for installing a beta OS on a machine you care about.
tzs 9y ago
Converting between different file systems in place is an interesting technical problem. Let's say you have a disk that is formatted with filesystem S (for "Source"), and you want it change it to filesystem D (for "Destination"). Let's assume for now that S is not encrypted. Let's also assume that we have a fair amount of scratch storage available somewhere not on the disk we are trying to convert. (I'll address removing this requirement later). One approach is to make something that given a list of files, including metadata, and a blank disk, can create a D filesystem on that disk and then copies the files to it. You also need something that given a file on S can give you a list of the block numbers that hold that file's data. That's an S to D copier, not an S to D in place converter, but it is useful for testing your understanding of D. Once you are confident that you know how D works, you can then make a new version of your D copier. The new version doesn't actually write anything out to the blank disk. Instead, it writes instructions for making the disk out to a file in scratch space. For blocks that would hold metadata (e.g., inode blocks on Unix filesystem), it records in the scratch space what their content would be and where they would go on D. For blocks that would hold file data, you write into the scratch area the list of where that data resides on S, and where that data would go on D. What you end up with, then, in the scratch space is essentially a set of block by block instructions for copying S to a blank disk and ending up with a D filesystem, where instructions are either "copy block X from S to block Y of D" or "Fill block Y of D with the following literal data". So to do an in place conversion, just follow those instructions, but with S and D the same, right? NO!!!!! That would likely totally trash things, because it is quite possible that a source block in one instruction may have already been overwritten as a destination in an earlier instruction. What you first need is something that goes through the whole list of block copy/fill instructions and figures out an order they can be done in that is safe. It is possible that there is no such order. In that case, you also need some free space that can used for temporary storage. Given some free space you can always find an order for the block copy/fill that works. The free space can either be on another device, or if S is not full you can find and use its free space. Anyway, once you do all this, you end up with a simple list of data move/fill steps that will convert S to D in place. You just have to figure out how to actually execute them. The tricky case is when S is your system disk so you cannot un-mount it. Your OS might get very unhappy if you start converting it in place, and you also have to worry that it might write things to it in the middle of the conversion that could corrupt it. Probably best would be to do the conversion booted from something like a Linux live CD. I said earlier that you needed some separate scratch space to store the information for the new filesystem. If there is free space on S you can probably use that. This comment is getting long, so I'm going to cut it off here. In a reply to it I'll toss in a couple of implementation tricks that can be useful, especially if D is a proprietary filesystem that you do not have a complete spec for and so cannot make your thing that figures out where to write D metadata and file data.
- tzs 9y ago
  (continuation of above comment) If you do not have a good spec for D, a kludge is to make a virtual disk driver. Make a blank D volume on that virtual disk. Then you can copy everything from S to D, except you replace the actual data for each file with blocks of the form: <signature> <block # on S> <some constant fill pattern>. For each block written that does NOT match that pattern, your virtual disk driver actually does save the data somewhere. For blocks that do match that pattern, it just records what block on S they came from. From the information gathered by the virtual disk driver you have enough information to construct a D file system from the S filesystem. If you do not have an operating system that understand both S and D (common in dual boot scenarios), you can still do the above, but first you have to have a step where from the OS that understands S you run something that makes a pseudo dump of S. For example, a modified version of tar. This modified tar would act like normal tar, except instead of copying the actual file data, it would just make a list of disk blocks. Then you switch to the OS that understands D, and you can use your virtual disk driver and the output from the modified tar to make the virtual D filesystem and gather the necessary information. Note that this only has a chance if D does not care about the actual contents of files. If D does care, for example maybe it stores a checksum of the file in the metadata, then we are screwed because we do not have the original data in this scenario. Whether or not this can be worked around depends on just how much we know about D. When you are ready to do the actual conversion, you face the question of where to run it. If S is not your boot volume, you might be able to get away with dismounting it and doing the conversion under your normal operating system. When we were playing around with something like the above in the late '90s (part of a project considering making a PartitionMagic competitor, although we didn't get much past the experimenting stage before management decided the market was too small, PartitionMagic was too dominant in it, and their patent was scary and so cancelled the project) we were leaning toward using a Linux live CD (for everything, not just the actual copying). You probably want the code that actually goes through the block copy/fill list and executes it to have some sort of fault tolerance that allows it to pick up where it left off if something interrupts it such as a power failure.
- Someone 9y ago
  I think it is easier. Split your source disk in: - directory blocks that describe the directory structure and, for each file, where to find this data on disk. - data blocks that contain actual file data. - free space. Chances are you don’t need to touch the data blocks, and you have the free space as scratch space. All you have to do is to write a new set of directory blocks for the new disk format, and then delete the set of old directory blocks. If the on-disk format is flexible enough w.r.t. the location of directory and data blocks and you’re somewhat lucky, you can mostly write the new set of directory blocks in existing free space on the source disk. That would lead to an intermediate state where the source disk still has the old format, but has one extra file that contains most of the new directory structure. The final, risky part then is fairly small: overwrite the disk’s root blocks so that the system sees the disk as having the new file format with the new directory structure. If the disk formats use different disk blocks as first starting points, you can even have a disk that can be used in either format (IIRC, that was possible with FAT and HFS. FAT has its master directory block in sector 0, HFS has it in sector 2)

LeoPanthera 9y ago

The instructions are backup, reformat, restore.

bgentry 9y ago
Hah, good to know. I guess that’s what you get for installing a beta OS on a machine you care about.

tzs 9y ago

Converting between different file systems in place is an interesting technical problem. Let's say you have a disk that is formatted with filesystem S (for "Source"), and you want it change it to filesystem D (for "Destination"). Let's assume for now that S is not encrypted. Let's also assume that we have a fair amount of scratch storage available somewhere not on the disk we are trying to convert. (I'll address removing this requirement later). One approach is to make something that given a list of files, including metadata, and a blank disk, can create a D filesystem on that disk and then copies the files to it. You also need something that given a file on S can give you a list of the block numbers that hold that file's data. That's an S to D copier, not an S to D in place converter, but it is useful for testing your understanding of D. Once you are confident that you know how D works, you can then make a new version of your D copier. The new version doesn't actually write anything out to the blank disk. Instead, it writes instructions for making the disk out to a file in scratch space. For blocks that would hold metadata (e.g., inode blocks on Unix filesystem), it records in the scratch space what their content would be and where they would go on D. For blocks that would hold file data, you write into the scratch area the list of where that data resides on S, and where that data would go on D. What you end up with, then, in the scratch space is essentially a set of block by block instructions for copying S to a blank disk and ending up with a D filesystem, where instructions are either "copy block X from S to block Y of D" or "Fill block Y of D with the following literal data". So to do an in place conversion, just follow those instructions, but with S and D the same, right? NO!!!!! That would likely totally trash things, because it is quite possible that a source block in one instruction may have already been overwritten as a destination in an earlier instruction. What you first need is something that goes through the whole list of block copy/fill instructions and figures out an order they can be done in that is safe. It is possible that there is no such order. In that case, you also need some free space that can used for temporary storage. Given some free space you can always find an order for the block copy/fill that works. The free space can either be on another device, or if S is not full you can find and use its free space. Anyway, once you do all this, you end up with a simple list of data move/fill steps that will convert S to D in place. You just have to figure out how to actually execute them. The tricky case is when S is your system disk so you cannot un-mount it. Your OS might get very unhappy if you start converting it in place, and you also have to worry that it might write things to it in the middle of the conversion that could corrupt it. Probably best would be to do the conversion booted from something like a Linux live CD. I said earlier that you needed some separate scratch space to store the information for the new filesystem. If there is free space on S you can probably use that. This comment is getting long, so I'm going to cut it off here. In a reply to it I'll toss in a couple of implementation tricks that can be useful, especially if D is a proprietary filesystem that you do not have a complete spec for and so cannot make your thing that figures out where to write D metadata and file data.

tzs 9y ago
(continuation of above comment) If you do not have a good spec for D, a kludge is to make a virtual disk driver. Make a blank D volume on that virtual disk. Then you can copy everything from S to D, except you replace the actual data for each file with blocks of the form: <signature> <block # on S> <some constant fill pattern>. For each block written that does NOT match that pattern, your virtual disk driver actually does save the data somewhere. For blocks that do match that pattern, it just records what block on S they came from. From the information gathered by the virtual disk driver you have enough information to construct a D file system from the S filesystem. If you do not have an operating system that understand both S and D (common in dual boot scenarios), you can still do the above, but first you have to have a step where from the OS that understands S you run something that makes a pseudo dump of S. For example, a modified version of tar. This modified tar would act like normal tar, except instead of copying the actual file data, it would just make a list of disk blocks. Then you switch to the OS that understands D, and you can use your virtual disk driver and the output from the modified tar to make the virtual D filesystem and gather the necessary information. Note that this only has a chance if D does not care about the actual contents of files. If D does care, for example maybe it stores a checksum of the file in the metadata, then we are screwed because we do not have the original data in this scenario. Whether or not this can be worked around depends on just how much we know about D. When you are ready to do the actual conversion, you face the question of where to run it. If S is not your boot volume, you might be able to get away with dismounting it and doing the conversion under your normal operating system. When we were playing around with something like the above in the late '90s (part of a project considering making a PartitionMagic competitor, although we didn't get much past the experimenting stage before management decided the market was too small, PartitionMagic was too dominant in it, and their patent was scary and so cancelled the project) we were leaning toward using a Linux live CD (for everything, not just the actual copying). You probably want the code that actually goes through the block copy/fill list and executes it to have some sort of fault tolerance that allows it to pick up where it left off if something interrupts it such as a power failure.
Someone 9y ago
I think it is easier. Split your source disk in: - directory blocks that describe the directory structure and, for each file, where to find this data on disk. - data blocks that contain actual file data. - free space. Chances are you don’t need to touch the data blocks, and you have the free space as scratch space. All you have to do is to write a new set of directory blocks for the new disk format, and then delete the set of old directory blocks. If the on-disk format is flexible enough w.r.t. the location of directory and data blocks and you’re somewhat lucky, you can mostly write the new set of directory blocks in existing free space on the source disk. That would lead to an intermediate state where the source disk still has the old format, but has one extra file that contains most of the new directory structure. The final, risky part then is fairly small: overwrite the disk’s root blocks so that the system sees the disk as having the new file format with the new directory structure. If the disk formats use different disk blocks as first starting points, you can even have a disk that can be used in either format (IIRC, that was possible with FAT and HFS. FAT has its master directory block in sector 0, HFS has it in sector 2)