5 ms·

Author here, happy to answer any questions.

yowlingcat 7y ago
What makes this particular FS "cursed" besides the obvious intuitive answer you can glean from the reference twitter thread?
- NieDzejkob 7y ago
  I feel like this name nicely captures the fact that this almost surely has no real-world use and is likely to break stuff (file system autodetection, for one...). I also like to draw parallels to the notion of a cursed idea.
DannyB2 7y ago
From GitHub: > It turns out this is surprisingly simple to do: just create a FAT volume with a lot of reserved sectors and put the ext2 into the reserved sectors. This works because the filesystems choose different places to put their superblock: FAT uses the very first sector, while ext2 leaves the first kilobyte unused. So, is it really two independent file systems? At least when it is read only? It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks.
- NieDzejkob 7y ago
  > So, is it really two independent file systems? Yes, they just don't interact with each other. > It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks. The entirety of ext2 stays within FAT32's reserved blocks (that is, between the bootsector and the File Allocation Table), so FAT32 won't write over ext2. The inverse is also true, as ext2 knows its formatted size, so it won't write outside of the reserved sectors.
  - dividuum 7y ago
    So basically the perfect combination if you want to boot a Raspberry Pi from a single partition for some reason. I like it!
  - DannyB2 7y ago
    Since the two filesystems are independent, it seems they could have independent content. A filesystem that could boot DOS or Linux.
cesarb 7y ago
Can you add ISO 9660 and/or UDF to the mix? Both do not use the first several kilobytes of the volume (hybrid CDs created by genisoimage use that trick to combine HFS with ISO 9660).
- NieDzejkob 7y ago
  From a quick look at the specs, it would seem like it is possible to add either of those, but probably not both. I'll probably add this in "v2.0" - I'm planning to redo this in a way that actually interleaves the filesystems, which would also allow the ext* part to be larger than 32 MB.
- 1996 7y ago
  And can you add fuse support, to keep at least ext2 and vfat in sync? The best usecase I can see: /boot, used for both windows and Linux with a special entry to start a live linux for recovery. Having this partition on a thumbdrive and being able to cat it to /dev/sda1 would be ideal
TazeTSchnitzel 7y ago
Since ZIP files have their header at the end of the file, perhaps the disk image could also be a valid ZIP file?
- wahern 7y ago
  ZIP files also have inline headers with duplicate metadata that may or may not be the same (e.g. filenames can and do differ). That's why I've always felt conflicted over ZIP. On the one hand, having a contiguous index is nice from a performance optimization standpoint, and that's why ZIP is more commonly adopted to build upon, such as JAR files. On the other hand, the potential for (and actuality of) conflicting metadata is a nightmare from a security perspective, and it violates several rules of thumb in software engineering, such as that there should only ever be a single source of truth (major), a single parse (major), and to avoid duplicating data or context (minor, but can be sufficient to ensure there's only ever a single source of truth and a single parse), which generally ease implementation and help ensure consistent behavior. Likewise, while combining the container format and compression can improve convenience and user experience (especially if there's a single dominate implementation), it's also nice to keep these layers separate, which not only provides simplicity and consistency benefits, but also permits them to evolve independently.
ranma42 7y ago
This is also possible with NTFS+ext[2,3,4]. I did work on modifying mkfs.ntfs do create a polyglot filesystem like this, but ran into issues that the windows filesystem check was not happy with the result (though it was mountable as either ext4 or ntfs in Linux). https://pastebin.com/MujcN5MZ https://pastebin.com/MujcN5MZ
int_19h 7y ago
If the media is read-only (or we pretend that it is), is it possible to have the filesystems share data at least for some files, by reusing the blocks?
- NieDzejkob 7y ago
  Yeah, this is one of the things I'd like to explore when I make the formatter more full-fledged and not a 10 line bash script. I'm hoping to try marking the datastructures of each filesystem as a bad block or something similar in the others, so that as long as the shared data isn't removed or modified, writing is OK.

What makes this particular FS "cursed" besides the obvious intuitive answer you can glean from the reference twitter thread?

NieDzejkob 7y ago
I feel like this name nicely captures the fact that this almost surely has no real-world use and is likely to break stuff (file system autodetection, for one...). I also like to draw parallels to the notion of a cursed idea.

DannyB2 7y ago

From GitHub: > It turns out this is surprisingly simple to do: just create a FAT volume with a lot of reserved sectors and put the ext2 into the reserved sectors. This works because the filesystems choose different places to put their superblock: FAT uses the very first sector, while ext2 leaves the first kilobyte unused. So, is it really two independent file systems? At least when it is read only? It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks.

NieDzejkob 7y ago
> So, is it really two independent file systems? Yes, they just don't interact with each other. > It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks. The entirety of ext2 stays within FAT32's reserved blocks (that is, between the bootsector and the File Allocation Table), so FAT32 won't write over ext2. The inverse is also true, as ext2 knows its formatted size, so it won't write outside of the reserved sectors.
- dividuum 7y ago
  So basically the perfect combination if you want to boot a Raspberry Pi from a single partition for some reason. I like it!
- DannyB2 7y ago
  Since the two filesystems are independent, it seems they could have independent content. A filesystem that could boot DOS or Linux.

cesarb 7y ago

Can you add ISO 9660 and/or UDF to the mix? Both do not use the first several kilobytes of the volume (hybrid CDs created by genisoimage use that trick to combine HFS with ISO 9660).

NieDzejkob 7y ago
From a quick look at the specs, it would seem like it is possible to add either of those, but probably not both. I'll probably add this in "v2.0" - I'm planning to redo this in a way that actually interleaves the filesystems, which would also allow the ext* part to be larger than 32 MB.
1996 7y ago
And can you add fuse support, to keep at least ext2 and vfat in sync? The best usecase I can see: /boot, used for both windows and Linux with a special entry to start a live linux for recovery. Having this partition on a thumbdrive and being able to cat it to /dev/sda1 would be ideal

TazeTSchnitzel 7y ago

Since ZIP files have their header at the end of the file, perhaps the disk image could also be a valid ZIP file?

wahern 7y ago
ZIP files also have inline headers with duplicate metadata that may or may not be the same (e.g. filenames can and do differ). That's why I've always felt conflicted over ZIP. On the one hand, having a contiguous index is nice from a performance optimization standpoint, and that's why ZIP is more commonly adopted to build upon, such as JAR files. On the other hand, the potential for (and actuality of) conflicting metadata is a nightmare from a security perspective, and it violates several rules of thumb in software engineering, such as that there should only ever be a single source of truth (major), a single parse (major), and to avoid duplicating data or context (minor, but can be sufficient to ensure there's only ever a single source of truth and a single parse), which generally ease implementation and help ensure consistent behavior. Likewise, while combining the container format and compression can improve convenience and user experience (especially if there's a single dominate implementation), it's also nice to keep these layers separate, which not only provides simplicity and consistency benefits, but also permits them to evolve independently.

ranma42 7y ago

This is also possible with NTFS+ext[2,3,4]. I did work on modifying mkfs.ntfs do create a polyglot filesystem like this, but ran into issues that the windows filesystem check was not happy with the result (though it was mountable as either ext4 or ntfs in Linux). https://pastebin.com/MujcN5MZ https://pastebin.com/MujcN5MZ

int_19h 7y ago

If the media is read-only (or we pretend that it is), is it possible to have the filesystems share data at least for some files, by reusing the blocks?

NieDzejkob 7y ago
Yeah, this is one of the things I'd like to explore when I make the formatter more full-fledged and not a 10 line bash script. I'm hoping to try marking the datastructures of each filesystem as a bad block or something similar in the others, so that as long as the shared data isn't removed or modified, writing is OK.