7 ms·

CursedFS – Disk image that is simultaneously ext2 and FAT

188 points by codezero 7y ago

NieDzejkob 7y ago
Author here, happy to answer any questions.
- yowlingcat 7y ago
  What makes this particular FS "cursed" besides the obvious intuitive answer you can glean from the reference twitter thread?
  - NieDzejkob 7y ago
    I feel like this name nicely captures the fact that this almost surely has no real-world use and is likely to break stuff (file system autodetection, for one...). I also like to draw parallels to the notion of a cursed idea.
- DannyB2 7y ago
  From GitHub: > It turns out this is surprisingly simple to do: just create a FAT volume with a lot of reserved sectors and put the ext2 into the reserved sectors. This works because the filesystems choose different places to put their superblock: FAT uses the very first sector, while ext2 leaves the first kilobyte unused. So, is it really two independent file systems? At least when it is read only? It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks.
  - NieDzejkob 7y ago
    > So, is it really two independent file systems? Yes, they just don't interact with each other. > It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks. The entirety of ext2 stays within FAT32's reserved blocks (that is, between the bootsector and the File Allocation Table), so FAT32 won't write over ext2. The inverse is also true, as ext2 knows its formatted size, so it won't write outside of the reserved sectors.
    - dividuum 7y ago
      So basically the perfect combination if you want to boot a Raspberry Pi from a single partition for some reason. I like it!
    - DannyB2 7y ago
      Since the two filesystems are independent, it seems they could have independent content. A filesystem that could boot DOS or Linux.
- cesarb 7y ago
  Can you add ISO 9660 and/or UDF to the mix? Both do not use the first several kilobytes of the volume (hybrid CDs created by genisoimage use that trick to combine HFS with ISO 9660).
  - NieDzejkob 7y ago
    From a quick look at the specs, it would seem like it is possible to add either of those, but probably not both. I'll probably add this in "v2.0" - I'm planning to redo this in a way that actually interleaves the filesystems, which would also allow the ext* part to be larger than 32 MB.
  - 1996 7y ago
    And can you add fuse support, to keep at least ext2 and vfat in sync? The best usecase I can see: /boot, used for both windows and Linux with a special entry to start a live linux for recovery. Having this partition on a thumbdrive and being able to cat it to /dev/sda1 would be ideal
- TazeTSchnitzel 7y ago
  Since ZIP files have their header at the end of the file, perhaps the disk image could also be a valid ZIP file?
  - wahern 7y ago
    ZIP files also have inline headers with duplicate metadata that may or may not be the same (e.g. filenames can and do differ). That's why I've always felt conflicted over ZIP. On the one hand, having a contiguous index is nice from a performance optimization standpoint, and that's why ZIP is more commonly adopted to build upon, such as JAR files. On the other hand, the potential for (and actuality of) conflicting metadata is a nightmare from a security perspective, and it violates several rules of thumb in software engineering, such as that there should only ever be a single source of truth (major), a single parse (major), and to avoid duplicating data or context (minor, but can be sufficient to ensure there's only ever a single source of truth and a single parse), which generally ease implementation and help ensure consistent behavior. Likewise, while combining the container format and compression can improve convenience and user experience (especially if there's a single dominate implementation), it's also nice to keep these layers separate, which not only provides simplicity and consistency benefits, but also permits them to evolve independently.
- ranma42 7y ago
  This is also possible with NTFS+ext[2,3,4]. I did work on modifying mkfs.ntfs do create a polyglot filesystem like this, but ran into issues that the windows filesystem check was not happy with the result (though it was mountable as either ext4 or ntfs in Linux). https://pastebin.com/MujcN5MZ https://pastebin.com/MujcN5MZ
- int_19h 7y ago
  If the media is read-only (or we pretend that it is), is it possible to have the filesystems share data at least for some files, by reusing the blocks?
  - NieDzejkob 7y ago
    Yeah, this is one of the things I'd like to explore when I make the formatter more full-fledged and not a 10 line bash script. I'm hoping to try marking the datastructures of each filesystem as a bad block or something similar in the others, so that as long as the shared data isn't removed or modified, writing is OK.
nickik 7y ago
Seems pretty cool. Not sure where I personally would use it, but when interacting with legacy this could make things much easier.
- NieDzejkob 7y ago
  I didn't envision a usecase, I just found the question of whether this is possible interesting. If I had to guess, this will break more things than it will help with - the filesystems are independent, not "synchronized".
  - tyingq 7y ago
    Add encryption support for the ext2 part and it's a decent upper "security through obscurity" layer. If the wrong person finds your USB stick, they will likely only ever notice the FAT filesystem. Like for a spy dead drop use case.
    - tracker1 7y ago
      Looks like you, my good sir, have found a use case for this.
    - Fnoord 7y ago
      Yes, plausible deniability was the immediate use case I was thinking about as well. Curious if/how it would mount on Android (with USB-OTG cable)?
    - 1996 7y ago
      Or more pratical: a partition on the SD card of your phone that can boot to a live linux, then store the live data locally in the ext2 part I did try to do that with my android phone, but apparently it requires some specific kernel modules. I've to figure out which, and how to cross compile them for the phone kernel.
    - tedunangst 7y ago
      This seems about as likely to work as hiding emails in the draft folder and not sending them. Computer forensics have looked at all the bytes on the disk, not just the active files reachable from the root directory, for decades.
ComputerGuru 7y ago
The filesystems aren’t synchronized, though. Quite misleading! I’m not sure there’s a technical reason why they couldn’t theoretically be synchronized (either in real-time via a custom fs layer or as cron task, etc).
- saurik 7y ago
  I don't know if the title has changed, but "disk image that is simultaneously..." made it very clear to me that this would imply a static state carefully crafted file, not some synchronized dynamic file system driver.
makomk 7y ago
I'm pretty sure I managed to create one of these by accident when first using Linux, though with much messier results. Same root cause though: the ext2 and FAT superblocks didn't overlap, and back in the day some of the filesystem creation tools weren't so good at making sure to overwrite conflicting superblocks.
roywiggins 7y ago
For extra fun, fill them up with polyglot files! https://medium.com/swlh/polyglot-files-a-hackers-best-friend-850bf812dd8a https://medium.com/swlh/polyglot-files-a-hackers-best-friend...
gbin 7y ago
This is cool as a computer forensics exercise: the FS could look empty on one side but hide a lot of structured informing the other.
- AnIdiotOnTheNet 7y ago
  I have yet to meet any filesystem forensic software that wouldn't immediately identify this. Pretty much everything I've ever used would just tell you that it found multiple filesystems and show you both. Recovering files is often possible even without any filesystem metadata at all.
  - JNRowe 7y ago
    To make that point with an example, my go to for any new disk image or real drive is disktype¹ which produces: --- cursed.img Regular file, size 4 MiB (4194304 bytes) FAT12 file system (hints score 4 of 5) Volume size 3.916 MiB (4106240 bytes, 2005 clusters of 2 KiB) Ext2 file system UUID 2D74B033-E8A6-4738-98B2-02BCC3F0D98E (DCE, v4) Last mounted at "/home/kuba/cursedfs/mountpoint" Volume size 64 KiB (65536 bytes, 64 blocks of 1 KiB) 1. http://disktype.sf.net http://disktype.sf.net
crazygringo 7y ago
So the suspense is killing me :) ...if you insert a memory stick formatted this way into a Linux box, which one will show up? (Forgive my ignorance if the answer is obvious -- I've only ever used Linux as servers, never as a desktop.) And are there any other OS's that can read ext2? Neither Windows nor macOS seem to be able to, out of the box.
- NieDzejkob 7y ago
  I actually didn't try to let it autodetect... until now. ~/cursedfs% sudo mount cursed.img mountpoint/ mount: /home/kuba/cursedfs/mountpoint: more filesystems detected on /dev/loop0; use -t <type> or wipefs(8). It's too smart :/
  - andrewflnr 7y ago
    Props to whoever thought to check for the case of fricking polyglot filesystem images, though. Probably intended for something boring like corruption, but still, nice.
    - NieDzejkob 7y ago
      Apparently one could create something like this by accident in the early days of Linux: https://twitter.com/makomk/status/1217542964995678209?s=20 https://twitter.com/makomk/status/1217542964995678209?s=20
gdavisson 7y ago
IIRC Apple used to do something like this on their installer DVDs (back when they shipped the OS on DVDs). The Mac OS X installer was on an HFS+ volume pointed to by an Apple Partition Map (which would only be recognized by Mac OS X and the Mac's boot firmware), but the DVD also had an ISO 9660 header (that'd be recognized by Windows) pointing to Windows-compatible drivers for the Mac hardware. Completely separate volumes, with completely separate content.
- JdeBP 7y ago
  Microsoft's FAT to NTFS conversion tool has been doing this sort of thing for years, too. It just puts the NTFS master file table and other stuff into the FAT free space, or a designated file. Of course, being a conversion tool the intent was that both sets of data structures described the same files. * https://docs.microsoft.com/en-gb/windows-server/administration/windows-commands/convert https://docs.microsoft.com/en-gb/windows-server/administrati... Equally interesting are the tools that convert from the "MBR" style partition table to EFI partitioning. Again, this requires that the MBR-style partition table have free spaces into which the EFI partition table can be put. They have to be in fixed positions, though, unlike the NTFS MFT.
  - asdfaoeu 7y ago
    btrfs has a similar convertor with the other quirk that the source ext partition remains as a file in btrfs and deleting it "commit"s the conversion. https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-convert https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-conver...
    - cmurf 7y ago
      Small quibble, the original file system, named ext2_saved, is a subvolume/snapshot. As long as it's kept around, you can revert the conversion while losing any changes made to the Btrfs file system. The comments explain the strategy in some detail. ReiserFS conversion is supported in addition to ext[234]. Because Btrfs requires only a little fixed space, just its superblocks, in theory any file system could be converted. https://github.com/kdave/btrfs-progs/blob/master/convert/main.c https://github.com/kdave/btrfs-progs/blob/master/convert/mai...
tambourine_man 7y ago
When Apple created HFS+ they did a similar hack that would show the disk with a TeachText read me file named “where did all my files go?” if you tried to mount it as HFS. That means, Macs with older versions of the OS would deal gracefully with a format they were never intended to run. I remember thinking it was the coolest thing ever.
fctorial 7y ago
Should be called pornfs, you can use it to hide porn.

188 points by codezero 7y ago

NieDzejkob 7y ago

Author here, happy to answer any questions.

yowlingcat 7y ago
What makes this particular FS "cursed" besides the obvious intuitive answer you can glean from the reference twitter thread?
- NieDzejkob 7y ago
  I feel like this name nicely captures the fact that this almost surely has no real-world use and is likely to break stuff (file system autodetection, for one...). I also like to draw parallels to the notion of a cursed idea.
DannyB2 7y ago
From GitHub: > It turns out this is surprisingly simple to do: just create a FAT volume with a lot of reserved sectors and put the ext2 into the reserved sectors. This works because the filesystems choose different places to put their superblock: FAT uses the very first sector, while ext2 leaves the first kilobyte unused. So, is it really two independent file systems? At least when it is read only? It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks.
- NieDzejkob 7y ago
  > So, is it really two independent file systems? Yes, they just don't interact with each other. > It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks. The entirety of ext2 stays within FAT32's reserved blocks (that is, between the bootsector and the File Allocation Table), so FAT32 won't write over ext2. The inverse is also true, as ext2 knows its formatted size, so it won't write outside of the reserved sectors.
  - dividuum 7y ago
    So basically the perfect combination if you want to boot a Raspberry Pi from a single partition for some reason. I like it!
  - DannyB2 7y ago
    Since the two filesystems are independent, it seems they could have independent content. A filesystem that could boot DOS or Linux.
cesarb 7y ago
Can you add ISO 9660 and/or UDF to the mix? Both do not use the first several kilobytes of the volume (hybrid CDs created by genisoimage use that trick to combine HFS with ISO 9660).
- NieDzejkob 7y ago
  From a quick look at the specs, it would seem like it is possible to add either of those, but probably not both. I'll probably add this in "v2.0" - I'm planning to redo this in a way that actually interleaves the filesystems, which would also allow the ext* part to be larger than 32 MB.
- 1996 7y ago
  And can you add fuse support, to keep at least ext2 and vfat in sync? The best usecase I can see: /boot, used for both windows and Linux with a special entry to start a live linux for recovery. Having this partition on a thumbdrive and being able to cat it to /dev/sda1 would be ideal
TazeTSchnitzel 7y ago
Since ZIP files have their header at the end of the file, perhaps the disk image could also be a valid ZIP file?
- wahern 7y ago
  ZIP files also have inline headers with duplicate metadata that may or may not be the same (e.g. filenames can and do differ). That's why I've always felt conflicted over ZIP. On the one hand, having a contiguous index is nice from a performance optimization standpoint, and that's why ZIP is more commonly adopted to build upon, such as JAR files. On the other hand, the potential for (and actuality of) conflicting metadata is a nightmare from a security perspective, and it violates several rules of thumb in software engineering, such as that there should only ever be a single source of truth (major), a single parse (major), and to avoid duplicating data or context (minor, but can be sufficient to ensure there's only ever a single source of truth and a single parse), which generally ease implementation and help ensure consistent behavior. Likewise, while combining the container format and compression can improve convenience and user experience (especially if there's a single dominate implementation), it's also nice to keep these layers separate, which not only provides simplicity and consistency benefits, but also permits them to evolve independently.
ranma42 7y ago
This is also possible with NTFS+ext[2,3,4]. I did work on modifying mkfs.ntfs do create a polyglot filesystem like this, but ran into issues that the windows filesystem check was not happy with the result (though it was mountable as either ext4 or ntfs in Linux). https://pastebin.com/MujcN5MZ https://pastebin.com/MujcN5MZ
int_19h 7y ago
If the media is read-only (or we pretend that it is), is it possible to have the filesystems share data at least for some files, by reusing the blocks?
- NieDzejkob 7y ago
  Yeah, this is one of the things I'd like to explore when I make the formatter more full-fledged and not a 10 line bash script. I'm hoping to try marking the datastructures of each filesystem as a bad block or something similar in the others, so that as long as the shared data isn't removed or modified, writing is OK.

nickik 7y ago

Seems pretty cool. Not sure where I personally would use it, but when interacting with legacy this could make things much easier.

NieDzejkob 7y ago
I didn't envision a usecase, I just found the question of whether this is possible interesting. If I had to guess, this will break more things than it will help with - the filesystems are independent, not "synchronized".
- tyingq 7y ago
  Add encryption support for the ext2 part and it's a decent upper "security through obscurity" layer. If the wrong person finds your USB stick, they will likely only ever notice the FAT filesystem. Like for a spy dead drop use case.
  - tracker1 7y ago
    Looks like you, my good sir, have found a use case for this.
  - Fnoord 7y ago
    Yes, plausible deniability was the immediate use case I was thinking about as well. Curious if/how it would mount on Android (with USB-OTG cable)?
  - 1996 7y ago
    Or more pratical: a partition on the SD card of your phone that can boot to a live linux, then store the live data locally in the ext2 part I did try to do that with my android phone, but apparently it requires some specific kernel modules. I've to figure out which, and how to cross compile them for the phone kernel.
  - tedunangst 7y ago
    This seems about as likely to work as hiding emails in the draft folder and not sending them. Computer forensics have looked at all the bytes on the disk, not just the active files reachable from the root directory, for decades.

ComputerGuru 7y ago

The filesystems aren’t synchronized, though. Quite misleading! I’m not sure there’s a technical reason why they couldn’t theoretically be synchronized (either in real-time via a custom fs layer or as cron task, etc).

saurik 7y ago
I don't know if the title has changed, but "disk image that is simultaneously..." made it very clear to me that this would imply a static state carefully crafted file, not some synchronized dynamic file system driver.

makomk 7y ago

I'm pretty sure I managed to create one of these by accident when first using Linux, though with much messier results. Same root cause though: the ext2 and FAT superblocks didn't overlap, and back in the day some of the filesystem creation tools weren't so good at making sure to overwrite conflicting superblocks.

roywiggins 7y ago

For extra fun, fill them up with polyglot files! https://medium.com/swlh/polyglot-files-a-hackers-best-friend-850bf812dd8a https://medium.com/swlh/polyglot-files-a-hackers-best-friend...

gbin 7y ago

This is cool as a computer forensics exercise: the FS could look empty on one side but hide a lot of structured informing the other.

AnIdiotOnTheNet 7y ago
I have yet to meet any filesystem forensic software that wouldn't immediately identify this. Pretty much everything I've ever used would just tell you that it found multiple filesystems and show you both. Recovering files is often possible even without any filesystem metadata at all.
- JNRowe 7y ago
  To make that point with an example, my go to for any new disk image or real drive is disktype¹ which produces: --- cursed.img Regular file, size 4 MiB (4194304 bytes) FAT12 file system (hints score 4 of 5) Volume size 3.916 MiB (4106240 bytes, 2005 clusters of 2 KiB) Ext2 file system UUID 2D74B033-E8A6-4738-98B2-02BCC3F0D98E (DCE, v4) Last mounted at "/home/kuba/cursedfs/mountpoint" Volume size 64 KiB (65536 bytes, 64 blocks of 1 KiB) 1. http://disktype.sf.net http://disktype.sf.net

crazygringo 7y ago

So the suspense is killing me :) ...if you insert a memory stick formatted this way into a Linux box, which one will show up? (Forgive my ignorance if the answer is obvious -- I've only ever used Linux as servers, never as a desktop.) And are there any other OS's that can read ext2? Neither Windows nor macOS seem to be able to, out of the box.

NieDzejkob 7y ago
I actually didn't try to let it autodetect... until now. ~/cursedfs% sudo mount cursed.img mountpoint/ mount: /home/kuba/cursedfs/mountpoint: more filesystems detected on /dev/loop0; use -t <type> or wipefs(8). It's too smart :/
- andrewflnr 7y ago
  Props to whoever thought to check for the case of fricking polyglot filesystem images, though. Probably intended for something boring like corruption, but still, nice.
  - NieDzejkob 7y ago
    Apparently one could create something like this by accident in the early days of Linux: https://twitter.com/makomk/status/1217542964995678209?s=20 https://twitter.com/makomk/status/1217542964995678209?s=20

gdavisson 7y ago

IIRC Apple used to do something like this on their installer DVDs (back when they shipped the OS on DVDs). The Mac OS X installer was on an HFS+ volume pointed to by an Apple Partition Map (which would only be recognized by Mac OS X and the Mac's boot firmware), but the DVD also had an ISO 9660 header (that'd be recognized by Windows) pointing to Windows-compatible drivers for the Mac hardware. Completely separate volumes, with completely separate content.

JdeBP 7y ago
Microsoft's FAT to NTFS conversion tool has been doing this sort of thing for years, too. It just puts the NTFS master file table and other stuff into the FAT free space, or a designated file. Of course, being a conversion tool the intent was that both sets of data structures described the same files. * https://docs.microsoft.com/en-gb/windows-server/administration/windows-commands/convert https://docs.microsoft.com/en-gb/windows-server/administrati... Equally interesting are the tools that convert from the "MBR" style partition table to EFI partitioning. Again, this requires that the MBR-style partition table have free spaces into which the EFI partition table can be put. They have to be in fixed positions, though, unlike the NTFS MFT.
- asdfaoeu 7y ago
  btrfs has a similar convertor with the other quirk that the source ext partition remains as a file in btrfs and deleting it "commit"s the conversion. https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-convert https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-conver...
  - cmurf 7y ago
    Small quibble, the original file system, named ext2_saved, is a subvolume/snapshot. As long as it's kept around, you can revert the conversion while losing any changes made to the Btrfs file system. The comments explain the strategy in some detail. ReiserFS conversion is supported in addition to ext[234]. Because Btrfs requires only a little fixed space, just its superblocks, in theory any file system could be converted. https://github.com/kdave/btrfs-progs/blob/master/convert/main.c https://github.com/kdave/btrfs-progs/blob/master/convert/mai...

tambourine_man 7y ago

When Apple created HFS+ they did a similar hack that would show the disk with a TeachText read me file named “where did all my files go?” if you tried to mount it as HFS. That means, Macs with older versions of the OS would deal gracefully with a format they were never intended to run. I remember thinking it was the coolest thing ever.

fctorial 7y ago

Should be called pornfs, you can use it to hide porn.