3 ms·

Kaitai looks nice - have you used it enough to review how it handles? I'm just starting a project to deal with somewhat involved on-disk formats[0] and this mig

Kaitai looks nice - have you used it enough to review how it handles? I'm just starting a project to deal with somewhat involved on-disk formats[0] and this might be helpful. [0] The other day, someone was asking for a "tar2ext4" tool, and I thought "hey, that should exist, and I need a side project!". I was prepared to use an annotated hex viewer ( https://hachoir.readthedocs.io/en/latest/wx.html https://hachoir.readthedocs.io/en/latest/wx.html ) and hand roll the encoder/decoder, but I'll happily take tool assistance:)

pjc50 3y ago
Kaitai is good if: - your format is fully known (it's less helpful if you're trying to incrementally build a parser while reverse engineering) - you want to read files, but don't care about writing - you don't mind that the development is not very active For writing "tar2ext4" I would genuinely look at how much work it would be to run the ext4 code from the kernel in a different context; there's a lot of it to consider. Or do what the Apple "dmg" tooling does and make a ramdisk.
- yjftsjthsd-h 3y ago
  Okay, so of interest but maybe not applicable to my usecase. Thanks:) Yeah, it remains to be seen how complex the actual format/code is. Would need to balance the difficulty of recreating it (which I assume to be quite high) against difficulty of extracting kernel code... although https://github.com/lkl/linux https://github.com/lkl/linux exists so for all I know maybe it's easy¯\_(ツ)_/¯ And yes, if I needed to actually write a "tar2ext4" tool today - like, start working in the morning and have it done by EOD - I would absolutely use... actually probably a loopback device rather than a true ramdisk, but yeah. But that requires root access and fiddling with loopback config, which seems excessive for what is, ultimately, just another archive format (from a certain point of view). And honestly some of it is just that it sounds fun to get my hands dirty with filesystem code in userspace:)
  - yjftsjthsd-h 3y ago
    (Forgive my replying to myself; I'm out of the edit window and this seemed good to leave in case someone stumbles across this comment) Yeah so I'm maybe a touch dumb and missed this right in the lkl readme - their demos all but include that exact function: fs2tar - a tool that converts a filesystem image to a tar archive cptofs/cpfromfs - a tool that copies files to/from a filesystem image lklfuse - a tool that can mount a filesystem image in userspace, without root priviledges, using FUSE

pjc50 3y ago

Kaitai is good if: - your format is fully known (it's less helpful if you're trying to incrementally build a parser while reverse engineering) - you want to read files, but don't care about writing - you don't mind that the development is not very active For writing "tar2ext4" I would genuinely look at how much work it would be to run the ext4 code from the kernel in a different context; there's a lot of it to consider. Or do what the Apple "dmg" tooling does and make a ramdisk.

yjftsjthsd-h 3y ago
Okay, so of interest but maybe not applicable to my usecase. Thanks:) Yeah, it remains to be seen how complex the actual format/code is. Would need to balance the difficulty of recreating it (which I assume to be quite high) against difficulty of extracting kernel code... although https://github.com/lkl/linux https://github.com/lkl/linux exists so for all I know maybe it's easy¯\_(ツ)_/¯ And yes, if I needed to actually write a "tar2ext4" tool today - like, start working in the morning and have it done by EOD - I would absolutely use... actually probably a loopback device rather than a true ramdisk, but yeah. But that requires root access and fiddling with loopback config, which seems excessive for what is, ultimately, just another archive format (from a certain point of view). And honestly some of it is just that it sounds fun to get my hands dirty with filesystem code in userspace:)
- yjftsjthsd-h 3y ago
  (Forgive my replying to myself; I'm out of the edit window and this seemed good to leave in case someone stumbles across this comment) Yeah so I'm maybe a touch dumb and missed this right in the lkl readme - their demos all but include that exact function: fs2tar - a tool that converts a filesystem image to a tar archive cptofs/cpfromfs - a tool that copies files to/from a filesystem image lklfuse - a tool that can mount a filesystem image in userspace, without root priviledges, using FUSE