5 ms·

I wish that there were some more tools to build archives based on script-like description. E.g. "i want this file under this path with this metadata, go build m

I wish that there were some more tools to build archives based on script-like description. E.g. "i want this file under this path with this metadata, go build me a zip". Without it, my favorite way of building images rootless is still fakeroot. fakeroot + mknod + cpio can fully replace gen_init_cpio

Retr0id 3y ago
The python `zipfile` module (`tarfile` likewise) is fairly straight-forward to use for this use case (just with a literal script instead of a script-like).
- danudey 3y ago
  Using the zipfile/tarfile/etc. modules' functionality, it's easy to write a script that would take an input file of any format you want (such as the one in the example) and create an output file the same way it works in the article. It also allows for you to create archives compressed with gz, bz2, lzma, xz (via lzma), or potentially any other option that Python can provide.
  - Retr0id 3y ago
    I did consider writing such a script as an example, but it occurred to me that just using ad-hoc imperative python scripts might be simpler/clearer (i.e. do we actually benefit from using a "DSL" here, even if it's a trivial one?)
DHowett 3y ago
bsdtar can do this! It supports reading an mtree file (a text description of a filesystem or of the contents of an archive) as an input and producing ZIP as an output! Given an mtree file foo.mtree: #mtree baz/bar type=file content=quux You can produce a ZIP containing baz/bar with: bsdtar -c --format=zip -f out.zip @foo.mtree There’s a much better explanation in the bsdtar manual at https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&format=html#end https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&f... If you’re on Windows, “tar.exe” is bsdtar. With the caveat that the mtree file should be saved with LF line endings, this should still work.
akdev1l 3y ago
You can use podman and skip that dance. Just install gen_init_cpio from a rootless container.
- deleted 3y ago
  [deleted]
- blueflow 3y ago
  It seems like podman needs root to enable containers to run mknod.
  - zygology 3y ago
    You don't actually have to make a device node, you can just create it directly within the CPIO archive.
amluto 3y ago
Here's my old Python CPIO writer. It's not very maintained right now and it's probably missing some useful file types, but it works fine! https://github.com/amluto/virtme/blob/master/virtme/cpiowriter.py https://github.com/amluto/virtme/blob/master/virtme/cpiowrit...
mikepurvis 3y ago
Yes please, and for archive types like ISOs and disk images (where there may be a partition table in play as well). Debian has jigdo, but the recipe/template for it is derived after that fact by scanning the ISO and then blanking out the parts of it that match external files— there's no mechanism for authoring them pre asset creation.

Retr0id 3y ago

The python `zipfile` module (`tarfile` likewise) is fairly straight-forward to use for this use case (just with a literal script instead of a script-like).

danudey 3y ago
Using the zipfile/tarfile/etc. modules' functionality, it's easy to write a script that would take an input file of any format you want (such as the one in the example) and create an output file the same way it works in the article. It also allows for you to create archives compressed with gz, bz2, lzma, xz (via lzma), or potentially any other option that Python can provide.
- Retr0id 3y ago
  I did consider writing such a script as an example, but it occurred to me that just using ad-hoc imperative python scripts might be simpler/clearer (i.e. do we actually benefit from using a "DSL" here, even if it's a trivial one?)

DHowett 3y ago

bsdtar can do this! It supports reading an mtree file (a text description of a filesystem or of the contents of an archive) as an input and producing ZIP as an output! Given an mtree file foo.mtree: #mtree baz/bar type=file content=quux You can produce a ZIP containing baz/bar with: bsdtar -c --format=zip -f out.zip @foo.mtree There’s a much better explanation in the bsdtar manual at https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&format=html#end https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&f... If you’re on Windows, “tar.exe” is bsdtar. With the caveat that the mtree file should be saved with LF line endings, this should still work.

akdev1l 3y ago

You can use podman and skip that dance. Just install gen_init_cpio from a rootless container.

deleted 3y ago
[deleted]
blueflow 3y ago
It seems like podman needs root to enable containers to run mknod.
- zygology 3y ago
  You don't actually have to make a device node, you can just create it directly within the CPIO archive.

amluto 3y ago

Here's my old Python CPIO writer. It's not very maintained right now and it's probably missing some useful file types, but it works fine! https://github.com/amluto/virtme/blob/master/virtme/cpiowriter.py https://github.com/amluto/virtme/blob/master/virtme/cpiowrit...

mikepurvis 3y ago

Yes please, and for archive types like ISOs and disk images (where there may be a partition table in play as well). Debian has jigdo, but the recipe/template for it is derived after that fact by scanning the ISO and then blanking out the parts of it that match external files— there's no mechanism for authoring them pre asset creation.