3 ms·

It’s a shame you can’t cat tar archives together. It always seemed a neat feature of container formats like mpeg: cat chapter*.mpg >movie.mpg Are there

It’s a shame you can’t cat tar archives together. It always seemed a neat feature of container formats like mpeg: cat chapter*.mpg >movie.mpg Are there any file archive formats that support this?

hddqsb 6y ago
For some reason I thought this was a standard feature of tar, and was surprised when it didn't work. It turns out that GNU tar supports extraction of concatenated tarballs, but it requires the --ignore-zeros (-i) option: > Normally, tar stops reading when it encounters a block of zeros between file entries (which usually indicates the end of the archive). ‘--ignore-zeros’ (‘-i’) allows tar to completely read an archive which contains a block of zeros before the end (i.e., a damaged archive, or one that was created by concatenating several archives together). -- https://www.gnu.org/software/tar/manual/html_node/Ignore-Zeros.html https://www.gnu.org/software/tar/manual/html_node/Ignore-Zer... (As a bonus, this also works for concatenated .tar.gz files because gzip supports concatenation.) Regarding other formats that support concatenation, the Linux kernel's initramfs format is interesting because it is based on CPIO but is explicitly defined to be the concatenation of CPIO archives: https://www.kernel.org/doc/html/latest/driver-api/early-userspace/buffer-format.html https://www.kernel.org/doc/html/latest/driver-api/early-user... CPIO itself doesn't support concatenation. There is a neat hack to extract such concatenated archives: while cpio -i; do :; done < archive.cpio (source: https://unix.stackexchange.com/a/266090; https://unix.stackexchange.com/a/266090; also mentioned there is the tool skipcpio, which is part of dracut: https://github.com/dracutdevs/dracut/blob/master/skipcpio/skipcpio.c https://github.com/dracutdevs/dracut/blob/master/skipcpio/sk...) This hack would also work for extracting concatenating tarballs (without GNU tar's --ignore-zeros option). One annoyance is that it shows a warning when it gets to the end of the archive.
LeoPanthera 6y ago
You can cat .mpg files together, but you should not. ".mpg" is supposed to refer to an MPEG Program Stream, which does in fact have a header. You can cat Transport Streams, but these are supposed to have the ".ts" extension. You sometimes get transports streams named ".mpg", which complicates everything.

It’s a shame you can’t cat tar archives together. It always seemed a neat feature of container formats like mpeg: cat chapter*.mpg >movie.mpg Are there any file archive formats that support this?

hddqsb 6y ago

For some reason I thought this was a standard feature of tar, and was surprised when it didn't work. It turns out that GNU tar supports extraction of concatenated tarballs, but it requires the --ignore-zeros (-i) option: > Normally, tar stops reading when it encounters a block of zeros between file entries (which usually indicates the end of the archive). ‘--ignore-zeros’ (‘-i’) allows tar to completely read an archive which contains a block of zeros before the end (i.e., a damaged archive, or one that was created by concatenating several archives together). -- https://www.gnu.org/software/tar/manual/html_node/Ignore-Zeros.html https://www.gnu.org/software/tar/manual/html_node/Ignore-Zer... (As a bonus, this also works for concatenated .tar.gz files because gzip supports concatenation.) Regarding other formats that support concatenation, the Linux kernel's initramfs format is interesting because it is based on CPIO but is explicitly defined to be the concatenation of CPIO archives: https://www.kernel.org/doc/html/latest/driver-api/early-userspace/buffer-format.html https://www.kernel.org/doc/html/latest/driver-api/early-user... CPIO itself doesn't support concatenation. There is a neat hack to extract such concatenated archives: while cpio -i; do :; done < archive.cpio (source: https://unix.stackexchange.com/a/266090; https://unix.stackexchange.com/a/266090; also mentioned there is the tool skipcpio, which is part of dracut: https://github.com/dracutdevs/dracut/blob/master/skipcpio/skipcpio.c https://github.com/dracutdevs/dracut/blob/master/skipcpio/sk...) This hack would also work for extracting concatenating tarballs (without GNU tar's --ignore-zeros option). One annoyance is that it shows a warning when it gets to the end of the archive.

LeoPanthera 6y ago

You can cat .mpg files together, but you should not. ".mpg" is supposed to refer to an MPEG Program Stream, which does in fact have a header. You can cat Transport Streams, but these are supposed to have the ".ts" extension. You sometimes get transports streams named ".mpg", which complicates everything.