4 ms·

FSArchiver – Filesystem Archiver for Linux

53 points by electricant 4y ago

loeg 4y ago
Seems like tar, with internal checksums (instead of external checksums or FEC). I'd just stick with tar and add some par2 blocks if you are worried about bitrot.
- CaliforniaKarl 4y ago
  Interesting! Can you please point to an example?
  - mbreese 4y ago
    Here's the first example I could find. https://pthree.org/2014/04/01/protect-against-bit-rot-with-parchive/ https://pthree.org/2014/04/01/protect-against-bit-rot-with-p... Basically, you take a normal TAR file and creates external parity files that can be used to recover data in the case of bitrot. Something like this would have been great to have on a few data files I had to retrieve from tape that ended up with a bit flip somewhere in the network > disk > tape > disk process. However, it doesn't look like the program is maintained anymore... http://parchive.sourceforge.net/ http://parchive.sourceforge.net/ > DISCLAIMER: This project web space is not actively mantained and is presented here for archive purposes. However some project members still montior the project mailings lists if you have questions. (sic) The last release was in 2004 (Wikipedia says it was active until 2015, but I don't see that).
    - pessimizer 4y ago
      Seems easier and safer to use fsarchiver.
      - loeg 4y ago
        fsarchiver doesn't do FEC; it can only detect corruption, not fix it.
    - kevin_thibedeau 4y ago
      Feature complete tools don't need updates.
    - notpeter 4y ago
      Parchive development has moved GitHub. PAR2 (libpar2/par2cmdline) continues to see active (if sporadic) maintenance. The most recent tagged release was 2020-02-09 but there've been a handful of PR's merged since. [2] PAR3 has a reference implementation and alpha spec (libpar3/par3cmdline) which is based around Blake3. [3] OG PAR (libpar/parcmdline) is legit unmaintained; the last release was 21 years ago. [1] [2]: https://github.com/Parchive/par2cmdline https://github.com/Parchive/par2cmdline [3]: https://github.com/Parchive/par3cmdline https://github.com/Parchive/par3cmdline [1]: https://github.com/Parchive/parcmdline/ https://github.com/Parchive/parcmdline/
      - mbreese 4y ago
        Thanks for the update -- it was hard to find any information on the program. And I guess, more importantly, your first link (well, [2]), is also the source used for at least the Debian package, so it's at least more up to date!
- woleium 4y ago
  the third line of the site says: Unlike tar/dar, FSArchiver also creates the file-system when it extracts the data to partitions
  - loeg 4y ago
    That isn't something I've ever wanted from tar. But if I did, newfs is easy to use.
- pessimizer 4y ago
  > I'd just stick with tar Why?
  - loeg 4y ago
    Because it works?
    - Nasreddin_Hodja 4y ago
      Fsarchiver works better for me. With tar I have to manually specify a lot to exclude when making full filesystem backups, at least submountpoints. With fsarchiver you don't have to.
- naniwaduni 4y ago
  Even if you're going to layer par2 over an archive, I wouldn't want the underlying archive format to be tar.
cortesoft 4y ago
> If the archive is corrupt, you just loose the current file, not the whole archive. > In case one file is damaged, FSArchiver will restore all the other files from your archive, so you won’t loose all your data. Loose instead of lose is one my biggest pet peeves, and it appears at least twice on the main page!
- JadeNB 4y ago
  We all have our pet peeves, and this is one of mine, too. But it's surely orthogonal to issues of code reliability—someone can definitely write flawless English but terrible code, and it's definitely believable that the reverse could happen—so it's not clear what's the relevance here.
  - cortesoft 4y ago
    Honestly, the biggest thing it tells me is that this is likely a one person project… there was clearly no one proof reading or reviewing this text. I am always a bit suspect of relying on single person projects.
dataflow 4y ago
This sounds like it competes with... squashfs? Would be nice for them to have a comparison table.
- Nasreddin_Hodja 4y ago
  Unfortunately fsarchive files aren't mountable. But mksquashfs is not useful for archiving due to its lack of some options. There is squashfs-tools-ng though, which can convert tar into squashfs. But anyway, fsarchiver is my tool of choice for fs backups.
ninefathom 4y ago
This seems like it could be a useful tool. At first read, there's nothing here that cpio, md5sum, *zip*, and mkfs in tandem couldn't do, but it simplifies the process. The real question in my mind is if the "attic space" of learning a new tool and dragging it around everywhere is worth the convenience versus doing it The Old, Boring, Manual Way(TM). I suppose time will tell...
- Self-Perfection 4y ago
  FSArchiver reduces cognitive load by guaranteeing that resulting archive is accurate and exact mirror of original filesystem. AFAIK, cpio does not support extended attributes and one have to remember to call tar with specific flags to store xattr.
  - ninefathom 4y ago
    "by guaranteeing that resulting archive is accurate and exact mirror of original filesystem." ^^^ that is true enough "reduces cognitive load" ^^^ that I have doubts about. I expect this would be highly variable between people, which leads to my question about the trade-off. For those who are newcomers (say ten years or fewer) to the *nix world, it may well be an improvement. For those who have been using the standard-ish tools for 30+ years, learning yet-another-new-utility has a very high cost. That dilemma would be irrelevant if the person taking backups is the same person doing the restoration (as will often be the case). My concern is along the lines of "what if this becomes a de facto standard?"
8fingerlouie 4y ago
Isn't this basically just "tar cvzf output.tar.gz /source/path" ? I'm sure there's more to it, but reading the description tar seems to check most of (if not all) the boxes. - File attributes (ownership, permissions, dates, etc) - Extended attributes - Multiple filesystems (tar is somewhat filesystem agnostic) - Splitting into multiple volumes (tar -M for multi volume) - Checksumming (tar does checksums on metadata, but not data) - Restoring corrupt archives (tar skips unreadable parts of the data stream)

53 points by electricant 4y ago

loeg 4y ago

Seems like tar, with internal checksums (instead of external checksums or FEC). I'd just stick with tar and add some par2 blocks if you are worried about bitrot.

CaliforniaKarl 4y ago
Interesting! Can you please point to an example?
- mbreese 4y ago
  Here's the first example I could find. https://pthree.org/2014/04/01/protect-against-bit-rot-with-parchive/ https://pthree.org/2014/04/01/protect-against-bit-rot-with-p... Basically, you take a normal TAR file and creates external parity files that can be used to recover data in the case of bitrot. Something like this would have been great to have on a few data files I had to retrieve from tape that ended up with a bit flip somewhere in the network > disk > tape > disk process. However, it doesn't look like the program is maintained anymore... http://parchive.sourceforge.net/ http://parchive.sourceforge.net/ > DISCLAIMER: This project web space is not actively mantained and is presented here for archive purposes. However some project members still montior the project mailings lists if you have questions. (sic) The last release was in 2004 (Wikipedia says it was active until 2015, but I don't see that).
  - pessimizer 4y ago
    Seems easier and safer to use fsarchiver.
    - loeg 4y ago
      fsarchiver doesn't do FEC; it can only detect corruption, not fix it.
  - kevin_thibedeau 4y ago
    Feature complete tools don't need updates.
  - notpeter 4y ago
    Parchive development has moved GitHub. PAR2 (libpar2/par2cmdline) continues to see active (if sporadic) maintenance. The most recent tagged release was 2020-02-09 but there've been a handful of PR's merged since. [2] PAR3 has a reference implementation and alpha spec (libpar3/par3cmdline) which is based around Blake3. [3] OG PAR (libpar/parcmdline) is legit unmaintained; the last release was 21 years ago. [1] [2]: https://github.com/Parchive/par2cmdline https://github.com/Parchive/par2cmdline [3]: https://github.com/Parchive/par3cmdline https://github.com/Parchive/par3cmdline [1]: https://github.com/Parchive/parcmdline/ https://github.com/Parchive/parcmdline/
    - mbreese 4y ago
      Thanks for the update -- it was hard to find any information on the program. And I guess, more importantly, your first link (well, [2]), is also the source used for at least the Debian package, so it's at least more up to date!
woleium 4y ago
the third line of the site says: Unlike tar/dar, FSArchiver also creates the file-system when it extracts the data to partitions
- loeg 4y ago
  That isn't something I've ever wanted from tar. But if I did, newfs is easy to use.
pessimizer 4y ago
> I'd just stick with tar Why?
- loeg 4y ago
  Because it works?
  - Nasreddin_Hodja 4y ago
    Fsarchiver works better for me. With tar I have to manually specify a lot to exclude when making full filesystem backups, at least submountpoints. With fsarchiver you don't have to.
naniwaduni 4y ago
Even if you're going to layer par2 over an archive, I wouldn't want the underlying archive format to be tar.

cortesoft 4y ago

> If the archive is corrupt, you just loose the current file, not the whole archive. > In case one file is damaged, FSArchiver will restore all the other files from your archive, so you won’t loose all your data. Loose instead of lose is one my biggest pet peeves, and it appears at least twice on the main page!

JadeNB 4y ago
We all have our pet peeves, and this is one of mine, too. But it's surely orthogonal to issues of code reliability—someone can definitely write flawless English but terrible code, and it's definitely believable that the reverse could happen—so it's not clear what's the relevance here.
- cortesoft 4y ago
  Honestly, the biggest thing it tells me is that this is likely a one person project… there was clearly no one proof reading or reviewing this text. I am always a bit suspect of relying on single person projects.

dataflow 4y ago

This sounds like it competes with... squashfs? Would be nice for them to have a comparison table.

Nasreddin_Hodja 4y ago
Unfortunately fsarchive files aren't mountable. But mksquashfs is not useful for archiving due to its lack of some options. There is squashfs-tools-ng though, which can convert tar into squashfs. But anyway, fsarchiver is my tool of choice for fs backups.

ninefathom 4y ago

This seems like it could be a useful tool. At first read, there's nothing here that cpio, md5sum, *zip*, and mkfs in tandem couldn't do, but it simplifies the process. The real question in my mind is if the "attic space" of learning a new tool and dragging it around everywhere is worth the convenience versus doing it The Old, Boring, Manual Way(TM). I suppose time will tell...

Self-Perfection 4y ago
FSArchiver reduces cognitive load by guaranteeing that resulting archive is accurate and exact mirror of original filesystem. AFAIK, cpio does not support extended attributes and one have to remember to call tar with specific flags to store xattr.
- ninefathom 4y ago
  "by guaranteeing that resulting archive is accurate and exact mirror of original filesystem." ^^^ that is true enough "reduces cognitive load" ^^^ that I have doubts about. I expect this would be highly variable between people, which leads to my question about the trade-off. For those who are newcomers (say ten years or fewer) to the *nix world, it may well be an improvement. For those who have been using the standard-ish tools for 30+ years, learning yet-another-new-utility has a very high cost. That dilemma would be irrelevant if the person taking backups is the same person doing the restoration (as will often be the case). My concern is along the lines of "what if this becomes a de facto standard?"

8fingerlouie 4y ago

Isn't this basically just "tar cvzf output.tar.gz /source/path" ? I'm sure there's more to it, but reading the description tar seems to check most of (if not all) the boxes. - File attributes (ownership, permissions, dates, etc) - Extended attributes - Multiple filesystems (tar is somewhat filesystem agnostic) - Splitting into multiple volumes (tar -M for multi volume) - Checksumming (tar does checksums on metadata, but not data) - Restoring corrupt archives (tar skips unreadable parts of the data stream)