12 ms·

Snebu – Simple Network Encrypting Backup Utility

69 points by derekp7 6y ago

Aachen 6y ago
I'm excited to see another tool for off-site encrypted backups! But I'm also wondering why reinvent the wheel when we have Restic (or does this predate a usable version of restic?). How does it compare? It sounds like Restic does everything this does except it doesn't need a local database and this only skips files that are already uploaded rather than actual de duplication, but I might be wrong there. Why was the design decision made to hash filenames with sha1? I don't see a security flaw in this specific use-case, but why dance with the devil? If it's about shorter hashes, it's safer to truncate a sha2. Since it uses a public key to encrypt to, can a malicious server overwrite your data with encrypted but fake data or is the data also signed using a private or symmetric key on the client side? It might seem like an unusual use case, but why trust the server if we don't have to, and for servers the attacker might know what software it runs and therefore what the file structure is. If you can make someone restore software with modifications of your choosing, that would be rather powerful even if admittedly difficult to pull off. How does that local sqlite database grow, e.g. for a regular amount of files for a 1TB drive, do you get a bunch of gigabytes of state like Duplicity? (I gave up on duplicity because it ate too large a chunk of my ssd while restic didn't need local state at all.) What if the local file is gone, can it (albeit slower I guess) still run normally or do you now need to do a full back-up? Edit: Found one of the answers: it signs the data with hmac "using a combination of the RSA Public key and the passphrase used to protect the RSA Private key" as key. Erm, what? The private key is contained on the client? What's the point of public key encryption if not to not be able to decrypt the data unless you have access to the private key? If the client has the public and the private key, it might as well use symmetric encryption. But what's weirder: the passphrase that protects the private key is used for signing?! Why not use the private key to sign the hash if you have access to it?! And mixing in the public key does nothing, as this is known information to an attacker. Might as well store the plain hash if it weren't for the private key's passphrase.
- lhoff 6y ago
  Inital commit of snebu was from january 6th in 2013 while restic's alpha release was in 2015. So snebu is older. That probably also explains why sha1 was chosen in the first place.
- derekp7 6y ago
  Glad you asked. First, Snebu goes back about 8 years, although the encryption code was developed more recently. That also explains thy it the hash is sha1 (at the time the thinking was "it is good enough for Git"). Right now it is kept for unencrypted backups for backward compatibility with previous versions. But it is something I would like to address -- possibly by using sha-256 (or truncated sha256) or make it user selectable on new repositories. This will be on the next version, didn't want to make too many changes at once. However encrypted backups do use a sha256 HMAC. Compared to Borg and Restic, by making the system server-based, it does limit things like being able to add direc-to-cloud backup, but a future capability may be added to efficiently replicate a Snebu server backup to cloud. However what it gains is better manageability, especially if you are backing up a number of hosts to one repository. In my instance I'm backing up about 50 hosts with 14 daily, 6 weekly, and 12 monthly snapshots. Compared to Restic and Borg, it uses public key encryption (so you don't need to leave a sensitive symmetric key on your clients), but with HMAC-256 based hashes. So an attacker would need both your public key and the HMAC secret (only a one-way hash of the secret is stored on the server). The client reconstructs the HMAC key using the public key and the passphrase for the private key. Oh, and you can use multiple keys -- a client specific key, and a backup key (or more). Deduplication will work across all clients that share the same set of keys (the HMAC of all keys is used for the file storage hash). But for this attack scenario (forged data), you will need to keep the client key file safe and stored with appropriate permissions, and you can use a different key (and lose cross-client deduplication) across clients that are in different trust zones. As for the SQLite DB size -- on one of my installs the DB is taking 50 GB, and is storing over 1500 backup snapshots, from 73 hosts (CensOS and RHEL some with Oracle DBs), with a total of 4.5 TB of disk storage used. The SQLite DB is critical as it stores which file belongs to which directory path on which host -- so it should be backed up independently (at the end of the backup script I use sqlite3 commands to do a dump of it once a week and run a vacuum command). Todo: add these maintenance commands to the Snebu binary to limit the need to run the Sqlite3 CLI. Other features unique to Snebu -- you can have the server "pull" backups without installing an agent on the clients (the "tarcrypt" will still need to be on the client if encryption is needed). You can have the clients "push" data to the server using a restricted user account that allows backups but not deletes or restores, and have a separate restore ID for example. You can give different administrators IDs on the the backup server access to different groups of hosts in a look-but-don't-touch, etc. Another item I like -- let's say you tell it to expire all monthly backups older than 6 weeks. But then if the host hasn't successfully backed up in more than that time (say it has been taken offline), there is a default to keep the most recent 3 backups in a retention schedule, so you won't accidentally lose all you backups in this case (the "preserve" number can be adjusted via command line parameters).
- derekp7 6y ago
  To address the question in your edit: When the key file is created, you are prompted for a password to encrypted the private key. That password is used (along with the public key) to generate the HMAC key which gets recorded in the keyfile and is used to sign each file in the backup. The encrypted private key is sent to the server, and gets sent back when you do the restore, but the HMAC secret key stays on the client. Since you need to type in the password to restore the data, the HMAC key gets re-generated at restore time. The reason the passphrase is hashed with the public key (again using an HMAC-SHA-256 hash) is so that if the same passphrase is used on different hosts with unique encryption keys, the HMAC key still ends up being unique. So a compromised key file on the client will compromise the secret HMAC key, but it won't compromise the encrypted data. Which is better than having a single symmetric encryption key laying around for both encryption and authentication (encryption being the more important item). And the key is tied to the backup snapshot, not to the repository so each client can have their own keys. Further information is at https://www.snebu.com/tarcrypt.html https://www.snebu.com/tarcrypt.html which gives the inner workings of tarcrypt (some of the description of the tar file extensions are dependent on a working knowledge of the basic tar format and the PAX extensions, which are documented in the GNU tar documentation).
Galanwe 6y ago
The article doesn't go into much details on the atomicity of the backups: Are the backups performed while the mount points are still being written to? If the block device is locked during the backup, does the writes fail or just block?
- derekp7 6y ago
  It uses standard "tar" and "find" commands on the client. If you have something like a database, then you should exclude the dbf files and uses the database's tools to put the DB in hot backup mode (using a snebu-client plugin script -- there is a template script in the manpage for snebu-client-plugin(5). You can also create a plugin script that performs a disk snapshot (using LVM snapshot or similar), then mount and backup from that.
candiddevmike 6y ago
To echo the other comment, please use restic or borg instead of this. For being a "simple" tool, it appears more complicated and with more questionably useful features than restic or borg. Borg and restic have server components that allow clients to send backups, I believe restic also allows the clients to manage encryption/not trust the server.
- derekp7 6y ago
  The "simpleness" in Snebu comes from using standardized tools such as find and tar, file storage using LZOP, and the backup catalog in a standard SQLite3 DB, allowing manual recovery of files. However Restic and Borg do have the advantage of writing directly to cloud backups or "dumb" backend data stores such as sftp targets, or API-accessed cloud targets (although Restic doesn't compress and Borg has issues with multiple hosts going to the same repository, they do still have their use cases). Whereas Snebu is more of the use case that would be solved by more complicated to manage tools such as Bacula, Amanda, or commercial backup solutions but with lower setup and administrative requirements.
  - deleted 6y ago
    [deleted]
  - Aachen 6y ago
    > Restic doesn't compress I didn't actually know that, that's kinda weird. It doesn't seem to be vital to deduplication, looking at https://github.com/restic/restic/pull/2441 https://github.com/restic/restic/pull/2441, they just didn't originally implement it and are now wondering how to best introduce this breaking change (can be forwards compatible, but of course not backwards), trying to get it right in one try, keeping the format "robust", adding other breaking changes while they're at it... wonder if this will happen any time soon. To be honest, though, 98% of the time large files (anything over a few hundred kilobytes) is in a format that compresses to a certain extent. Pictures, audio, even text documents and spreadsheets are compressed these days. One person mentions huge XML files which sounds like they really should have been using a different tool (not a text file for starters, if you're not going to read through those gigabytes of text anyway), but there are other use cases like somewhat sparse database tables or disk images or so. Then again, if you have the space for those original files, doubling the space for a reserve copy is usually not a deal breaker, and Restic also deduplicates within and between files so if something were to be very redundant and compressible it is also likely to be caught by that algorithm. I'm not against increasing the complexity to add it, and breaking compatibility of course---it's a clear downside of Restic to not have this---but I also don't really get anyone who considers it a deal breaker (and only to an extent those that find it a big deal). Either way, thanks for pointing this out, I didn't know (and made an incorrect assumption since it seems so standard) and it's good to know about.
izacus 6y ago
Can anyone recommend a Linux (preferrably cross-platform) tool like this which is appropriate for use in personal laptops? It needs to: - Handle random sleeps / disconnects properly without corrupting backup, - Resume backup in progress if the process was interrupted due to sync or network issues. - Be resilient to sleeps/wakeups in a way that it won't just fail to backup the machine for months on end. - (Bonus) Be able to skip backup when connected to mobile hotspots or other slow networks. Any ideas? I'm very happy with Arq backup on Windows/macOS, but I just can't seem to find anything on Linux. Everything just assumes a hardwired workstation/server and completely falls apart on a laptop.
- specktr 6y ago
  I’d also be interested in recommendations that meet the above requirements. My current work around for my main linux laptop is setting rclone to run with crontab at a time that I know i’ll be online for an extended period (evenings during long homework sessions). I’ve been doing this for several years and have yet to run into major show stopping bugs, and I frequently do checksum verification on my backups.
  - izacus 6y ago
    Yeah, that's sadly not very useful for laptops I manage - people tend to turn them on and off at random times and I had multiple cases where the backups just ended up in hosed state (because the tool got interrupted at a bad time) or just didn't backup for week because the laptop never ran for enough time without interruption after creating a huge file.
- derekp7 6y ago
  What about a split or shadow repository? That is, have a host-specific local cache of the metadata from the main backup server, so that a snapshot backup can be made locally (say to an attached thumb drive or SD card), then have a process that sync's that up periodically when a stable network connection is available? If this is a strong enough use case then I have a couple ideas on how to implement it (I'll throw it in the wishlist). Otherwise, currently if you have an interrupted backup the next one will still re-use the files already on the server, even if it is part of a partial backup.
karmakaze 6y ago
Catchy name, sounds of a Dr. Seuss creation. I used to use byobu and rsync all the time thinking that byobu was a wrapper script around 'screen' for bring your own back-up. Later learned it's Japanese for those folding room divider screens. Perhaps someone could draw a snebu for the project.

69 points by derekp7 6y ago

I'm excited to see another tool for off-site encrypted backups! But I'm also wondering why reinvent the wheel when we have Restic (or does this predate a usable version of restic?). How does it compare? It sounds like Restic does everything this does except it doesn't need a local database and this only skips files that are already uploaded rather than actual de duplication, but I might be wrong there. Why was the design decision made to hash filenames with sha1? I don't see a security flaw in this specific use-case, but why dance with the devil? If it's about shorter hashes, it's safer to truncate a sha2. Since it uses a public key to encrypt to, can a malicious server overwrite your data with encrypted but fake data or is the data also signed using a private or symmetric key on the client side? It might seem like an unusual use case, but why trust the server if we don't have to, and for servers the attacker might know what software it runs and therefore what the file structure is. If you can make someone restore software with modifications of your choosing, that would be rather powerful even if admittedly difficult to pull off. How does that local sqlite database grow, e.g. for a regular amount of files for a 1TB drive, do you get a bunch of gigabytes of state like Duplicity? (I gave up on duplicity because it ate too large a chunk of my ssd while restic didn't need local state at all.) What if the local file is gone, can it (albeit slower I guess) still run normally or do you now need to do a full back-up? Edit: Found one of the answers: it signs the data with hmac "using a combination of the RSA Public key and the passphrase used to protect the RSA Private key" as key. Erm, what? The private key is contained on the client? What's the point of public key encryption if not to not be able to decrypt the data unless you have access to the private key? If the client has the public and the private key, it might as well use symmetric encryption. But what's weirder: the passphrase that protects the private key is used for signing?! Why not use the private key to sign the hash if you have access to it?! And mixing in the public key does nothing, as this is known information to an attacker. Might as well store the plain hash if it weren't for the private key's passphrase.

lhoff 6y ago
Inital commit of snebu was from january 6th in 2013 while restic's alpha release was in 2015. So snebu is older. That probably also explains why sha1 was chosen in the first place.
derekp7 6y ago
Glad you asked. First, Snebu goes back about 8 years, although the encryption code was developed more recently. That also explains thy it the hash is sha1 (at the time the thinking was "it is good enough for Git"). Right now it is kept for unencrypted backups for backward compatibility with previous versions. But it is something I would like to address -- possibly by using sha-256 (or truncated sha256) or make it user selectable on new repositories. This will be on the next version, didn't want to make too many changes at once. However encrypted backups do use a sha256 HMAC. Compared to Borg and Restic, by making the system server-based, it does limit things like being able to add direc-to-cloud backup, but a future capability may be added to efficiently replicate a Snebu server backup to cloud. However what it gains is better manageability, especially if you are backing up a number of hosts to one repository. In my instance I'm backing up about 50 hosts with 14 daily, 6 weekly, and 12 monthly snapshots. Compared to Restic and Borg, it uses public key encryption (so you don't need to leave a sensitive symmetric key on your clients), but with HMAC-256 based hashes. So an attacker would need both your public key and the HMAC secret (only a one-way hash of the secret is stored on the server). The client reconstructs the HMAC key using the public key and the passphrase for the private key. Oh, and you can use multiple keys -- a client specific key, and a backup key (or more). Deduplication will work across all clients that share the same set of keys (the HMAC of all keys is used for the file storage hash). But for this attack scenario (forged data), you will need to keep the client key file safe and stored with appropriate permissions, and you can use a different key (and lose cross-client deduplication) across clients that are in different trust zones. As for the SQLite DB size -- on one of my installs the DB is taking 50 GB, and is storing over 1500 backup snapshots, from 73 hosts (CensOS and RHEL some with Oracle DBs), with a total of 4.5 TB of disk storage used. The SQLite DB is critical as it stores which file belongs to which directory path on which host -- so it should be backed up independently (at the end of the backup script I use sqlite3 commands to do a dump of it once a week and run a vacuum command). Todo: add these maintenance commands to the Snebu binary to limit the need to run the Sqlite3 CLI. Other features unique to Snebu -- you can have the server "pull" backups without installing an agent on the clients (the "tarcrypt" will still need to be on the client if encryption is needed). You can have the clients "push" data to the server using a restricted user account that allows backups but not deletes or restores, and have a separate restore ID for example. You can give different administrators IDs on the the backup server access to different groups of hosts in a look-but-don't-touch, etc. Another item I like -- let's say you tell it to expire all monthly backups older than 6 weeks. But then if the host hasn't successfully backed up in more than that time (say it has been taken offline), there is a default to keep the most recent 3 backups in a retention schedule, so you won't accidentally lose all you backups in this case (the "preserve" number can be adjusted via command line parameters).
derekp7 6y ago
To address the question in your edit: When the key file is created, you are prompted for a password to encrypted the private key. That password is used (along with the public key) to generate the HMAC key which gets recorded in the keyfile and is used to sign each file in the backup. The encrypted private key is sent to the server, and gets sent back when you do the restore, but the HMAC secret key stays on the client. Since you need to type in the password to restore the data, the HMAC key gets re-generated at restore time. The reason the passphrase is hashed with the public key (again using an HMAC-SHA-256 hash) is so that if the same passphrase is used on different hosts with unique encryption keys, the HMAC key still ends up being unique. So a compromised key file on the client will compromise the secret HMAC key, but it won't compromise the encrypted data. Which is better than having a single symmetric encryption key laying around for both encryption and authentication (encryption being the more important item). And the key is tied to the backup snapshot, not to the repository so each client can have their own keys. Further information is at https://www.snebu.com/tarcrypt.html https://www.snebu.com/tarcrypt.html which gives the inner workings of tarcrypt (some of the description of the tar file extensions are dependent on a working knowledge of the basic tar format and the PAX extensions, which are documented in the GNU tar documentation).

Galanwe 6y ago

The article doesn't go into much details on the atomicity of the backups: Are the backups performed while the mount points are still being written to? If the block device is locked during the backup, does the writes fail or just block?

derekp7 6y ago
It uses standard "tar" and "find" commands on the client. If you have something like a database, then you should exclude the dbf files and uses the database's tools to put the DB in hot backup mode (using a snebu-client plugin script -- there is a template script in the manpage for snebu-client-plugin(5). You can also create a plugin script that performs a disk snapshot (using LVM snapshot or similar), then mount and backup from that.

candiddevmike 6y ago

To echo the other comment, please use restic or borg instead of this. For being a "simple" tool, it appears more complicated and with more questionably useful features than restic or borg. Borg and restic have server components that allow clients to send backups, I believe restic also allows the clients to manage encryption/not trust the server.

derekp7 6y ago
The "simpleness" in Snebu comes from using standardized tools such as find and tar, file storage using LZOP, and the backup catalog in a standard SQLite3 DB, allowing manual recovery of files. However Restic and Borg do have the advantage of writing directly to cloud backups or "dumb" backend data stores such as sftp targets, or API-accessed cloud targets (although Restic doesn't compress and Borg has issues with multiple hosts going to the same repository, they do still have their use cases). Whereas Snebu is more of the use case that would be solved by more complicated to manage tools such as Bacula, Amanda, or commercial backup solutions but with lower setup and administrative requirements.
- deleted 6y ago
  [deleted]
- Aachen 6y ago
  > Restic doesn't compress I didn't actually know that, that's kinda weird. It doesn't seem to be vital to deduplication, looking at https://github.com/restic/restic/pull/2441 https://github.com/restic/restic/pull/2441, they just didn't originally implement it and are now wondering how to best introduce this breaking change (can be forwards compatible, but of course not backwards), trying to get it right in one try, keeping the format "robust", adding other breaking changes while they're at it... wonder if this will happen any time soon. To be honest, though, 98% of the time large files (anything over a few hundred kilobytes) is in a format that compresses to a certain extent. Pictures, audio, even text documents and spreadsheets are compressed these days. One person mentions huge XML files which sounds like they really should have been using a different tool (not a text file for starters, if you're not going to read through those gigabytes of text anyway), but there are other use cases like somewhat sparse database tables or disk images or so. Then again, if you have the space for those original files, doubling the space for a reserve copy is usually not a deal breaker, and Restic also deduplicates within and between files so if something were to be very redundant and compressible it is also likely to be caught by that algorithm. I'm not against increasing the complexity to add it, and breaking compatibility of course---it's a clear downside of Restic to not have this---but I also don't really get anyone who considers it a deal breaker (and only to an extent those that find it a big deal). Either way, thanks for pointing this out, I didn't know (and made an incorrect assumption since it seems so standard) and it's good to know about.

izacus 6y ago

Can anyone recommend a Linux (preferrably cross-platform) tool like this which is appropriate for use in personal laptops? It needs to: - Handle random sleeps / disconnects properly without corrupting backup, - Resume backup in progress if the process was interrupted due to sync or network issues. - Be resilient to sleeps/wakeups in a way that it won't just fail to backup the machine for months on end. - (Bonus) Be able to skip backup when connected to mobile hotspots or other slow networks. Any ideas? I'm very happy with Arq backup on Windows/macOS, but I just can't seem to find anything on Linux. Everything just assumes a hardwired workstation/server and completely falls apart on a laptop.

specktr 6y ago
I’d also be interested in recommendations that meet the above requirements. My current work around for my main linux laptop is setting rclone to run with crontab at a time that I know i’ll be online for an extended period (evenings during long homework sessions). I’ve been doing this for several years and have yet to run into major show stopping bugs, and I frequently do checksum verification on my backups.
- izacus 6y ago
  Yeah, that's sadly not very useful for laptops I manage - people tend to turn them on and off at random times and I had multiple cases where the backups just ended up in hosed state (because the tool got interrupted at a bad time) or just didn't backup for week because the laptop never ran for enough time without interruption after creating a huge file.
derekp7 6y ago
What about a split or shadow repository? That is, have a host-specific local cache of the metadata from the main backup server, so that a snapshot backup can be made locally (say to an attached thumb drive or SD card), then have a process that sync's that up periodically when a stable network connection is available? If this is a strong enough use case then I have a couple ideas on how to implement it (I'll throw it in the wishlist). Otherwise, currently if you have an interrupted backup the next one will still re-use the files already on the server, even if it is part of a partial backup.

karmakaze 6y ago

Catchy name, sounds of a Dr. Seuss creation. I used to use byobu and rsync all the time thinking that byobu was a wrapper script around 'screen' for bring your own back-up. Later learned it's Japanese for those folding room divider screens. Perhaps someone could draw a snebu for the project.