4 ms·

Two reasons why I do not want to use the service in its current state (meant as constructive criticism): 1. If I would send my backups to some SASS there is no

Two reasons why I do not want to use the service in its current state (meant as constructive criticism): 1. If I would send my backups to some SASS there is no way I would do that without encryption. 2. I like to backup my filesystem and not just the files in it (to make sure I've got everything and make restoring easy). Currently, I just dd my block devices, but I am sure that could be optimized to not upload a complete image every time. Sure, I could solve those two problems myself, but then I could also use AWS S3/Backblaze ;-)

ars 8y ago
> Currently, I just dd my block devices That's actually not a good idea at all. It's very fragile, the slightest problem and you may lose the entire backup. Silent corruption may get backed up for months and you won't notice until it's too late. Doing a restore means having space for the full file, just to restore a single small file. If you don't know when your file changed you may have to do that multiple times instead of just being able to check when that single file changed. In short: don't do that. Copy the full directory structure, sure. But not the disk image.
- cellularmitosis 8y ago
  Rather than "don't do that", I would say "also, backup individual files". Fragile seems to be a matter of perspective. I do a dd block-device backup before doing anything "risky" with a machine (i.e. upgrading major OS version, etc), because it is by far the most bullet-proof / fail-proof way to go back in time.
- arendtio 8y ago
  Why is dd fragile? I mean, yes doing so while the filesystem is mounted is like Russian Roulette, but if the filesystem is unmounted? I see that it isn't very efficient, but I never had any problems with reliability. Silent corruption might actually be a problem, but that is something you won't solve by backing up files and directories instead of block devices. After all, a block devices backup just contains more information than the files and directories backup. The only advantage of filesystem backups is that you can easier validate if a file should have changed, but even if you detect that it changed even if it shouldn't have, you still need some kind of checksum or so to find out which version is correct. On the other hand, if you back up the files and directories you have to care about the filesystem type and if there are special types like links, devices nodes and the like. On that side, I had enough unpleasant experiences that I am trying to avoid that trouble. To restore single files I can simply mount the image on a loop device, so no problem there.
  - zulgan 8y ago
    there is absolutely no technical problem doing it on unmounted filesystem but it just imposes certain restrictions that I dont think are worth their cost considering in many cases you want to copy live files, you have to copy them to place you can unmount, so now you have 2 problems if you dont gzip the dd you risk pure cosmic microwave background bitflips and no checksums, and they can corrupt superblocks and etc if you gzip it (to have crc sums) you cant restore partial files from it so easily unless you use squashfs and significantly complicate the process so tar and gzip is more robust (compared to the dd) just because of the crc (of gzip not of tar), size and portability
zulgan 8y ago
tbh you I wouldnt trust anyone with my encryption, that is why in Baxx I advise people to simply upload encrypted cat file | encrypt -k .pass | curl --data-binary cat file | encrypt -k .pass > file.enc && curl -T file.enc https://baxx.dev/io/$BAXX_TOKEN/file (encrypt: https://github.com/jackdoe/updown/blob/master/cmd/encrypt/main.go https://github.com/jackdoe/updown/blob/master/cmd/encrypt/ma...) offtopic: it is a bit annoying that cat | curl makes curl buffed the whole standard input, so for big files you have to either you have to do -H "Transfer-Encoding: chunked" or use -T file 2) dd block devices is very fragile, its better if you just tar --exclude/dev ... (https://help.ubuntu.com/community/BackupYourSystem/TAR https://help.ubuntu.com/community/BackupYourSystem/TAR - Alternate backup)

ars 8y ago

> Currently, I just dd my block devices That's actually not a good idea at all. It's very fragile, the slightest problem and you may lose the entire backup. Silent corruption may get backed up for months and you won't notice until it's too late. Doing a restore means having space for the full file, just to restore a single small file. If you don't know when your file changed you may have to do that multiple times instead of just being able to check when that single file changed. In short: don't do that. Copy the full directory structure, sure. But not the disk image.

cellularmitosis 8y ago
Rather than "don't do that", I would say "also, backup individual files". Fragile seems to be a matter of perspective. I do a dd block-device backup before doing anything "risky" with a machine (i.e. upgrading major OS version, etc), because it is by far the most bullet-proof / fail-proof way to go back in time.
arendtio 8y ago
Why is dd fragile? I mean, yes doing so while the filesystem is mounted is like Russian Roulette, but if the filesystem is unmounted? I see that it isn't very efficient, but I never had any problems with reliability. Silent corruption might actually be a problem, but that is something you won't solve by backing up files and directories instead of block devices. After all, a block devices backup just contains more information than the files and directories backup. The only advantage of filesystem backups is that you can easier validate if a file should have changed, but even if you detect that it changed even if it shouldn't have, you still need some kind of checksum or so to find out which version is correct. On the other hand, if you back up the files and directories you have to care about the filesystem type and if there are special types like links, devices nodes and the like. On that side, I had enough unpleasant experiences that I am trying to avoid that trouble. To restore single files I can simply mount the image on a loop device, so no problem there.
- zulgan 8y ago
  there is absolutely no technical problem doing it on unmounted filesystem but it just imposes certain restrictions that I dont think are worth their cost considering in many cases you want to copy live files, you have to copy them to place you can unmount, so now you have 2 problems if you dont gzip the dd you risk pure cosmic microwave background bitflips and no checksums, and they can corrupt superblocks and etc if you gzip it (to have crc sums) you cant restore partial files from it so easily unless you use squashfs and significantly complicate the process so tar and gzip is more robust (compared to the dd) just because of the crc (of gzip not of tar), size and portability

zulgan 8y ago

tbh you I wouldnt trust anyone with my encryption, that is why in Baxx I advise people to simply upload encrypted cat file | encrypt -k .pass | curl --data-binary cat file | encrypt -k .pass > file.enc && curl -T file.enc https://baxx.dev/io/$BAXX_TOKEN/file (encrypt: https://github.com/jackdoe/updown/blob/master/cmd/encrypt/main.go https://github.com/jackdoe/updown/blob/master/cmd/encrypt/ma...) offtopic: it is a bit annoying that cat | curl makes curl buffed the whole standard input, so for big files you have to either you have to do -H "Transfer-Encoding: chunked" or use -T file 2) dd block devices is very fragile, its better if you just tar --exclude/dev ... (https://help.ubuntu.com/community/BackupYourSystem/TAR https://help.ubuntu.com/community/BackupYourSystem/TAR - Alternate backup)