3 ms·

My current backup solution for my personal server is on a daily basis run sudo -u postgres pg_dumpall > /var/base/backups/db/postgres/backup.sql and then ship t

by mey 10y ago

My current backup solution for my personal server is on a daily basis run sudo -u postgres pg_dumpall > /var/base/backups/db/postgres/backup.sql and then ship that plus many other things off to tarsnap. This is a naive approach because the data set is small, but it could be tuned and improved to be delta based at some point. The joy is Tarsnap is managing the history of the data for me on that file and retains it as long as I wish to pay for. I have a separate script that manages cleaning up of backups older than a certain age, etc.

koolba 10y ago
Within the realm of pg_dump completing in a reasonable amount of time, this is a fine strategy. Bonus points if the server that runs pg_dump and uploads to tarsnap has write-only access (to tarsnap), uses unique unguessable prefixes for the backups, and is distinct from the server that manages cleaning out old backups. Course that's probably way overkill for a personal server :D
- mey 10y ago
  Once I have a private cloud running in my house I'll head in that direction, but need to get around to seriously upgrading my networking environment and building out a new NAS. :) Edit: Not sure why'd you want unique unguessable prefixes, the data is accessed with the key. I actually encode timestamp information into my backup names for easy display.
  - koolba 10y ago
    > Edit: Not sure why'd you want unique unguessable prefixes, the data is accessed with the key. I actually encode timestamp information into my backup names for easy display. The random prefix is to prevent overwriting an existing backup. See the sibling comment from cperciva and my reply to it.
- gleb 10y ago
  If you had ever set this up - a separate server to rotate/clean old tarsnap backups - I'd be curious how you deal with tarsnap local cache getting out of date. tarsnap fsck on every run?
- cperciva 10y ago
  uses unique unguessable prefixes for the backups Why? If you're worried about someone overwriting old backups, don't: You can't overwrite or modify archives in tarsnap. Once created they can only be read (if you have the read keys) or deleted (if you have the delete keys).
  - koolba 10y ago
    > Why? If you're worried about someone overwriting old backups, don't: You can't overwrite or modify archives in tarsnap. Once created they can only be read (if you have the read keys) or deleted (if you have the delete keys). Nice. Some reason I thought a write key would be able to overwrite as well. If that's not possible then yes it's not necessary to have random prefixes. Fyi, most of my tarsnap usage is in "set it and forget" mode so been a while since I perused the docs. The random prefixes would apply if you're using something that doesn't provide overwrite protection like pushing blobs directly to S3.

by mey 10y ago

koolba 10y ago

Within the realm of pg_dump completing in a reasonable amount of time, this is a fine strategy. Bonus points if the server that runs pg_dump and uploads to tarsnap has write-only access (to tarsnap), uses unique unguessable prefixes for the backups, and is distinct from the server that manages cleaning out old backups. Course that's probably way overkill for a personal server :D

mey 10y ago
Once I have a private cloud running in my house I'll head in that direction, but need to get around to seriously upgrading my networking environment and building out a new NAS. :) Edit: Not sure why'd you want unique unguessable prefixes, the data is accessed with the key. I actually encode timestamp information into my backup names for easy display.
- koolba 10y ago
  > Edit: Not sure why'd you want unique unguessable prefixes, the data is accessed with the key. I actually encode timestamp information into my backup names for easy display. The random prefix is to prevent overwriting an existing backup. See the sibling comment from cperciva and my reply to it.
gleb 10y ago
If you had ever set this up - a separate server to rotate/clean old tarsnap backups - I'd be curious how you deal with tarsnap local cache getting out of date. tarsnap fsck on every run?
cperciva 10y ago
uses unique unguessable prefixes for the backups Why? If you're worried about someone overwriting old backups, don't: You can't overwrite or modify archives in tarsnap. Once created they can only be read (if you have the read keys) or deleted (if you have the delete keys).
- koolba 10y ago
  > Why? If you're worried about someone overwriting old backups, don't: You can't overwrite or modify archives in tarsnap. Once created they can only be read (if you have the read keys) or deleted (if you have the delete keys). Nice. Some reason I thought a write key would be able to overwrite as well. If that's not possible then yes it's not necessary to have random prefixes. Fyi, most of my tarsnap usage is in "set it and forget" mode so been a while since I perused the docs. The random prefixes would apply if you're using something that doesn't provide overwrite protection like pushing blobs directly to S3.