5 ms·

In my book, backup tools don't count unless it has been used widely for a while without major issues repeatedly being reported. Kopia is too new for that state

In my book, backup tools don't count unless it has been used widely for a while without major issues repeatedly being reported. Kopia is too new for that state.

Silhouette 4y ago
That's a reasonable point but the solution - as with all things backup - is diversification. Otherwise if everyone followed your logic then no new backup software or storage service could ever become established no matter how good it actually was. Given that all of the options being discussed here look technically better and in some cases more than an order of magnitude cheaper than other popular backup services and software discussed on HN in the past you could afford to run full redundant backups with multiple combinations of software and backing storage and still have more options for a much lower price than a few years ago.
eli 4y ago
If you regularly actually test it, it should be fine. (Using a popular solution without testing it is riskier)
g_p 4y ago
Kopia is pretty new, but looking through the documentation I felt it had some pretty good features that suggest it will be a good option in the future. A few highlights (for me at least): - They have implemented support for object locking and versioning to prevent a compromised set of S3 keys being used to erase backups. There seems to be a pull request outstanding to dynamically shift forward the compliance hold each time an object file is "deemed still used", but very few s3 bucket based backup tools seem to really think about resisting deletion and documenting best practice. I like to use minimum privilege and deny DeleteObject permission, but using versioning and object locking policies seems to be an interesting potential solution. - Inclusion of (optional) Reed Solomon error correction codes in backup objects. - a CLI verification option that will test a given % of your backup objects (at random) every time it's run, to give you opportunistic random verification to try find issues. - reasonably well documented CLI options for recovering data were anything to go wrong - as the developers mention, there are a wide range of ways to recover data. Interestingly, they appear to have actually implemented these, rather than leave them as theoretical. For example, CLI commands to interact with indexes, manifests, blobs, snapshots, etc. You don't need them normally, but having these available lets you look around a simple backup and understand the format and gain confidence in it, and that the tools work. The only feature it seems to me to be lacking is asymmetric encryption of backups, so you can keep the key needed for recovery away from the host you are backing up.
- hashhar 4y ago
  Some more context. > - a CLI verification option that will test a given % of your backup objects (at random) every time it's run, to give you opportunistic random verification to try find issues. FWIW Restic also supports this - see restic check --read-data-subset (https://restic.readthedocs.io/en/stable/045_working_with_repos.html#checking-integrity-and-consistency https://restic.readthedocs.io/en/stable/045_working_with_rep...) Also Restic doesn't have any problems with buckets with deletions disallowed as long as you allow them for just the `locks` directory. A policy I used for one of my targets looked like: { "Statement": [ { "Sid": "AllowAdditions", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": "arn:aws:s3:::BUCKETNAME" }, { "Sid": "AllowDeleteLocks", "Effect": "Allow", "Action": "s3:DeleteObject", "Resource": "arn:aws:s3:::BUCKETNAME/locks/*" } ] } And it even works with buckets with object locking enabled since >= 0.13.
  - aborsy 4y ago
    I guess if you comment out the delete disallowing the action on lock, you can prune the repository once in a while without reindexing? Then pruning could be done once every few months. The egress fees could be high though.

In my book, backup tools don't count unless it has been used widely for a while without major issues repeatedly being reported. Kopia is too new for that state.

Silhouette 4y ago

That's a reasonable point but the solution - as with all things backup - is diversification. Otherwise if everyone followed your logic then no new backup software or storage service could ever become established no matter how good it actually was. Given that all of the options being discussed here look technically better and in some cases more than an order of magnitude cheaper than other popular backup services and software discussed on HN in the past you could afford to run full redundant backups with multiple combinations of software and backing storage and still have more options for a much lower price than a few years ago.

eli 4y ago

If you regularly actually test it, it should be fine. (Using a popular solution without testing it is riskier)

g_p 4y ago

Kopia is pretty new, but looking through the documentation I felt it had some pretty good features that suggest it will be a good option in the future. A few highlights (for me at least): - They have implemented support for object locking and versioning to prevent a compromised set of S3 keys being used to erase backups. There seems to be a pull request outstanding to dynamically shift forward the compliance hold each time an object file is "deemed still used", but very few s3 bucket based backup tools seem to really think about resisting deletion and documenting best practice. I like to use minimum privilege and deny DeleteObject permission, but using versioning and object locking policies seems to be an interesting potential solution. - Inclusion of (optional) Reed Solomon error correction codes in backup objects. - a CLI verification option that will test a given % of your backup objects (at random) every time it's run, to give you opportunistic random verification to try find issues. - reasonably well documented CLI options for recovering data were anything to go wrong - as the developers mention, there are a wide range of ways to recover data. Interestingly, they appear to have actually implemented these, rather than leave them as theoretical. For example, CLI commands to interact with indexes, manifests, blobs, snapshots, etc. You don't need them normally, but having these available lets you look around a simple backup and understand the format and gain confidence in it, and that the tools work. The only feature it seems to me to be lacking is asymmetric encryption of backups, so you can keep the key needed for recovery away from the host you are backing up.

hashhar 4y ago
Some more context. > - a CLI verification option that will test a given % of your backup objects (at random) every time it's run, to give you opportunistic random verification to try find issues. FWIW Restic also supports this - see restic check --read-data-subset (https://restic.readthedocs.io/en/stable/045_working_with_repos.html#checking-integrity-and-consistency https://restic.readthedocs.io/en/stable/045_working_with_rep...) Also Restic doesn't have any problems with buckets with deletions disallowed as long as you allow them for just the `locks` directory. A policy I used for one of my targets looked like: { "Statement": [ { "Sid": "AllowAdditions", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": "arn:aws:s3:::BUCKETNAME" }, { "Sid": "AllowDeleteLocks", "Effect": "Allow", "Action": "s3:DeleteObject", "Resource": "arn:aws:s3:::BUCKETNAME/locks/*" } ] } And it even works with buckets with object locking enabled since >= 0.13.
- aborsy 4y ago
  I guess if you comment out the delete disallowing the action on lock, you can prune the repository once in a while without reindexing? Then pruning could be done once every few months. The egress fees could be high though.