5 ms·

While everyone is of course free to start his/hers project there is a whole bunch of projects which basically does the same thing like: deduplication, client ca

While everyone is of course free to start his/hers project there is a whole bunch of projects which basically does the same thing like: deduplication, client can append backups only, server side client backup rotation / pruning, etc. For example restic ( https://restic.net/ https://restic.net/ ) does everything the author describes in the ideal feature set ( https://packnback.github.io/blog/work_begins/ https://packnback.github.io/blog/work_begins/ ) except for asymmetric encrpytion. Though there is an issue open for that and iirc they are working on it. Contributing to that project would probably be easier? Another existing very interesting project to contribute to is rdedup ( https://github.com/dpc/rdedup https://github.com/dpc/rdedup ) which is written in Rust. There is a whole list over here of existing backup solutions: https://github.com/restic/others https://github.com/restic/others Judging from the comparisons made in the introduction blog post by the author I think he / she missed the existence of a bunch of existing solutions. EDIT: This was supposed to be a reply to the comment in which marmaduke asked "why start a new project?"

blattimwind 8y ago
Asymmetric crypto for backups feels overrated to me. The "write-only mode" argument makes sense, if the backup target holds data from other nodes, but that doesn't require asymmetric crypto. The "shouldn't be able to read own backups by default because what if it's hacked" argument doesn't seem to hold much water to me, because the most valuable secrets will generally be those in active use on the machine.
- justin_oaks 8y ago
  Asymmetric crypto allows you to use the same public key across all your machines without the concern that one compromised machine will leak the key for all of them.
  - blattimwind 8y ago
    Like I said, access control does not require asymmetric crypto.
    - justin_oaks 8y ago
      Layered security. If there's a vulnerability in your access control, the encryption prevents data from being accessed (decrypted). If there's a vulnerability in the encryption, the access control prevents the data from being read.
      - blattimwind 8y ago
        I'm not saying it shouldn't be done, but I've seen a number of projects which have been started (and afaik abandoned as soon as the author saw that backup software isn't quite so simple to develop) specifically because asymmetric crypto was seen as a missing flagship feature. What I'm saying here is, if you start a new backup software project today, there are more important things to consider and failings of existing software to address than not supporting asymmetric crypto. Do the encryption right. Do proper access controls. Get concurrency right. Get deduplication right. Get your archive/snapshot format right. Get filesystem interaction and testing right. Maybe you think existing software does all of that already; it doesn't. (Though I am actually convinced that you can't design a piece fulfilling these requirements without having a lot of prior experience developing backup software — conversely that's the reason why most backup software is so half-baked: they're all first designs, by the time a developer has the necessary experience to do a better second design, they're fed up with backup software and move on).
- FooBarWidget 8y ago
  > The "shouldn't be able to read own backups by default because what if it's hacked" argument doesn't seem to hold much water to me, because the most valuable secrets will generally be those in active use on the machine. Leaking data is only one consequence of getting hacked. Another consequence is corrupting data. Even if you can't prevent the former, there are still times when you want to recover from the latter. If the attacker can overwrite old backups then you are toast.
  - blattimwind 8y ago
    "read" vs "write"
  - ericpauley 8y ago
    Unfortunately asymmetric encryption on the data itself only prevents leaking. It doesn't ensure integrity or prevent deletion. For that you need some system that explicitly enforces those permissions.
FooBarWidget 8y ago
> client can append backups only You mention Restic but I don't see how it enforces append-only security. According to its S3 documentation[1], Restic requires PutObject, DeleteObject and ListBucket permissions. [1] https://restic.readthedocs.io/en/stable/080_examples.html#setting-up-restic-with-amazon-s3 https://restic.readthedocs.io/en/stable/080_examples.html#se...
- jsiepkes 8y ago
  You can do it with the restic server ( https://github.com/restic/rest-server https://github.com/restic/rest-server ). Don't know if it is implemented in the S3 backend.
- regecks 8y ago
  You can also just enable versioning on the bucket which prevents PutObject from doing any permanent damage. If restic needs DeleteObject, you can also configure IAM to not allow deleting of specific versions, which means it can only tombstone objects.

blattimwind 8y ago

Asymmetric crypto for backups feels overrated to me. The "write-only mode" argument makes sense, if the backup target holds data from other nodes, but that doesn't require asymmetric crypto. The "shouldn't be able to read own backups by default because what if it's hacked" argument doesn't seem to hold much water to me, because the most valuable secrets will generally be those in active use on the machine.

justin_oaks 8y ago
Asymmetric crypto allows you to use the same public key across all your machines without the concern that one compromised machine will leak the key for all of them.
- blattimwind 8y ago
  Like I said, access control does not require asymmetric crypto.
  - justin_oaks 8y ago
    Layered security. If there's a vulnerability in your access control, the encryption prevents data from being accessed (decrypted). If there's a vulnerability in the encryption, the access control prevents the data from being read.
    - blattimwind 8y ago
      I'm not saying it shouldn't be done, but I've seen a number of projects which have been started (and afaik abandoned as soon as the author saw that backup software isn't quite so simple to develop) specifically because asymmetric crypto was seen as a missing flagship feature. What I'm saying here is, if you start a new backup software project today, there are more important things to consider and failings of existing software to address than not supporting asymmetric crypto. Do the encryption right. Do proper access controls. Get concurrency right. Get deduplication right. Get your archive/snapshot format right. Get filesystem interaction and testing right. Maybe you think existing software does all of that already; it doesn't. (Though I am actually convinced that you can't design a piece fulfilling these requirements without having a lot of prior experience developing backup software — conversely that's the reason why most backup software is so half-baked: they're all first designs, by the time a developer has the necessary experience to do a better second design, they're fed up with backup software and move on).
FooBarWidget 8y ago
> The "shouldn't be able to read own backups by default because what if it's hacked" argument doesn't seem to hold much water to me, because the most valuable secrets will generally be those in active use on the machine. Leaking data is only one consequence of getting hacked. Another consequence is corrupting data. Even if you can't prevent the former, there are still times when you want to recover from the latter. If the attacker can overwrite old backups then you are toast.
- blattimwind 8y ago
  "read" vs "write"
- ericpauley 8y ago
  Unfortunately asymmetric encryption on the data itself only prevents leaking. It doesn't ensure integrity or prevent deletion. For that you need some system that explicitly enforces those permissions.

FooBarWidget 8y ago

> client can append backups only You mention Restic but I don't see how it enforces append-only security. According to its S3 documentation[1], Restic requires PutObject, DeleteObject and ListBucket permissions. [1] https://restic.readthedocs.io/en/stable/080_examples.html#setting-up-restic-with-amazon-s3 https://restic.readthedocs.io/en/stable/080_examples.html#se...

jsiepkes 8y ago
You can do it with the restic server ( https://github.com/restic/rest-server https://github.com/restic/rest-server ). Don't know if it is implemented in the S3 backend.
regecks 8y ago
You can also just enable versioning on the bucket which prevents PutObject from doing any permanent damage. If restic needs DeleteObject, you can also configure IAM to not allow deleting of specific versions, which means it can only tombstone objects.