4 ms·

We did this for a while in our house, but with Bittorrent/Resilio Sync on later Pi models, and it was a pretty miserable experience. I/O is terrible on most mod

We did this for a while in our house, but with Bittorrent/Resilio Sync on later Pi models, and it was a pretty miserable experience. I/O is terrible on most models (which is somewhat important for storage). IIRC there is no acceleration of crypto primitives, so AES encryption used by Resilio Sync and Syncthing is slow. Finally, we found that even with a proper power supply, the Pi does not really provide the stability you'd want from a storage node. We ended up replacing the Pi(s) by an Intel NUC, which was just over 100 Euro in 2018 without memory and storage. For relatively low cost, you get SATA speeds (or NVMe if you are willing to spend a bit more) and an AES-NI capable CPU. Our current NUC has been humming along for 4 years. Ps. Syncthing is not backup, even with a read-only node. You are just one Syncthing bug or sysadmin failure away from erasing your data. Having the files on btrts on a single SSD only makes it worse. Just get some redundant block storage somewhere that supports object lock (to prevent accidental deletion or ransomware encrypting all your files) and do incremental backups with something like Arq or Restic. There are some good services where 100GB storage costs $1 monthly.

hddherman 4y ago
This isn't the only backup I have of the data, it's just another copy of the data. BTRFS snapshots should be enough protection against the ransomware use case as well, assuming I discover it fast enough. And if I don't, I still have offline backups.
nsomaru 4y ago
Can you share an example of what might be a good service. Re ransomware, how do you prevent it from deleting remote backups?
- croon 4y ago
  Write-Once-Read-Many and retention times, for example.
  - bluGill 4y ago
    There can still be bugs such that you can write a second time much of the time the write once is just a software driver that won't write a second time so if you can write new firmware of your own design you can erase. Even if the media is physically not erasable, you can still write all ones over the top of what is there, corrupting data which is the same result. Of course finding the bugs in firmware and writing a custom replacement is not trivial. It is conceptually possible though. The only safe answer is a device in a vault without power. Of course once you retrieve it from the vault you risk whatever erased your data in the first place returning to get this too. Good luck, you get to decide how paranoid you want go be.
    - microtonal 4y ago
      We are talking about object lock as supported by something like Amazon S3 or Backblaze B2. If you find a way to override them, you probably want to apply for a bug bounty, since that'd be a huge security issue. It works really nicely, I can't even remove my own data with my own credentials. It's really a nice additional security layer.
      - bluGill 4y ago
        > If you find a way to override them, you probably want to apply for a bug bounty, since that'd be a huge security issue. Big security issue, and honest people will just apply for a bug bounty or otherwise responsible disclosure. Most honest people are not actively looking for such bugs though, so evil people are more likely to find them. An evil person is can make even more money from a successful ransom. (there are also honest people looking, but many wrote the code so they may be too close to the problem to see it)
- microtonal 4y ago
  E.g. Amazon S3 and Backblaze B2 support this. Re ransomware, how do you prevent it from deleting remote backups? It can't, locks on objects cannot be removed in compliance mode. They can only expire. https://www.backblaze.com/b2/docs/file_lock.html https://www.backblaze.com/b2/docs/file_lock.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/object...
sirtaj 4y ago
Hetzner's storage boxes provide native borgbackup support, I've found it very useful as an offsite backup store.
amelius 4y ago
With a cloud solution, how are you not one bug or sysadmin failure away from data-loss or leaks?
- deafpolygon 4y ago
  Depends on your cloud solution, but OneDrive has version history so you can just roll it back.
  - amelius 4y ago
    Btrfs has snapshots too. That doesn't rule out bugs in it. Making something someone else's problem doesn't magically make the problem go away.
    - microtonal 4y ago
      Are we really having a debate on whether cloud storage with proper redundancy, reliable hardware with ECC memory and dedicated sysadmins is as vulnerable to mishaps as a consumer-grade SSD taped to a Raspberry Pi? I hope we can at least agree that it is 3-4 orders of magnitude more likely that an SSD with btrfs and no RAID attached to a Pi to lose data within a given time period than, say S3?
  - ipython 4y ago
    Unless OneDrive silently corrupts your files upon upload: https://github.com/OneDrive/onedrive-api-docs/issues/1577 https://github.com/OneDrive/onedrive-api-docs/issues/1577
- microtonal 4y ago
  You avoid backup software bugs by using object lock. Even if your backup software messes up, it cannot remove older backups. On the cloud-side, sure bad things can happen, but a data loss event through a sysadmin or hardware failure is far less likely to happen on e.g. Amazon S3 or Backblaze B2 than with a home Raspberry Pi with a consumer-grade SSD taped on top of it. And you are probably backing up to at least two different destinations (at different companies) if you care about your data.

hddherman 4y ago

This isn't the only backup I have of the data, it's just another copy of the data. BTRFS snapshots should be enough protection against the ransomware use case as well, assuming I discover it fast enough. And if I don't, I still have offline backups.

nsomaru 4y ago

Can you share an example of what might be a good service. Re ransomware, how do you prevent it from deleting remote backups?

croon 4y ago
Write-Once-Read-Many and retention times, for example.
- bluGill 4y ago
  There can still be bugs such that you can write a second time much of the time the write once is just a software driver that won't write a second time so if you can write new firmware of your own design you can erase. Even if the media is physically not erasable, you can still write all ones over the top of what is there, corrupting data which is the same result. Of course finding the bugs in firmware and writing a custom replacement is not trivial. It is conceptually possible though. The only safe answer is a device in a vault without power. Of course once you retrieve it from the vault you risk whatever erased your data in the first place returning to get this too. Good luck, you get to decide how paranoid you want go be.
  - microtonal 4y ago
    We are talking about object lock as supported by something like Amazon S3 or Backblaze B2. If you find a way to override them, you probably want to apply for a bug bounty, since that'd be a huge security issue. It works really nicely, I can't even remove my own data with my own credentials. It's really a nice additional security layer.
    - bluGill 4y ago
      > If you find a way to override them, you probably want to apply for a bug bounty, since that'd be a huge security issue. Big security issue, and honest people will just apply for a bug bounty or otherwise responsible disclosure. Most honest people are not actively looking for such bugs though, so evil people are more likely to find them. An evil person is can make even more money from a successful ransom. (there are also honest people looking, but many wrote the code so they may be too close to the problem to see it)
microtonal 4y ago
E.g. Amazon S3 and Backblaze B2 support this. Re ransomware, how do you prevent it from deleting remote backups? It can't, locks on objects cannot be removed in compliance mode. They can only expire. https://www.backblaze.com/b2/docs/file_lock.html https://www.backblaze.com/b2/docs/file_lock.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/object...

sirtaj 4y ago

Hetzner's storage boxes provide native borgbackup support, I've found it very useful as an offsite backup store.

amelius 4y ago

With a cloud solution, how are you not one bug or sysadmin failure away from data-loss or leaks?

deafpolygon 4y ago
Depends on your cloud solution, but OneDrive has version history so you can just roll it back.
- amelius 4y ago
  Btrfs has snapshots too. That doesn't rule out bugs in it. Making something someone else's problem doesn't magically make the problem go away.
  - microtonal 4y ago
    Are we really having a debate on whether cloud storage with proper redundancy, reliable hardware with ECC memory and dedicated sysadmins is as vulnerable to mishaps as a consumer-grade SSD taped to a Raspberry Pi? I hope we can at least agree that it is 3-4 orders of magnitude more likely that an SSD with btrfs and no RAID attached to a Pi to lose data within a given time period than, say S3?
- ipython 4y ago
  Unless OneDrive silently corrupts your files upon upload: https://github.com/OneDrive/onedrive-api-docs/issues/1577 https://github.com/OneDrive/onedrive-api-docs/issues/1577
microtonal 4y ago
You avoid backup software bugs by using object lock. Even if your backup software messes up, it cannot remove older backups. On the cloud-side, sure bad things can happen, but a data loss event through a sysadmin or hardware failure is far less likely to happen on e.g. Amazon S3 or Backblaze B2 than with a home Raspberry Pi with a consumer-grade SSD taped on top of it. And you are probably backing up to at least two different destinations (at different companies) if you care about your data.