7 ms·

I use rclone's B2 driver (https://rclone.org/b2/ https://rclone.org/b2/) as an rsync-style backup solution of about 1TB worth of my pictures and other media. Al

I use rclone's B2 driver (https://rclone.org/b2/ https://rclone.org/b2/) as an rsync-style backup solution of about 1TB worth of my pictures and other media. Also, it's encrypted with my own local key using rclone's crypt module (https://rclone.org/crypt/ https://rclone.org/crypt/). rclone supports multithreaded upload, and even has experimental support for FUSE mounting. However, the sync command gets you Dropbox-like behavior and can be cronned: https://rclone.org/commands/rclone_sync/ https://rclone.org/commands/rclone_sync/ I really like the price of B2, I hope it stays low :-)

gordo4 8y ago
rclone is vulnerable to data exfiltration attacks https://www.danieldent.com/blog/restless-vulnerability-non-browser-cross-domain-http-request-attacks/ https://www.danieldent.com/blog/restless-vulnerability-non-b...
- Operyl 8y ago
  I quickly skimmed, but this entire attack is assuming that the attacker has successfully MITMed the API. At that point everything is already nuked, so of course you can fabricate any number of attacks. Did I miss something important?
- howeyc 8y ago
  Am I reading this right? Google/B2/... might send your data to another URL you didn't expect. Not sure why that matters, or why it's an attack. Since they have your data anyway, as that's the whole point of the service, to store your data on their hard drives. Why go through the trouble of sending it elsewhere? To play games with your data for giggles?
  - namibj 8y ago
    No, the API can tell your software to send some private LAN files, e.g. some IP-filtered secret NFS store, to an URL of it's choosing (so to itself, or your competitor). This is bad, as long as you don't heavily jail and firewall the software to prevent it from ever accessing anything it shouldn't (need to).
bad_user 8y ago
I'm also backing up my photo archive on B2 via rclone and it works great, it costs like $1 for 200 GB which is awesome. But I'm not encrypting the files. Encryption raises the barrier for both third-parties and family. In case something happens to me, I want the technical barrier to be low enough for my family to discover the backup. Another reason is that in my experience encrypted data is more sensitive to bit rot and bugs than unencrypted data. I'm backing up important work stuff with Arq Backup for example and I've had my archive corrupted once. Not sure if it was the software's fault or the storage. My rule of thumb is ... if the data should be discovered by my family in case I'm not around, then I won't encrypt it. Photos are not worth encrypting anyway, since a lot of them end up being shared on Facebook, Flickr, Instagram, etc, as photos are meant to be shared, at least with your family. That said I still expect Backblaze or Dropbox to keep my data private. Not secret, but private and there is a difference.
- eddieroger 8y ago
  I've thought about that a fair a lot recently myself, and came to a different conclusion, but like yours as a counterpoint. What I've been considering was going analog and writing down how to get to the few places I've kept my secrets and employing something like a Yubikey to access an admin account on my computer and paper instructions in an envelope in the family safety deposit box. It's not a complex as a two-person thing I read about once, but seems like a nice compromise where I can still feel like I'm protected, but the nuclear keys exist if someone ever needed them. Where do you draw the line around data that needs to be discovered? I'm thinking about instructions to access things like bank accounts or such that they may or may not already have access to, where I'd want them encrypted but accessible. Not that I've got secret Cayman accounts or anything, but financials are usually things i want heavily encrypted, but do want family access to in case of the worst.
  - sdhgaiojfsa 8y ago
    I normally think of financials as things that don't need to be heavily encrypted, because we have laws limiting liability in case financial information is stolen and misused. What makes you feel it needs encryption?
james_in_the_uk 8y ago
In the spirit of sharing details of open source tools that work well with B2, I am using restic and finding it to work well.
- jpeeler 8y ago
  Apparently restic now supports using rclone as a backend: https://restic.net/blog/2018-04-01/rclone-backend https://restic.net/blog/2018-04-01/rclone-backend
- AgentME 8y ago
  +1 for restic. I switched to restic(+B2) from duply+duplicity(+S3) (another backup tool supporting dumb remote storage, encryption, incremental backups, and snapshots) and life is so much better. Duplicity needs the duply front-end for too many basics, it regularly needs full backups to be made and stored, it's not built at all for random access (it takes eons in order to list or fetch a specific file from a specific incremental backup), and it needs a barely documented command line switch in order to not bug out with S3 if you have too many files (why is that option not default? Its default S3 configuration has a max limit on file size. Duplicity splits the archives containing your files, but it has one un-split archive listing filenames or some kind of metadata that it doesn't support splitting, so if that file gets too big, then your backups just start failing). Restic is nice.
kachurovskiy 8y ago
Why not use $60 you're paying per year for 1Tb of online storage to buy a 2Tb hard drive and use it for the backup?
- chillydawg 8y ago
  Hard drives fail and you need them physically to access the data. Online backups allow you access everywhere and are generally much more reliable than a single, local disk.
- chowell 8y ago
  Presumably so that the backup is offsite.
- billyhoffman 8y ago
  Online/offsite backup is a different use case. They are paying $60/year so that, if their house burns down, gets flooded, disk gets fried by lightning, they still have their family pictures. Local backup is cheap and fast, and you should do it too. But it doesn't provide geographic redundancy.
  - kachurovskiy 8y ago
    Another $60 buys a water and fireproof safe for the hard drive. I assume it also helps against lightning :-) I honestly think remote backup is an overkill for personal needs and new risks you get by placing your valuable data on someone else's hard drive are not always internalized.
    - nathankunicki 8y ago
      By putting your drive in a safe, you now have to unlock it, take it out, plug it in, sync, unplug it, and lock it away. Which is a hassle if you want to stick to regular (hourly? daily?) backups. It is proven that if you introduce friction into process, over time that process will be followed less.
    - scarface74 8y ago
artpar 8y ago
rclone is excellent piece of code and has solved a large part of issue for my own project. I maintain a slightly modified fork of rclone [1] and integrated that in daptin [2] as a server side piece. So I can seamlessly work with any cloud storage for assets/uploads thru daptin. I wrote up a walk-thru some time back. The changes basically include replacing all the "fatal logs" with "error logs". I keep merging the upstream back regularly. [1] https://github.com/artpar/rclone https://github.com/artpar/rclone [2] https://github.com/daptin/daptin https://github.com/daptin/daptin [3] https://medium.com/@012parth/daptin-walk-through-oauth2-google-drive-subsites-and-grapejs-a6de27d9658a https://medium.com/@012parth/daptin-walk-through-oauth2-goog...
MacroChip 8y ago
Cool setup. I'm not a Backblaze customer, so I'm curious. How is your setup better than using their client for personal backup? The webpage mentions threading and encryption.
- tomfanning 8y ago
  Backblaze's backup product doesn't support backing up a NAS. It is for backing up a single Windows or Mac computer, and priced as such. They state this policy is to avoid abuse. Fair enough. Backblaze's object storage product, B2, is priced per GB-month, so you pay for what you use. Fair enough. Because it is charged this way, it is open for whatever creative use developers can come up with. I use B2 because I'm locked out of using Backblaze Online Backup - and that's fine with me, because it's the right product for the job.
bollockitis 8y ago
That's interesting. I have been considering a similar setup. Are you running this from Linux?
- ereyes01 8y ago
  Yes, I do all this on Linux, but they support other environments too (most of the environments Go can build for): https://rclone.org/downloads/ https://rclone.org/downloads/
asenna 8y ago
I'll definitely give this a try. I'm currently backing up about 1.7TB of pictures to B2 from my Qnap NAS. Qnap has a backup app called Hybrid Backup Sync. The problem is, while doing the one-way upload sync, the Qnap app downloaded a lot of data as well. I got confused why I was showing a lot of 'b2_download_file_by_name' API calls on the Backblaze reports page (600 GB upload resulted in 700 GB download calls). I've also opened a thread on the forum - https://forum.qnap.com/viewtopic.php?p=673557 https://forum.qnap.com/viewtopic.php?p=673557 Contacted Qnap support and they said a little bit of download is normal but this looks abnormal. Logs are all fine on the Qnap so they suggested I contact Backblaze. I wonder if anyone else has faced this.

gordo4 8y ago

rclone is vulnerable to data exfiltration attacks https://www.danieldent.com/blog/restless-vulnerability-non-browser-cross-domain-http-request-attacks/ https://www.danieldent.com/blog/restless-vulnerability-non-b...

Operyl 8y ago
I quickly skimmed, but this entire attack is assuming that the attacker has successfully MITMed the API. At that point everything is already nuked, so of course you can fabricate any number of attacks. Did I miss something important?
howeyc 8y ago
Am I reading this right? Google/B2/... might send your data to another URL you didn't expect. Not sure why that matters, or why it's an attack. Since they have your data anyway, as that's the whole point of the service, to store your data on their hard drives. Why go through the trouble of sending it elsewhere? To play games with your data for giggles?
- namibj 8y ago
  No, the API can tell your software to send some private LAN files, e.g. some IP-filtered secret NFS store, to an URL of it's choosing (so to itself, or your competitor). This is bad, as long as you don't heavily jail and firewall the software to prevent it from ever accessing anything it shouldn't (need to).

bad_user 8y ago

I'm also backing up my photo archive on B2 via rclone and it works great, it costs like $1 for 200 GB which is awesome. But I'm not encrypting the files. Encryption raises the barrier for both third-parties and family. In case something happens to me, I want the technical barrier to be low enough for my family to discover the backup. Another reason is that in my experience encrypted data is more sensitive to bit rot and bugs than unencrypted data. I'm backing up important work stuff with Arq Backup for example and I've had my archive corrupted once. Not sure if it was the software's fault or the storage. My rule of thumb is ... if the data should be discovered by my family in case I'm not around, then I won't encrypt it. Photos are not worth encrypting anyway, since a lot of them end up being shared on Facebook, Flickr, Instagram, etc, as photos are meant to be shared, at least with your family. That said I still expect Backblaze or Dropbox to keep my data private. Not secret, but private and there is a difference.

eddieroger 8y ago
I've thought about that a fair a lot recently myself, and came to a different conclusion, but like yours as a counterpoint. What I've been considering was going analog and writing down how to get to the few places I've kept my secrets and employing something like a Yubikey to access an admin account on my computer and paper instructions in an envelope in the family safety deposit box. It's not a complex as a two-person thing I read about once, but seems like a nice compromise where I can still feel like I'm protected, but the nuclear keys exist if someone ever needed them. Where do you draw the line around data that needs to be discovered? I'm thinking about instructions to access things like bank accounts or such that they may or may not already have access to, where I'd want them encrypted but accessible. Not that I've got secret Cayman accounts or anything, but financials are usually things i want heavily encrypted, but do want family access to in case of the worst.
- sdhgaiojfsa 8y ago
  I normally think of financials as things that don't need to be heavily encrypted, because we have laws limiting liability in case financial information is stolen and misused. What makes you feel it needs encryption?

james_in_the_uk 8y ago

In the spirit of sharing details of open source tools that work well with B2, I am using restic and finding it to work well.

jpeeler 8y ago
Apparently restic now supports using rclone as a backend: https://restic.net/blog/2018-04-01/rclone-backend https://restic.net/blog/2018-04-01/rclone-backend
AgentME 8y ago
+1 for restic. I switched to restic(+B2) from duply+duplicity(+S3) (another backup tool supporting dumb remote storage, encryption, incremental backups, and snapshots) and life is so much better. Duplicity needs the duply front-end for too many basics, it regularly needs full backups to be made and stored, it's not built at all for random access (it takes eons in order to list or fetch a specific file from a specific incremental backup), and it needs a barely documented command line switch in order to not bug out with S3 if you have too many files (why is that option not default? Its default S3 configuration has a max limit on file size. Duplicity splits the archives containing your files, but it has one un-split archive listing filenames or some kind of metadata that it doesn't support splitting, so if that file gets too big, then your backups just start failing). Restic is nice.

kachurovskiy 8y ago

Why not use $60 you're paying per year for 1Tb of online storage to buy a 2Tb hard drive and use it for the backup?

chillydawg 8y ago
Hard drives fail and you need them physically to access the data. Online backups allow you access everywhere and are generally much more reliable than a single, local disk.
chowell 8y ago
Presumably so that the backup is offsite.
billyhoffman 8y ago
Online/offsite backup is a different use case. They are paying $60/year so that, if their house burns down, gets flooded, disk gets fried by lightning, they still have their family pictures. Local backup is cheap and fast, and you should do it too. But it doesn't provide geographic redundancy.
- kachurovskiy 8y ago
  Another $60 buys a water and fireproof safe for the hard drive. I assume it also helps against lightning :-) I honestly think remote backup is an overkill for personal needs and new risks you get by placing your valuable data on someone else's hard drive are not always internalized.
  - nathankunicki 8y ago
    By putting your drive in a safe, you now have to unlock it, take it out, plug it in, sync, unplug it, and lock it away. Which is a hassle if you want to stick to regular (hourly? daily?) backups. It is proven that if you introduce friction into process, over time that process will be followed less.
  - scarface74 8y ago

artpar 8y ago

rclone is excellent piece of code and has solved a large part of issue for my own project. I maintain a slightly modified fork of rclone [1] and integrated that in daptin [2] as a server side piece. So I can seamlessly work with any cloud storage for assets/uploads thru daptin. I wrote up a walk-thru some time back. The changes basically include replacing all the "fatal logs" with "error logs". I keep merging the upstream back regularly. [1] https://github.com/artpar/rclone https://github.com/artpar/rclone [2] https://github.com/daptin/daptin https://github.com/daptin/daptin [3] https://medium.com/@012parth/daptin-walk-through-oauth2-google-drive-subsites-and-grapejs-a6de27d9658a https://medium.com/@012parth/daptin-walk-through-oauth2-goog...

MacroChip 8y ago

Cool setup. I'm not a Backblaze customer, so I'm curious. How is your setup better than using their client for personal backup? The webpage mentions threading and encryption.

tomfanning 8y ago
Backblaze's backup product doesn't support backing up a NAS. It is for backing up a single Windows or Mac computer, and priced as such. They state this policy is to avoid abuse. Fair enough. Backblaze's object storage product, B2, is priced per GB-month, so you pay for what you use. Fair enough. Because it is charged this way, it is open for whatever creative use developers can come up with. I use B2 because I'm locked out of using Backblaze Online Backup - and that's fine with me, because it's the right product for the job.

bollockitis 8y ago

That's interesting. I have been considering a similar setup. Are you running this from Linux?

ereyes01 8y ago
Yes, I do all this on Linux, but they support other environments too (most of the environments Go can build for): https://rclone.org/downloads/ https://rclone.org/downloads/

asenna 8y ago

I'll definitely give this a try. I'm currently backing up about 1.7TB of pictures to B2 from my Qnap NAS. Qnap has a backup app called Hybrid Backup Sync. The problem is, while doing the one-way upload sync, the Qnap app downloaded a lot of data as well. I got confused why I was showing a lot of 'b2_download_file_by_name' API calls on the Backblaze reports page (600 GB upload resulted in 700 GB download calls). I've also opened a thread on the forum - https://forum.qnap.com/viewtopic.php?p=673557 https://forum.qnap.com/viewtopic.php?p=673557 Contacted Qnap support and they said a little bit of download is normal but this looks abnormal. Logs are all fine on the Qnap so they suggested I contact Backblaze. I wonder if anyone else has faced this.