4 ms·

Does Backblaze support Linux yet? I have store almost all of my data on a NAS, and it would be great to have that automatically backed up online, but the last t

by foldor 8y ago

Does Backblaze support Linux yet? I have store almost all of my data on a NAS, and it would be great to have that automatically backed up online, but the last time I looked into it they only supported Mac/Windows. I assume the reason for this was to keep out users with large NAS capacity and allow them to keep advertising unlimited storage. I would be ok with having Linux computers capped, I'm not storing more than about 2TB of data.

geephroh 8y ago
They don't have their own client offering, but super-easy to roll your own with their B2 cloud service. There's support for backblaze targets in duplicity, hashbackup, restic and rclone -- my org uses it as a special remote for git-annex, and also use it to back up our synology units (see: https://www.backblaze.com/b2/integrations.html https://www.backblaze.com/b2/integrations.html). 2TB would cost you about $10/month for the storage ($0.005/GB), plus $0.01/GB for downloads.
capelio 8y ago
We use their Python-based CLI to do daily off-site backups of Postgres snapshots from a Linux box. https://www.backblaze.com/b2/docs/quick_command_line.html https://www.backblaze.com/b2/docs/quick_command_line.html
qeternity 8y ago
As much as I love B2, their SDK support is pretty terrible. There are a handful of half-baked and abandoned libraries. We've been using an internal python library which we're starting to open source and plan on maintaining. We've also forked wal-e to support this library and B2 endpoint (currently have basic upload functionality working). It would not take much effort from Backblaze to throw some resources behind these efforts, or hell, to host their own S3 proxy (I'll never understand the design choices behind their API).
- brianwski 8y ago
  Disclaimer: I work at Backblaze. > Backblaze's APIs are not S3 compatible. .... I'll never understand the design choices behind their API It is all clear if you understand our APIs are cheaper to implement than Amazon S3's APIs. I freely admit the APIs are LITTLE BIT more complicated than S3 (which is not a good thing). Let me start from the beginning and explain why: We originally never planned to open the APIs up for general use. We created the Backblaze Personal Backup client and we owned both the client and the server, so we were originally willing to do a tiny bit more programming to get rid of some expensive equipment and choke points in the datacenter. In the S3's APIs, you simply "upload" and you are done. This means there is a very high availability choke point on upload. With Backblaze client (and later "B2" APIs), there are no choke points. The Backblaze client contacts (HTTPS) a central dispatch server and asks the dispatch server for a pod (computer) that contains some spare disk space. Then something very very important occurs that does not occur with Amazon S3 -> the client entirely stops talking with the central dispatch server. And then the client contacts the pod DIRECTLY to perform the uploads. Now the "contract" with the storage pods is that if that one storage pod crashes, or fills up, or decides not to talk with the client anymore, the client is absolutely responsible to go back to the "dispatch server" and request a new pod somewhere else in the Backblaze datacenter to upload data into. By requiring these one or two extra steps, Backblaze does not have to buy any expensive load balancing hardware, and we only need 10 Gbit network cards, there are no central choke points requiring multi-Terabit networking speeds like Amazon. Instead we have several thousand individual 10 Gbit network cards which cost us almost nothing because they come built into the Intel motherboards nowadays. The switches cost a few bucks per port. I assume that Amazon S3 accepts the data, then has to "move it" to the final destination with a network copy. In Backblaze, the data literally lands in the final destination coming straight from the client. Fewer copies means lower cost and higher performance. Backblaze purchases no load balancers, except for some "tricks" to make the "dispatch servers" highly redundant. But the dispatch servers have only TINY trickles of info coming and going, and the clients only have to contact the dispatch server ONCE every few days (then upload endlessly to the final destination pods), so even the dispatch servers can be "only" 10 Gbit. > Backblaze is incompatible with Amazon S3 TL;DR - by requiring clients to do a small amount of extra programming and making programmers understand the "contract" (which is the client must retry some things and handle a few more errors), Backblaze cuts out a lot of cost out of our datacenter while INCREASING scalability over S3. We pass the money savings along to our customers. We throw in the increase scalability over S3 for free. :-)

by foldor 8y ago

geephroh 8y ago

They don't have their own client offering, but super-easy to roll your own with their B2 cloud service. There's support for backblaze targets in duplicity, hashbackup, restic and rclone -- my org uses it as a special remote for git-annex, and also use it to back up our synology units (see: https://www.backblaze.com/b2/integrations.html https://www.backblaze.com/b2/integrations.html). 2TB would cost you about $10/month for the storage ($0.005/GB), plus $0.01/GB for downloads.

capelio 8y ago

We use their Python-based CLI to do daily off-site backups of Postgres snapshots from a Linux box. https://www.backblaze.com/b2/docs/quick_command_line.html https://www.backblaze.com/b2/docs/quick_command_line.html

qeternity 8y ago

As much as I love B2, their SDK support is pretty terrible. There are a handful of half-baked and abandoned libraries. We've been using an internal python library which we're starting to open source and plan on maintaining. We've also forked wal-e to support this library and B2 endpoint (currently have basic upload functionality working). It would not take much effort from Backblaze to throw some resources behind these efforts, or hell, to host their own S3 proxy (I'll never understand the design choices behind their API).

brianwski 8y ago
Disclaimer: I work at Backblaze. > Backblaze's APIs are not S3 compatible. .... I'll never understand the design choices behind their API It is all clear if you understand our APIs are cheaper to implement than Amazon S3's APIs. I freely admit the APIs are LITTLE BIT more complicated than S3 (which is not a good thing). Let me start from the beginning and explain why: We originally never planned to open the APIs up for general use. We created the Backblaze Personal Backup client and we owned both the client and the server, so we were originally willing to do a tiny bit more programming to get rid of some expensive equipment and choke points in the datacenter. In the S3's APIs, you simply "upload" and you are done. This means there is a very high availability choke point on upload. With Backblaze client (and later "B2" APIs), there are no choke points. The Backblaze client contacts (HTTPS) a central dispatch server and asks the dispatch server for a pod (computer) that contains some spare disk space. Then something very very important occurs that does not occur with Amazon S3 -> the client entirely stops talking with the central dispatch server. And then the client contacts the pod DIRECTLY to perform the uploads. Now the "contract" with the storage pods is that if that one storage pod crashes, or fills up, or decides not to talk with the client anymore, the client is absolutely responsible to go back to the "dispatch server" and request a new pod somewhere else in the Backblaze datacenter to upload data into. By requiring these one or two extra steps, Backblaze does not have to buy any expensive load balancing hardware, and we only need 10 Gbit network cards, there are no central choke points requiring multi-Terabit networking speeds like Amazon. Instead we have several thousand individual 10 Gbit network cards which cost us almost nothing because they come built into the Intel motherboards nowadays. The switches cost a few bucks per port. I assume that Amazon S3 accepts the data, then has to "move it" to the final destination with a network copy. In Backblaze, the data literally lands in the final destination coming straight from the client. Fewer copies means lower cost and higher performance. Backblaze purchases no load balancers, except for some "tricks" to make the "dispatch servers" highly redundant. But the dispatch servers have only TINY trickles of info coming and going, and the clients only have to contact the dispatch server ONCE every few days (then upload endlessly to the final destination pods), so even the dispatch servers can be "only" 10 Gbit. > Backblaze is incompatible with Amazon S3 TL;DR - by requiring clients to do a small amount of extra programming and making programmers understand the "contract" (which is the client must retry some things and handle a few more errors), Backblaze cuts out a lot of cost out of our datacenter while INCREASING scalability over S3. We pass the money savings along to our customers. We throw in the increase scalability over S3 for free. :-)