14 ms·

Show HN: B2blaze – A Backblaze B2 library for Python

359 points by gsibble 8y ago

Lindahawley 8y ago
I Basically think we all don't have to face all this lies from fake hackers out there, I came across a friend who was seeking for the help of a legit credit repair specialist/ethical hacker.I sharply advice her to contact DARKWEB SERVICE because he help me delete late payments which i incurred as far back as 2016 when i lost my job and was struggling with my finances, blemishes that been on my credit report and two inquiries included, increase my credit score with a value of over 810 which he called a golden score.He also recommended few steps on how to maintain the score for ever and i have been making use of it without going through a stress and now free from scammers.I told my friend to sharply send a message to his email like i said to her earlier on. darkwebcyberservice@gmail.com. I was saved by a lot of stress and hardship through the help of this wizard and i must review him online for those who need his assistants.
dzek69 8y ago
Hah, I hadn't heard about Blackbaze in a while and I was even thinking about creating Ask HN asking if anyone was maybe using Blackbaze for a longer while and can say something about them (speed, data reliability). Now I'll take my chance: Had you used Blackbaze B2? How was your experience?
- trigoman 8y ago
  I’ve been using BackBlaze B2 for about 6 months and it’s been very stable! The Java SDK is OK. So I plan on eventually having my application talk to Minio which will allow me to use the S3 API.
  - gsibble 8y ago
    The idea behind this was to avoid Minio at least when using Python with a very simple to use SDK.
- chrisper 8y ago
  I have used B2 for a couple of months. From Europe it is way too slow. They only have like 2 DC, which are both on the west coast I believe. As I am using this for backups only, I went back to Google Drive where I can max out my Gbit upload.
  - jonatron 8y ago
    It'd be interesting to find out if it's acceptably fast when proxying via a cloud provider.
    - gsibble 8y ago
      Here is the CTO of Backblaze addressing speed questions earlier today: https://www.reddit.com/r/Python/comments/8u40xa/i_wrote_a_library_for_backblaze_b2_75_cheaper/e1codwz https://www.reddit.com/r/Python/comments/8u40xa/i_wrote_a_li...
      - voltagex_ 8y ago
        Maybe I misunderstand his point, but isn't transferring data from Australia to the US or reverse always going to be slower due to the speed of light? What he's said doesn't negate the fact that he's only got one POP.
        gsibble 8y ago
        I'm not sure how many POPs they have so I do agree with that. I'm not shilling for B2 and have no affiliation with them. I've had good performance them but am US based.
        glenneroo 8y ago
        My individual connections to B2 are slow (200-500 KB/s) coming from Europe but that's why the multi-threaded option makes B2 very usable.
        monort 8y ago
- manigandham 8y ago
  It's very cheap and effective for archival storage without having to deal with time/cost issues when you actually need to retrieve something. I use it for all my media so I can store terabytes and download in minutes for viewing. Bandwidth is limited since they aren't connected like the major clouds, but it's workable if you don't need gigabit speeds. Single API key for permissions and lacks all the other features like events, object lifecycle, etc. Basic reporting but shows bucket size in real-time which is nice. API can be annoying because it requires a request to "start" an upload (to get the address of where to upload), then doing the actual upload itself, but this can be automated away. Only single region for now (with multiple datacenters that aren't visible to you) so no global replication for extra durability or locality. They have a partnership with https://www.packet.net https://www.packet.net (cloud bare metal) for free interconnect between their servers and B2 so you can do processing on your data without the public internet bottleneck and fees. Allows for an interesting data lake/warehouse option. Use Cyberduck for a decent GUI client. If you just need personal computer backup, then use their actual backup offering which is unlimited storage and has auto-uploading background app.
  - lithiumfrost 8y ago
    Perhaps. Personally I use Arq with B2 for a back end, and this usually costs me less than $1/mo (their regular backup starts at $5). In addition, using B2 with something like duplicity is the better approach for backing up Linux or NAS boxes, where their official client is not supported.
- maxyme 8y ago
  I've used B2 for some internal backup handling (several 100s of GB but millions of files) and have largely found it inexpensive and performant. A few considerations: Their web UI can not handle large amounts of files (support said after a few million the file browser will not work). Sometimes when making a large number of deletions at once the API may serve 500 errors and the web UI give Java Servlet errors (this only happened a few times and resolved itself in a hour or two). As another user noted the per file/fragment upload speed isn't fantastic but I could max out my gigabit fiber with many concurrent downloads/uploads. The API has no concept of folders, only file path strings (which is mildly annoying to work with). Lastly I think all the data is currently housed in one geographic area but they are working on a DC in Phoenix. Overall a pretty smooth experience but I was mostly using it for cold data.
  - voltagex_ 8y ago
    Have you reported this stuff to support? I found their support was great, even when dealing with me noodling around in C# and breaking things.
- yani 8y ago
  Only good if you are using it from the states.
- haywirez 8y ago
  I'm trying to use it as an S3 replacement for audio content delivery - seems slow and laggy unfortunately. Uploads also fail frequently enough. I don't know if CDNs would make a big difference. (Europe) Edit: lack of webhooks or something similar for doing follow-up after successful uploads is also irritating.
voltagex_ 8y ago
Are you able to try multithreaded uploads too? I found that single stream uploads were too slow (< 10 megabytes per second) but I could get ~35 megabytes per second from packet.net to B2 by using 4 threads. Edit: removed incorrect stuff.
- thisacctforreal 8y ago
  If you use the large_file API (needed for multithreaded uploads), you do only hash chunks not the file.
  - voltagex_ 8y ago
    Okay, I'll have to go back and have a look at some of the client libraries I tried - it may have been the machine I was using wasn't quick enough to hash ~30GB in a reasonable amount of time.
  - brianwski 8y ago
    Disclaimer: I work at Backblaze. > If you use the large_file API (needed for multithreaded uploads) We recommend for small files that you use multi-threaded where each thread sends a totally separate file. So if you have to upload both cat.jpg and dog.jpg, you upload cat.jpg in one thread and dog.jpg in another thread. Based on the Backblaze architecture, that means cat.jpg will be sent to one "vault" in the Backblaze datacenter with one thread, and dog.jpg will be sent to a totally different "vault" in the Backblaze datacenter with another thread. This scales incredibly well, in that it should be twice as fast for two files, and 20 times as fast for 20 files if you do it correctly. Source: I wrote a lot of the Backblaze Personal Backup client, which uses this philosophy.
- gsibble 8y ago
  I haven't added multi threaded stuff yet because I wanted it to be compatible with single threaded web servers like flask and django. I can and will add it if you want to add an issue.
  - voltagex_ 8y ago
ereyes01 8y ago
I use rclone's B2 driver (https://rclone.org/b2/ https://rclone.org/b2/) as an rsync-style backup solution of about 1TB worth of my pictures and other media. Also, it's encrypted with my own local key using rclone's crypt module (https://rclone.org/crypt/ https://rclone.org/crypt/). rclone supports multithreaded upload, and even has experimental support for FUSE mounting. However, the sync command gets you Dropbox-like behavior and can be cronned: https://rclone.org/commands/rclone_sync/ https://rclone.org/commands/rclone_sync/ I really like the price of B2, I hope it stays low :-)
- gordo4 8y ago
  rclone is vulnerable to data exfiltration attacks https://www.danieldent.com/blog/restless-vulnerability-non-browser-cross-domain-http-request-attacks/ https://www.danieldent.com/blog/restless-vulnerability-non-b...
  - Operyl 8y ago
    I quickly skimmed, but this entire attack is assuming that the attacker has successfully MITMed the API. At that point everything is already nuked, so of course you can fabricate any number of attacks. Did I miss something important?
  - howeyc 8y ago
    Am I reading this right? Google/B2/... might send your data to another URL you didn't expect. Not sure why that matters, or why it's an attack. Since they have your data anyway, as that's the whole point of the service, to store your data on their hard drives. Why go through the trouble of sending it elsewhere? To play games with your data for giggles?
    - namibj 8y ago
      No, the API can tell your software to send some private LAN files, e.g. some IP-filtered secret NFS store, to an URL of it's choosing (so to itself, or your competitor). This is bad, as long as you don't heavily jail and firewall the software to prevent it from ever accessing anything it shouldn't (need to).
scottybowl 8y ago
Any ideas on using Backblaze as a driver in a Laravel project? We're currently using Digitalocean Spaces but it's very unreliable.
- gsibble 8y ago
  I believe Laravel is PHP so you'd need a PHP library while this one is Python.
  - deleted 8y ago
    [deleted]
- shakna 8y ago
  amelia/backblaze [0] seems to be the highest quality, but there are a few around. [0] https://github.com/ameliaikeda/backblaze https://github.com/ameliaikeda/backblaze
logeek 8y ago
Is there a fundamental reason why B2 is (and will remain) cheaper than S3, or is it just because they need to compete with AWS and once successful the prices will be the same (or higher)?
- agopaul 8y ago
  Objects on S3 are replicated on 3 AZs (datacenters) by default and I can't find any info on B2 if they also replicate the data on multiple DCs. That can definitely change the cost per GB for them if that's the case.
  - gsibble 8y ago
    B2 has 99.999999% durability
    - snvzz 8y ago
      That's just 8 9s. S3 has 11 9s.
      - gsibble 8y ago
        Depends upon your client SLA then. You can also store the file twice on B2 for still less than the cost of S3 :)
        kccqzy 8y ago
        Will failures be correlated then?
        snvzz 8y ago
        Then you should be comparing it to reduced redundancy S3, which is cheaper than regular S3.
    - scarface74 8y ago
      That’s nice in theory, but if B2 has a castrophic disaster in thier one data center, that durability goes to zero.
Dawny33 8y ago
The best part is this API looks and feels very similar to AWS's boto S3 API. Great job!
- gsibble 8y ago
  Thank you!
DanielDent 8y ago
B2's API design has security implications: https://www.danieldent.com/blog/restless-vulnerability-non-browser-cross-domain-http-request-attacks/ https://www.danieldent.com/blog/restless-vulnerability-non-b...
- voltagex_ 8y ago
  I don't see how B2 is affected by this unless you assume malicious control of a B2 API server.
  - DanielDent 8y ago
    It's fairly standard for network clients to assume potential malicious control of the server they are connecting to. It helps reduce the blast radius of a compromised server. In the case where the server is operated by a third party (as is the case with the B2 API server), there can be many compliance implications if that third-party-operated server has access to an internal network. We don't accept when SSH clients or web browsers have the ability to do things they shouldn't based on instructions sent by the server they connect to. Why would we suddenly have lower expectations of our file storage API clients? (or any other network/HTTP clients for that matter)
    - voltagex_ 8y ago
      Ah, I see what you're getting at. It'd be better if the URL for get_upload_url (I think that's what the API was called) could be calculated client side. At the moment, you're probably still more at risk of downloading a malicious library from PyPi or npm but this is sure to turn up in a CTF at some point - even curl is technically vulnerable. Have you talked to anyone from Backblaze about this?
      - DanielDent 8y ago
        Yes, client-side URL calculation and/or a whitelist of acceptable URLs would be a significant improvement. Thankfully command-line curl won't follow redirects unless you pass it a special flag, though if you do need it to follow redirects, I'm not sure what the best way is to restrict the range of redirects that it will follow. This issue was part of a broader coordinated disclosure and was only published today. I've gotten in touch with B2 support & I'm hoping my support ticket will make it to the correct people.
krn 8y ago
Hetzner Storage Box[1] is an interesting alternative to Backblaze B2. It's not cloud-based, but provides free automated snapshots, free 1 Gbps bandwidth, and supports FTP, FTPS, SFTP, SCP, rsync and BorgBackup[2]. [1] https://www.hetzner.com/storage-box https://www.hetzner.com/storage-box [2] https://wiki.hetzner.de/index.php/Storage_Boxes/en https://wiki.hetzner.de/index.php/Storage_Boxes/en
- hardwaresofton 8y ago
  +1 for Hetzner, they have an amazing suite of services for excellent prices. Hetzner's robot marketplace[0] changed my (operations) life. [0]: https://robot.your-server.de/order/market https://robot.your-server.de/order/market
  - apkallum 8y ago
    How did they do so?
    - hardwaresofton 8y ago
      tl;dr - Shifted my knowledge of the hosting market by exposing me to cheap dedicated servers I had previously thought that dedicated servers were doomed to be too expensive/heavy weight for me. I also felt like most VPS providers charged too much (especially true in the case of AWS -- $10/mo for a t2.micro is ridiculous). I first found INIZ (http://iniz.com/ http://iniz.com/) and was super happy with them, then someone introduced me to Hetzner Robot Marketplace and I was blown away by the affordable prices (+/- setup fee) and have had one ever since. Hetzner also has a cloud offering that is also pretty great -- slight limits on operating system choice and some other features and you can have very competitively priced machines in a more cloud-friendly fire-up-and-go format. I wrote about the revelation here (including link to HN thread where it happened): https://vadosware.io/post/fresh-dedicated-server-to-single-node-kubernetes-cluster-on-coreos/#why-change-my-infrastructure-at-all https://vadosware.io/post/fresh-dedicated-server-to-single-n... Now I have a ~6 Core (12 vCore/hyper-thread) 24GB RAM monster that I can run experiments with for a decent monthly price. If you go to other providers like Packet, OVH or Amazon, you're going to see way higher prices -- I'm don't have too many requirements so Hetzner worked for me.
Neil44 8y ago
Duplicity has support for B2 back end now, I use it for a lot of backup and it works well. Versioning, encryption etc.
- gsibble 8y ago
  This library is meant more for web apps and other applications built on Python. Not necessarily for backups.
simula67 8y ago
> Backblaze B2 provides the cheapest cloud object storage and transfer available on the internet Are you sure ? http://gaul.org/object-store-comparison/ http://gaul.org/object-store-comparison/ says there are cheaper options EDIT: Edited the link away from https://wasabi.com/pricing/ https://wasabi.com/pricing/. Wasabi seems cheaper than B2 and claims to be a hot storage solution
- gsibble 8y ago
  That does indeed look cheaper. I'll have to make an SDK for them next :D edit: Woah, you edited your comment. What made you change from Wasabi?
  - simula67 8y ago
    Sorry, I was thinking about Wasabi when I was reading your post, but then I checked if there were even cheaper options
  - qeternity 8y ago
    Wasabi has a 90 day minimum storage period, just like AWS Glacier. This means it's pretty unusable for things like nightly backups if your retention is less than 90 days.
    - koolba 8y ago
      If your backup process involves downloading the backup artifacts in a different region (say for true off site DR + validation) then it's still a net win as the 90-day storage costs are less than the insanely high $.09/GB AWS charges for outbound bandwidth.
- qeternity 8y ago
  We are big users of OVH, AWS and B2 object storage. OVH charge for ingress and egress even if local. AWS Glacier has 90 minimum storage time. For most uses cases, B2 is much much cheaper. EDIT: Response to your edit, Wasabi also has 90 day minimum storage policy.
gtt 8y ago
Can I use B2 as replacement for S3? Namely, I need "get upload URL" like pre-signed S3 URLs.
metakermit 8y ago
BTW, the Backblaze team themselves officially support a Python implementation of the API wrapper (the B2Api and Bucket classes): https://github.com/Backblaze/B2_Command_Line_Tool/blob/master/b2/api.py https://github.com/Backblaze/B2_Command_Line_Tool/blob/maste... As these are used internally in their CLI, there's probably a higher chance that they'll continue to work in the future.
- gsibble 8y ago
  Indeed. It's just never been publicized and personally I think the implementation could be better. Although with so many stars on this repo now I plan on maintaining it for the foreseeable future. Start adding feature requests everyone! :) PS: They also don't even use their own library in their code examples so I don't think they meant it to be used in that fashion.
  - metakermit 8y ago
    That would be great – it's always good to have some competition ;) Regarding feature requests I'd love to see a well-maintained B2 Django Storage. I'm currently using an existing implementation, but it's not that well maintained: https://github.com/royendgel/django-backblazeb2-storage https://github.com/royendgel/django-backblazeb2-storage
    - gsibble 8y ago
      I saw that one. I'm not a particular fan of Django but integrating my library apart from Django's storage library wouldn't be difficult. Neither would be building a django library on top of mine. Any takers? :)
gsibble 8y ago
admins: Will you please change the title to a Show HN? Thank you!
- sctb 8y ago
  Sure, done!
  - gsibble 8y ago
    Thanks!
Aissen 8y ago
According to online.net "cold storage" C14 comparison, they are cheaper than Backblaze, most of the time: https://www.online.net/en/c14 https://www.online.net/en/c14
- gsibble 8y ago
  B2 is warm storage much more like S3 while cold storage is like glacier.
- qeternity 8y ago
  C14 is really not at all an object store. Getting data in and out is a huge pain, even compared to other cold stores like AWS or OVH. We evaluated them and passed.
  - Aissen 8y ago
    Thanks, I was looking for this type of feedback. The pricing comparison is still interesting. Could you do a summary of your evaluation for others that didn't test most services?
    - qeternity 8y ago
      Yeah, here's a quick stab: S3 - not much to say, fast, durable, expensive...the gold standard. Given limitations of below, we use for rotating nightly backups despite cost. Glacier - great for cold storage/archive, but has 90 day minimum OVH hot - open stack based, cheaper than S3 but not absurdly cheap, charged for egress even intra-DC which is absurd and kills many use cases. They have crippled OpenStack permission management (i.e. no write-only keys with lifetime management per bucket which is necessary for doing backups securely) OVH cold - charges for ingress but then storage is crazy cheap, and egress not as bad as Glacier. This is our preferred archival option. C14 - not object storage, more like a "cold" ftp dump B2 - pricing is epic, S3-incompatibility is a pain and lack of Backblaze-sponsored libraries (the library in the python b2 cli is not a proper API)...we've been working on adding B2 to WAL-E. However, their permission/user management doesn't cut it. Wasabi - S3 compatible, great pricing if not for 90 day minimum, which they hide in the fine print
piqufoh 8y ago
This feels like a handy tool! The first thing I read when opening new code is the test suite - it's worth getting that right at the start. Would you consider deeper unit testing? S3 (and aws) have the indispensible `moto` boto mocks, I think something similar would be dead handy here.
- gsibble 8y ago
  Yes, on my TODO list is much deeper unit testing. I made this in four days and was just testing that it worked. It already has about 92% code coverage but I want to cover that fields are returned properly and such. Some help would be appreciated if people would like to, including mocking it up.
weitzj 8y ago
It seems like nobody mentioned it, yet. Another great product is https://www.rsync.net/ https://www.rsync.net/ and this just works. There are no bad surprises. You can overshoot your backup limits, and they will send you an email to fix this. But still you have your backup. Your interface is rsync/scp/ssh. They give you ZFS snapshots, you can use s3cmd from their machines, so you can delegate uploads to S3 via rsync.net. Our prior backup setup was duplicity with GPG hitting S3, and this sometimes was flaky for listing the current keys. Glad I read HN, I heard about rsync.net. They even have/had a HN discount. You should use the search functionality to find other threads. https://www.rsync.net/products/platform.html https://www.rsync.net/products/platform.html https://arstechnica.com/information-technology/2015/12/rsync-net-zfs-replication-to-the-cloud-is-finally-here-and-its-fast/ https://arstechnica.com/information-technology/2015/12/rsync...
- cahuk 8y ago
  Speaking of ZFS, I use B2 with zfsbackup-go. Sanoid is making snapshots and zfsbackup-go is uploading them. [zfsbackup-go](https://github.com/someone1/zfsbackup-go https://github.com/someone1/zfsbackup-go)
- alexktz 8y ago
  Great product but too $$$$$$$$$$$$$ for my taste.
  - anc84 8y ago
    It's almost like they try to be a self-sustaining, viable business instead of burning through VC money with irresponsible pricing that kills fair competitors.
    - Karunamon 8y ago
      The snark is really not necessary or contributing to the conversation. AWS Glacier, hardly a VC-backed startup, charges a literal tenth of the cost. Given that most people are going to be holding on to their backups rather than retrieving them regularly, the pricing math works out better even though it's a bit more complicated. Say you push 2TB up to Rsync, AWS Glacier, and Backblaze B2, and you need that data back a year later. Rsync will cost you $80x12: $960, bottom line. Glacier will cost you $8.00x12: $96 for the storage, plus .01 for a thousand retrieve requests, plus 0.01 per gigabyte retreival, plus 0.09 per gigabyte transfer. $96 + .01 + $20 + $180 = $296.10 Backblaze B2: $10x12 = $120 for the storage, plus 0.01 per gigabyte retrieved: $120 + $20 = $140 I'm guessing the "startup" dig was directed at Backblaze, but they're actually charging more for the plain storage than AWS, where you're paying more for the bandwidth!
erickj 8y ago
Hey that's cool. I made a B2 Ruby gem a few years back that gives a library and cli for the API. +1,000,000 for backblaze! https://github.com/erickj/bkblz https://github.com/erickj/bkblz
- gsibble 8y ago
  Awesome!
Sami_Lehtinen 8y ago
Is this the best way of dealing with Python 3 compatibility? What's wrong with sys.version_info? Or is there even some better way? Ref: https://github.com/sibblegp/b2blaze/blob/master/b2blaze/utilities.py https://github.com/sibblegp/b2blaze/blob/master/b2blaze/util...
artellectual 8y ago
That's funny, I did the same thing for elixir, feel free to check it out https://github.com/upmaru/upstream https://github.com/upmaru/upstream. Not well documented yet but it's coming.
scarface74 8y ago
I use Backblaze now and once I get my NAS, I’ll probably end up using a B2 based backup. But let’s make an honest comparison. Backblaze does not replicate your data across data centers. The standard S3 storage class does (0.23/gb). The comparible storage class for S3 is one zone infrequent access (.01/gb). B2 still comes out ahead, but I wouldn’t use either one for primary storage. For thier suggested “3-2-1” backup strategy, sure. Then again, just for backup, I could use S3 glacier for $.004/gb. That’s cheaper than B2 and I get multiple AZ storage. The data charges would be higher - but its backup. If catastrophe struck and I lost my primary and my local backups, getting my data fast is the last thing I would worry about.
- deleted 8y ago
  [deleted]
- josteink 8y ago
  > Then again, just for backup, I could use S3 glacier for $.004/gb Having done that in the past, I have to say that's just a million times less practical than basic S3-like storage. And if you want to automate that setup, Glacier is even worse.
  - scarface74 8y ago
    Why do you say that? I could see using something like rsync + Cloudberry (maps S3 and make it look like a network drive). Set it up to use one zone infrequent access, and then after x days use a lifecycle policy to move it to Glacier. My use case for backups is solely for movies and music. For source code I use hosted git repos, pictures Google photos, and for regular office documents, they are either on Google docs or One Drive.
    - josteink 8y ago
      Last time I used Glacier, it was a separate product from S3 and had its own API. You had to upload pre-prepared "tapes" for backups. You couldn't mutate an existing backup, you had to create a new one. And frequently fetching and/or deleting existing "tapes" (backups) would cost you money (more so than the original cost of the backup). That meant you couldn't just ZIP it all up, backup the latest version and the delete the previous one to avoid being doubly charged for storage either. Basically at time of archiving you needed to determine what was already archived and create a new bundle with only what's new, and archive that only. In the same spirit, restore meant piecing together multiple such tapes into a full restore-set. Absolutely terrible. It was like having traditional backup-software constraints, but none of the software-support. If Amazon has improved on that now, good for them, but I figured they probably had to if they wanted any users at all.
tobias3 8y ago
This whole library wouldn't be necessary if Backbalze implemented a S3 compatible API. They give reasons like being able to load balance on the client for their API (which I do no think is a good reason), but ultimately they just push work from their end to a lot of applications and developers. Maybe it also has a strategic advantage? Now every product has to announce they support B2 whereas nobody has to announce they support Wasabi, because they support any S3 compatible storage such as AWS S3, Google Cloud Storage or Wasabi.
- ianamartin 8y ago
  Meh, I can see why they didn't. they aren't really in the same business. It makes sense for followers to implement APIs compatible with market leaders. Riak CS is API compatible with S3, which is nice. But it's literally intended to be an open source version of S3 that you can host and scale yourself. Backblaze is in a different market. They may be finding out that there's overlap and allowing that use. But they are not the same and probably aren't prepared for developers to start using b2 en masse. I think it makes business sense. You want to save some money? Do a little extra work for the cheaper product. Want to save even more money? Roll your own with Riak CS. Cloud services all work along the same spectrum where you pay more for convenience and ease of use, and you pay less up front if you're willing to pay in developer or devops or infrastructure costs. I think this fits in nicely on that spectrum.
- gaul 8y ago
  Object stores unfortunately innovate on their APIs instead of their implementations. I wrote S3Proxy to bridge the gap between S3 applications and a variety of object stores including B2: https://github.com/gaul/s3proxy https://github.com/gaul/s3proxy
brian_herman 8y ago
B2 does not have the durability guarantees that s3 has it is not a fair comparison.
iosDrone 8y ago
I thought the knock against backblaze is that (compared to Google or Amazon), uploading large amounts of data to it is very slow.
- namibj 8y ago
  If you follow the best practices, that is not an issue.
- brianwski 8y ago
  Disclaimer: I work for Backblaze so I'm biased. > uploading large amounts of data to [Backblaze B2] is very slow All reports we (Backblaze) hear is that if you only use one thread, Backblaze B2 is slightly slower than S3 (like maybe 90% of the performance). If somebody has better numbers I would LOVE to see them! If clients use multiple threads, this issue goes entirely away. Using 500 threads can provably be 500 times as fast with Backblaze. This is because the Backblaze B2 architecture means there are no "choke points" like Amazon S3 has. Each thread will most likely be talking to a completely different "Backblaze Vault" maybe even in a completely separate Backblaze datacenter. Since they don't share any network switches or load balancers in common, there is no way they will slow down. But again, I would love any measurements or reproducible tests showing differences so we can chase them down and improve Backblaze B2!
  - iosDrone 8y ago
    Fair enough. This is just what I've heard second-hand, so I definitely don't have any data to back it up. I may try it out.
post_break 8y ago
I just wish Backblaze would fix their snapshots. You still have no way to tag a snapshot, put in any notes, anything. You literally make two snap shots 5 minutes apart and they only thing that differentiates them is the time stamp. unforgivable.
- brianwski 8y ago
  Disclaimer: I work at Backblaze. > You still have no way to tag a snapshot, put in any notes, anything. We totally agree, and the project is fully spec'ed, just waiting for an available engineer to implement it! On a side note, we also have open recs for engineers. :-)
  - post_break 8y ago
    Thank you! Please send it out in an email or something when it's ready. We've been waiting for this for a while now which is why I'm so grumpy lol.
jonotime 8y ago
I'm loving B2 for my Linux desktop backup. I used Crashplan for many years until they pulled the plug recently. Now I'm using B2 via Duplicati and I'm actually saving money (I have about 500Gb of backup). Borg was a runner up, but Duplicati had built in B2 support, provided scheduling, and a web interface which makes navigating for specific files in a tree easy when needed. Crashplan had a nice Linux client, but it was blackbox/closed source, so problems came up from time to time that were hard to debug. So its nice to have more control of my data as well.
hinkley 8y ago
A little off topic. Did I miss the backblaze hard drive reliability report? I think it's been a while since they've done one.

359 points by gsibble 8y ago

I Basically think we all don't have to face all this lies from fake hackers out there, I came across a friend who was seeking for the help of a legit credit repair specialist/ethical hacker.I sharply advice her to contact DARKWEB SERVICE because he help me delete late payments which i incurred as far back as 2016 when i lost my job and was struggling with my finances, blemishes that been on my credit report and two inquiries included, increase my credit score with a value of over 810 which he called a golden score.He also recommended few steps on how to maintain the score for ever and i have been making use of it without going through a stress and now free from scammers.I told my friend to sharply send a message to his email like i said to her earlier on. darkwebcyberservice@gmail.com. I was saved by a lot of stress and hardship through the help of this wizard and i must review him online for those who need his assistants.

dzek69 8y ago

Hah, I hadn't heard about Blackbaze in a while and I was even thinking about creating Ask HN asking if anyone was maybe using Blackbaze for a longer while and can say something about them (speed, data reliability). Now I'll take my chance: Had you used Blackbaze B2? How was your experience?

trigoman 8y ago
I’ve been using BackBlaze B2 for about 6 months and it’s been very stable! The Java SDK is OK. So I plan on eventually having my application talk to Minio which will allow me to use the S3 API.
- gsibble 8y ago
  The idea behind this was to avoid Minio at least when using Python with a very simple to use SDK.
chrisper 8y ago
I have used B2 for a couple of months. From Europe it is way too slow. They only have like 2 DC, which are both on the west coast I believe. As I am using this for backups only, I went back to Google Drive where I can max out my Gbit upload.
- jonatron 8y ago
  It'd be interesting to find out if it's acceptably fast when proxying via a cloud provider.
  - gsibble 8y ago
    Here is the CTO of Backblaze addressing speed questions earlier today: https://www.reddit.com/r/Python/comments/8u40xa/i_wrote_a_library_for_backblaze_b2_75_cheaper/e1codwz https://www.reddit.com/r/Python/comments/8u40xa/i_wrote_a_li...
    - voltagex_ 8y ago
      Maybe I misunderstand his point, but isn't transferring data from Australia to the US or reverse always going to be slower due to the speed of light? What he's said doesn't negate the fact that he's only got one POP.
      - gsibble 8y ago
        I'm not sure how many POPs they have so I do agree with that. I'm not shilling for B2 and have no affiliation with them. I've had good performance them but am US based.
        glenneroo 8y ago
        My individual connections to B2 are slow (200-500 KB/s) coming from Europe but that's why the multi-threaded option makes B2 very usable.
      - monort 8y ago
manigandham 8y ago
It's very cheap and effective for archival storage without having to deal with time/cost issues when you actually need to retrieve something. I use it for all my media so I can store terabytes and download in minutes for viewing. Bandwidth is limited since they aren't connected like the major clouds, but it's workable if you don't need gigabit speeds. Single API key for permissions and lacks all the other features like events, object lifecycle, etc. Basic reporting but shows bucket size in real-time which is nice. API can be annoying because it requires a request to "start" an upload (to get the address of where to upload), then doing the actual upload itself, but this can be automated away. Only single region for now (with multiple datacenters that aren't visible to you) so no global replication for extra durability or locality. They have a partnership with https://www.packet.net https://www.packet.net (cloud bare metal) for free interconnect between their servers and B2 so you can do processing on your data without the public internet bottleneck and fees. Allows for an interesting data lake/warehouse option. Use Cyberduck for a decent GUI client. If you just need personal computer backup, then use their actual backup offering which is unlimited storage and has auto-uploading background app.
- lithiumfrost 8y ago
  Perhaps. Personally I use Arq with B2 for a back end, and this usually costs me less than $1/mo (their regular backup starts at $5). In addition, using B2 with something like duplicity is the better approach for backing up Linux or NAS boxes, where their official client is not supported.
maxyme 8y ago
I've used B2 for some internal backup handling (several 100s of GB but millions of files) and have largely found it inexpensive and performant. A few considerations: Their web UI can not handle large amounts of files (support said after a few million the file browser will not work). Sometimes when making a large number of deletions at once the API may serve 500 errors and the web UI give Java Servlet errors (this only happened a few times and resolved itself in a hour or two). As another user noted the per file/fragment upload speed isn't fantastic but I could max out my gigabit fiber with many concurrent downloads/uploads. The API has no concept of folders, only file path strings (which is mildly annoying to work with). Lastly I think all the data is currently housed in one geographic area but they are working on a DC in Phoenix. Overall a pretty smooth experience but I was mostly using it for cold data.
- voltagex_ 8y ago
  Have you reported this stuff to support? I found their support was great, even when dealing with me noodling around in C# and breaking things.
yani 8y ago
Only good if you are using it from the states.
haywirez 8y ago
I'm trying to use it as an S3 replacement for audio content delivery - seems slow and laggy unfortunately. Uploads also fail frequently enough. I don't know if CDNs would make a big difference. (Europe) Edit: lack of webhooks or something similar for doing follow-up after successful uploads is also irritating.

voltagex_ 8y ago

Are you able to try multithreaded uploads too? I found that single stream uploads were too slow (< 10 megabytes per second) but I could get ~35 megabytes per second from packet.net to B2 by using 4 threads. Edit: removed incorrect stuff.

thisacctforreal 8y ago
If you use the large_file API (needed for multithreaded uploads), you do only hash chunks not the file.
- voltagex_ 8y ago
  Okay, I'll have to go back and have a look at some of the client libraries I tried - it may have been the machine I was using wasn't quick enough to hash ~30GB in a reasonable amount of time.
- brianwski 8y ago
  Disclaimer: I work at Backblaze. > If you use the large_file API (needed for multithreaded uploads) We recommend for small files that you use multi-threaded where each thread sends a totally separate file. So if you have to upload both cat.jpg and dog.jpg, you upload cat.jpg in one thread and dog.jpg in another thread. Based on the Backblaze architecture, that means cat.jpg will be sent to one "vault" in the Backblaze datacenter with one thread, and dog.jpg will be sent to a totally different "vault" in the Backblaze datacenter with another thread. This scales incredibly well, in that it should be twice as fast for two files, and 20 times as fast for 20 files if you do it correctly. Source: I wrote a lot of the Backblaze Personal Backup client, which uses this philosophy.
gsibble 8y ago
I haven't added multi threaded stuff yet because I wanted it to be compatible with single threaded web servers like flask and django. I can and will add it if you want to add an issue.
- voltagex_ 8y ago

ereyes01 8y ago

I use rclone's B2 driver (https://rclone.org/b2/ https://rclone.org/b2/) as an rsync-style backup solution of about 1TB worth of my pictures and other media. Also, it's encrypted with my own local key using rclone's crypt module (https://rclone.org/crypt/ https://rclone.org/crypt/). rclone supports multithreaded upload, and even has experimental support for FUSE mounting. However, the sync command gets you Dropbox-like behavior and can be cronned: https://rclone.org/commands/rclone_sync/ https://rclone.org/commands/rclone_sync/ I really like the price of B2, I hope it stays low :-)

gordo4 8y ago
rclone is vulnerable to data exfiltration attacks https://www.danieldent.com/blog/restless-vulnerability-non-browser-cross-domain-http-request-attacks/ https://www.danieldent.com/blog/restless-vulnerability-non-b...
- Operyl 8y ago
  I quickly skimmed, but this entire attack is assuming that the attacker has successfully MITMed the API. At that point everything is already nuked, so of course you can fabricate any number of attacks. Did I miss something important?
- howeyc 8y ago
  Am I reading this right? Google/B2/... might send your data to another URL you didn't expect. Not sure why that matters, or why it's an attack. Since they have your data anyway, as that's the whole point of the service, to store your data on their hard drives. Why go through the trouble of sending it elsewhere? To play games with your data for giggles?
  - namibj 8y ago
    No, the API can tell your software to send some private LAN files, e.g. some IP-filtered secret NFS store, to an URL of it's choosing (so to itself, or your competitor). This is bad, as long as you don't heavily jail and firewall the software to prevent it from ever accessing anything it shouldn't (need to).

scottybowl 8y ago

Any ideas on using Backblaze as a driver in a Laravel project? We're currently using Digitalocean Spaces but it's very unreliable.

gsibble 8y ago
I believe Laravel is PHP so you'd need a PHP library while this one is Python.
- deleted 8y ago
  [deleted]
shakna 8y ago
amelia/backblaze [0] seems to be the highest quality, but there are a few around. [0] https://github.com/ameliaikeda/backblaze https://github.com/ameliaikeda/backblaze

logeek 8y ago

Is there a fundamental reason why B2 is (and will remain) cheaper than S3, or is it just because they need to compete with AWS and once successful the prices will be the same (or higher)?

agopaul 8y ago
Objects on S3 are replicated on 3 AZs (datacenters) by default and I can't find any info on B2 if they also replicate the data on multiple DCs. That can definitely change the cost per GB for them if that's the case.
- gsibble 8y ago
  B2 has 99.999999% durability
  - snvzz 8y ago
    That's just 8 9s. S3 has 11 9s.
    - gsibble 8y ago
      Depends upon your client SLA then. You can also store the file twice on B2 for still less than the cost of S3 :)
      - kccqzy 8y ago
        Will failures be correlated then?
      - snvzz 8y ago
        Then you should be comparing it to reduced redundancy S3, which is cheaper than regular S3.
  - scarface74 8y ago
    That’s nice in theory, but if B2 has a castrophic disaster in thier one data center, that durability goes to zero.

Dawny33 8y ago

The best part is this API looks and feels very similar to AWS's boto S3 API. Great job!

gsibble 8y ago
Thank you!

DanielDent 8y ago

B2's API design has security implications: https://www.danieldent.com/blog/restless-vulnerability-non-browser-cross-domain-http-request-attacks/ https://www.danieldent.com/blog/restless-vulnerability-non-b...

voltagex_ 8y ago
I don't see how B2 is affected by this unless you assume malicious control of a B2 API server.
- DanielDent 8y ago
  It's fairly standard for network clients to assume potential malicious control of the server they are connecting to. It helps reduce the blast radius of a compromised server. In the case where the server is operated by a third party (as is the case with the B2 API server), there can be many compliance implications if that third-party-operated server has access to an internal network. We don't accept when SSH clients or web browsers have the ability to do things they shouldn't based on instructions sent by the server they connect to. Why would we suddenly have lower expectations of our file storage API clients? (or any other network/HTTP clients for that matter)
  - voltagex_ 8y ago
    Ah, I see what you're getting at. It'd be better if the URL for get_upload_url (I think that's what the API was called) could be calculated client side. At the moment, you're probably still more at risk of downloading a malicious library from PyPi or npm but this is sure to turn up in a CTF at some point - even curl is technically vulnerable. Have you talked to anyone from Backblaze about this?
    - DanielDent 8y ago
      Yes, client-side URL calculation and/or a whitelist of acceptable URLs would be a significant improvement. Thankfully command-line curl won't follow redirects unless you pass it a special flag, though if you do need it to follow redirects, I'm not sure what the best way is to restrict the range of redirects that it will follow. This issue was part of a broader coordinated disclosure and was only published today. I've gotten in touch with B2 support & I'm hoping my support ticket will make it to the correct people.

krn 8y ago

Hetzner Storage Box[1] is an interesting alternative to Backblaze B2. It's not cloud-based, but provides free automated snapshots, free 1 Gbps bandwidth, and supports FTP, FTPS, SFTP, SCP, rsync and BorgBackup[2]. [1] https://www.hetzner.com/storage-box https://www.hetzner.com/storage-box [2] https://wiki.hetzner.de/index.php/Storage_Boxes/en https://wiki.hetzner.de/index.php/Storage_Boxes/en

hardwaresofton 8y ago
+1 for Hetzner, they have an amazing suite of services for excellent prices. Hetzner's robot marketplace[0] changed my (operations) life. [0]: https://robot.your-server.de/order/market https://robot.your-server.de/order/market
- apkallum 8y ago
  How did they do so?
  - hardwaresofton 8y ago
    tl;dr - Shifted my knowledge of the hosting market by exposing me to cheap dedicated servers I had previously thought that dedicated servers were doomed to be too expensive/heavy weight for me. I also felt like most VPS providers charged too much (especially true in the case of AWS -- $10/mo for a t2.micro is ridiculous). I first found INIZ (http://iniz.com/ http://iniz.com/) and was super happy with them, then someone introduced me to Hetzner Robot Marketplace and I was blown away by the affordable prices (+/- setup fee) and have had one ever since. Hetzner also has a cloud offering that is also pretty great -- slight limits on operating system choice and some other features and you can have very competitively priced machines in a more cloud-friendly fire-up-and-go format. I wrote about the revelation here (including link to HN thread where it happened): https://vadosware.io/post/fresh-dedicated-server-to-single-node-kubernetes-cluster-on-coreos/#why-change-my-infrastructure-at-all https://vadosware.io/post/fresh-dedicated-server-to-single-n... Now I have a ~6 Core (12 vCore/hyper-thread) 24GB RAM monster that I can run experiments with for a decent monthly price. If you go to other providers like Packet, OVH or Amazon, you're going to see way higher prices -- I'm don't have too many requirements so Hetzner worked for me.

Neil44 8y ago

Duplicity has support for B2 back end now, I use it for a lot of backup and it works well. Versioning, encryption etc.

gsibble 8y ago
This library is meant more for web apps and other applications built on Python. Not necessarily for backups.

simula67 8y ago

> Backblaze B2 provides the cheapest cloud object storage and transfer available on the internet Are you sure ? http://gaul.org/object-store-comparison/ http://gaul.org/object-store-comparison/ says there are cheaper options EDIT: Edited the link away from https://wasabi.com/pricing/ https://wasabi.com/pricing/. Wasabi seems cheaper than B2 and claims to be a hot storage solution

gsibble 8y ago
That does indeed look cheaper. I'll have to make an SDK for them next :D edit: Woah, you edited your comment. What made you change from Wasabi?
- simula67 8y ago
  Sorry, I was thinking about Wasabi when I was reading your post, but then I checked if there were even cheaper options
- qeternity 8y ago
  Wasabi has a 90 day minimum storage period, just like AWS Glacier. This means it's pretty unusable for things like nightly backups if your retention is less than 90 days.
  - koolba 8y ago
    If your backup process involves downloading the backup artifacts in a different region (say for true off site DR + validation) then it's still a net win as the 90-day storage costs are less than the insanely high $.09/GB AWS charges for outbound bandwidth.
qeternity 8y ago
We are big users of OVH, AWS and B2 object storage. OVH charge for ingress and egress even if local. AWS Glacier has 90 minimum storage time. For most uses cases, B2 is much much cheaper. EDIT: Response to your edit, Wasabi also has 90 day minimum storage policy.

gtt 8y ago

Can I use B2 as replacement for S3? Namely, I need "get upload URL" like pre-signed S3 URLs.

metakermit 8y ago

BTW, the Backblaze team themselves officially support a Python implementation of the API wrapper (the B2Api and Bucket classes): https://github.com/Backblaze/B2_Command_Line_Tool/blob/master/b2/api.py https://github.com/Backblaze/B2_Command_Line_Tool/blob/maste... As these are used internally in their CLI, there's probably a higher chance that they'll continue to work in the future.

gsibble 8y ago
Indeed. It's just never been publicized and personally I think the implementation could be better. Although with so many stars on this repo now I plan on maintaining it for the foreseeable future. Start adding feature requests everyone! :) PS: They also don't even use their own library in their code examples so I don't think they meant it to be used in that fashion.
- metakermit 8y ago
  That would be great – it's always good to have some competition ;) Regarding feature requests I'd love to see a well-maintained B2 Django Storage. I'm currently using an existing implementation, but it's not that well maintained: https://github.com/royendgel/django-backblazeb2-storage https://github.com/royendgel/django-backblazeb2-storage
  - gsibble 8y ago
    I saw that one. I'm not a particular fan of Django but integrating my library apart from Django's storage library wouldn't be difficult. Neither would be building a django library on top of mine. Any takers? :)

gsibble 8y ago

admins: Will you please change the title to a Show HN? Thank you!

sctb 8y ago
Sure, done!
- gsibble 8y ago
  Thanks!

Aissen 8y ago

According to online.net "cold storage" C14 comparison, they are cheaper than Backblaze, most of the time: https://www.online.net/en/c14 https://www.online.net/en/c14

gsibble 8y ago
B2 is warm storage much more like S3 while cold storage is like glacier.
qeternity 8y ago
C14 is really not at all an object store. Getting data in and out is a huge pain, even compared to other cold stores like AWS or OVH. We evaluated them and passed.
- Aissen 8y ago
  Thanks, I was looking for this type of feedback. The pricing comparison is still interesting. Could you do a summary of your evaluation for others that didn't test most services?
  - qeternity 8y ago
    Yeah, here's a quick stab: S3 - not much to say, fast, durable, expensive...the gold standard. Given limitations of below, we use for rotating nightly backups despite cost. Glacier - great for cold storage/archive, but has 90 day minimum OVH hot - open stack based, cheaper than S3 but not absurdly cheap, charged for egress even intra-DC which is absurd and kills many use cases. They have crippled OpenStack permission management (i.e. no write-only keys with lifetime management per bucket which is necessary for doing backups securely) OVH cold - charges for ingress but then storage is crazy cheap, and egress not as bad as Glacier. This is our preferred archival option. C14 - not object storage, more like a "cold" ftp dump B2 - pricing is epic, S3-incompatibility is a pain and lack of Backblaze-sponsored libraries (the library in the python b2 cli is not a proper API)...we've been working on adding B2 to WAL-E. However, their permission/user management doesn't cut it. Wasabi - S3 compatible, great pricing if not for 90 day minimum, which they hide in the fine print

piqufoh 8y ago

This feels like a handy tool! The first thing I read when opening new code is the test suite - it's worth getting that right at the start. Would you consider deeper unit testing? S3 (and aws) have the indispensible `moto` boto mocks, I think something similar would be dead handy here.

gsibble 8y ago
Yes, on my TODO list is much deeper unit testing. I made this in four days and was just testing that it worked. It already has about 92% code coverage but I want to cover that fields are returned properly and such. Some help would be appreciated if people would like to, including mocking it up.

weitzj 8y ago

It seems like nobody mentioned it, yet. Another great product is https://www.rsync.net/ https://www.rsync.net/ and this just works. There are no bad surprises. You can overshoot your backup limits, and they will send you an email to fix this. But still you have your backup. Your interface is rsync/scp/ssh. They give you ZFS snapshots, you can use s3cmd from their machines, so you can delegate uploads to S3 via rsync.net. Our prior backup setup was duplicity with GPG hitting S3, and this sometimes was flaky for listing the current keys. Glad I read HN, I heard about rsync.net. They even have/had a HN discount. You should use the search functionality to find other threads. https://www.rsync.net/products/platform.html https://www.rsync.net/products/platform.html https://arstechnica.com/information-technology/2015/12/rsync-net-zfs-replication-to-the-cloud-is-finally-here-and-its-fast/ https://arstechnica.com/information-technology/2015/12/rsync...

cahuk 8y ago
Speaking of ZFS, I use B2 with zfsbackup-go. Sanoid is making snapshots and zfsbackup-go is uploading them. [zfsbackup-go](https://github.com/someone1/zfsbackup-go https://github.com/someone1/zfsbackup-go)
alexktz 8y ago
Great product but too $$$$$$$$$$$$$ for my taste.
- anc84 8y ago
  It's almost like they try to be a self-sustaining, viable business instead of burning through VC money with irresponsible pricing that kills fair competitors.
  - Karunamon 8y ago
    The snark is really not necessary or contributing to the conversation. AWS Glacier, hardly a VC-backed startup, charges a literal tenth of the cost. Given that most people are going to be holding on to their backups rather than retrieving them regularly, the pricing math works out better even though it's a bit more complicated. Say you push 2TB up to Rsync, AWS Glacier, and Backblaze B2, and you need that data back a year later. Rsync will cost you $80x12: $960, bottom line. Glacier will cost you $8.00x12: $96 for the storage, plus .01 for a thousand retrieve requests, plus 0.01 per gigabyte retreival, plus 0.09 per gigabyte transfer. $96 + .01 + $20 + $180 = $296.10 Backblaze B2: $10x12 = $120 for the storage, plus 0.01 per gigabyte retrieved: $120 + $20 = $140 I'm guessing the "startup" dig was directed at Backblaze, but they're actually charging more for the plain storage than AWS, where you're paying more for the bandwidth!

erickj 8y ago

Hey that's cool. I made a B2 Ruby gem a few years back that gives a library and cli for the API. +1,000,000 for backblaze! https://github.com/erickj/bkblz https://github.com/erickj/bkblz

gsibble 8y ago
Awesome!

Sami_Lehtinen 8y ago

Is this the best way of dealing with Python 3 compatibility? What's wrong with sys.version_info? Or is there even some better way? Ref: https://github.com/sibblegp/b2blaze/blob/master/b2blaze/utilities.py https://github.com/sibblegp/b2blaze/blob/master/b2blaze/util...

artellectual 8y ago

That's funny, I did the same thing for elixir, feel free to check it out https://github.com/upmaru/upstream https://github.com/upmaru/upstream. Not well documented yet but it's coming.

scarface74 8y ago

I use Backblaze now and once I get my NAS, I’ll probably end up using a B2 based backup. But let’s make an honest comparison. Backblaze does not replicate your data across data centers. The standard S3 storage class does (0.23/gb). The comparible storage class for S3 is one zone infrequent access (.01/gb). B2 still comes out ahead, but I wouldn’t use either one for primary storage. For thier suggested “3-2-1” backup strategy, sure. Then again, just for backup, I could use S3 glacier for $.004/gb. That’s cheaper than B2 and I get multiple AZ storage. The data charges would be higher - but its backup. If catastrophe struck and I lost my primary and my local backups, getting my data fast is the last thing I would worry about.

deleted 8y ago
[deleted]
josteink 8y ago
> Then again, just for backup, I could use S3 glacier for $.004/gb Having done that in the past, I have to say that's just a million times less practical than basic S3-like storage. And if you want to automate that setup, Glacier is even worse.
- scarface74 8y ago
  Why do you say that? I could see using something like rsync + Cloudberry (maps S3 and make it look like a network drive). Set it up to use one zone infrequent access, and then after x days use a lifecycle policy to move it to Glacier. My use case for backups is solely for movies and music. For source code I use hosted git repos, pictures Google photos, and for regular office documents, they are either on Google docs or One Drive.
  - josteink 8y ago
    Last time I used Glacier, it was a separate product from S3 and had its own API. You had to upload pre-prepared "tapes" for backups. You couldn't mutate an existing backup, you had to create a new one. And frequently fetching and/or deleting existing "tapes" (backups) would cost you money (more so than the original cost of the backup). That meant you couldn't just ZIP it all up, backup the latest version and the delete the previous one to avoid being doubly charged for storage either. Basically at time of archiving you needed to determine what was already archived and create a new bundle with only what's new, and archive that only. In the same spirit, restore meant piecing together multiple such tapes into a full restore-set. Absolutely terrible. It was like having traditional backup-software constraints, but none of the software-support. If Amazon has improved on that now, good for them, but I figured they probably had to if they wanted any users at all.

tobias3 8y ago

This whole library wouldn't be necessary if Backbalze implemented a S3 compatible API. They give reasons like being able to load balance on the client for their API (which I do no think is a good reason), but ultimately they just push work from their end to a lot of applications and developers. Maybe it also has a strategic advantage? Now every product has to announce they support B2 whereas nobody has to announce they support Wasabi, because they support any S3 compatible storage such as AWS S3, Google Cloud Storage or Wasabi.

ianamartin 8y ago
Meh, I can see why they didn't. they aren't really in the same business. It makes sense for followers to implement APIs compatible with market leaders. Riak CS is API compatible with S3, which is nice. But it's literally intended to be an open source version of S3 that you can host and scale yourself. Backblaze is in a different market. They may be finding out that there's overlap and allowing that use. But they are not the same and probably aren't prepared for developers to start using b2 en masse. I think it makes business sense. You want to save some money? Do a little extra work for the cheaper product. Want to save even more money? Roll your own with Riak CS. Cloud services all work along the same spectrum where you pay more for convenience and ease of use, and you pay less up front if you're willing to pay in developer or devops or infrastructure costs. I think this fits in nicely on that spectrum.
gaul 8y ago
Object stores unfortunately innovate on their APIs instead of their implementations. I wrote S3Proxy to bridge the gap between S3 applications and a variety of object stores including B2: https://github.com/gaul/s3proxy https://github.com/gaul/s3proxy

brian_herman 8y ago

B2 does not have the durability guarantees that s3 has it is not a fair comparison.

iosDrone 8y ago

I thought the knock against backblaze is that (compared to Google or Amazon), uploading large amounts of data to it is very slow.

namibj 8y ago
If you follow the best practices, that is not an issue.
brianwski 8y ago
Disclaimer: I work for Backblaze so I'm biased. > uploading large amounts of data to [Backblaze B2] is very slow All reports we (Backblaze) hear is that if you only use one thread, Backblaze B2 is slightly slower than S3 (like maybe 90% of the performance). If somebody has better numbers I would LOVE to see them! If clients use multiple threads, this issue goes entirely away. Using 500 threads can provably be 500 times as fast with Backblaze. This is because the Backblaze B2 architecture means there are no "choke points" like Amazon S3 has. Each thread will most likely be talking to a completely different "Backblaze Vault" maybe even in a completely separate Backblaze datacenter. Since they don't share any network switches or load balancers in common, there is no way they will slow down. But again, I would love any measurements or reproducible tests showing differences so we can chase them down and improve Backblaze B2!
- iosDrone 8y ago
  Fair enough. This is just what I've heard second-hand, so I definitely don't have any data to back it up. I may try it out.

post_break 8y ago

I just wish Backblaze would fix their snapshots. You still have no way to tag a snapshot, put in any notes, anything. You literally make two snap shots 5 minutes apart and they only thing that differentiates them is the time stamp. unforgivable.

brianwski 8y ago
Disclaimer: I work at Backblaze. > You still have no way to tag a snapshot, put in any notes, anything. We totally agree, and the project is fully spec'ed, just waiting for an available engineer to implement it! On a side note, we also have open recs for engineers. :-)
- post_break 8y ago
  Thank you! Please send it out in an email or something when it's ready. We've been waiting for this for a while now which is why I'm so grumpy lol.

jonotime 8y ago

I'm loving B2 for my Linux desktop backup. I used Crashplan for many years until they pulled the plug recently. Now I'm using B2 via Duplicati and I'm actually saving money (I have about 500Gb of backup). Borg was a runner up, but Duplicati had built in B2 support, provided scheduling, and a web interface which makes navigating for specific files in a tree easy when needed. Crashplan had a nice Linux client, but it was blackbox/closed source, so problems came up from time to time that were hard to debug. So its nice to have more control of my data as well.

hinkley 8y ago

A little off topic. Did I miss the backblaze hard drive reliability report? I think it's been a while since they've done one.