19 ms·

If you'll be using B2 for live production data, a couple points: First, B2's pricing is pretty amazing, especially compared to S3 and similar competitors: http

If you'll be using B2 for live production data, a couple points: First, B2's pricing is pretty amazing, especially compared to S3 and similar competitors: https://www.backblaze.com/b2/cloud-storage-pricing.html https://www.backblaze.com/b2/cloud-storage-pricing.html Second, be aware those savings come with some downsides. The major one for us has been their maintenance window every Thursday from 2:00-3:00pm Pacific Time. Usually there's no outage, but sometimes there is. There's no warning - it's just down sometimes during that window. So, if uptime is important for your data, consider the cost of also implementing a fallback solution to cover your production use during those maintenance windows. https://www.backblaze.com/scheduled-maintenance.html https://www.backblaze.com/scheduled-maintenance.html

tedmiston 6y ago
What a strange time for a company with lots of US customers to schedule a maintenance window.
- Twisol 6y ago
  I'm not so sure. It's better than a late night window in that it forces their customers to actually deal with the engineering window, rather than just cross their fingers.
  - aequitas 6y ago
    Also, your office is full of awake engineers at that time. Which is better than a handful of on-call sleepy engineers and crossing your fingers the rest of the team wakes up when you call them if something goes completely haywire.
- johnjboren 6y ago
  Don't push to production on a Friday, but Thursday at the latest, sure...
  - ericbarrett 6y ago
    Thursday is the best maintenance day. If it goes haywire, you have a full business day to fix before losing a weekend. And if you can't, well, you've given your coworkers a "free Friday," which is far less likely to result in complaints than screwing up M-Th.
    - lupire 6y ago
      What's a free Friday?
      - ericbarrett 6y ago
        Ever go into work on Friday and "the system is down"? You can't get anything done, because the tools you use to do your job aren't working, and the fix is out of your hands. Your coworkers are all affected, too. First people are frustrated, because they have tasks deadlines, and those tasks won't get done and the deadlines won't be met. A few are really freaked out and start calling bosses and getting VPs to yell at other VPs. But soon, most people in the company realize that everybody else is in the same boat, and nobody will be meeting their OKRs this week, and the status reports probably won't be filed. Then they relax. If they're in offices, they group up, maybe in the break room. A rousing game of ping-pong breaks out. Remote coworkers ping each other on Slack. Maybe a few start a round of Among Us. Bread dough is kneaded. Kids get a little more help with their schoolwork. Everybody takes a very long lunch. By 2pm, people realize the entire day is gone. Almost everybody has left or signed off by 3. Some roll out to bars; others go home to their kids, or to their gardens or garages or battlestations. Everybody beats the traffic. Come Monday, the system is fixed. People are a little stressed out, since there's so much catching-up to do, status reports to be filed, widgets to be tracked and poked. But everybody agrees that was an amazing couple of days, and they got lots of rest, and it sure was nice. And hey, I had this great idea over the weekend—
        monsieurbanana 6y ago
        Sometimes I miss working in big companies.
- bretpiatt 6y ago
  With the B2 target use case of backup and archive data I suspect it's actually a good time to do it for their customer base (and it also then happens to be good for their engineering team too, awake and alert!).
truetraveller 6y ago
Ouch. Does B2 Cloud actually go down during this time. That's a major show-stopper. Was actually counting on B2. Not sure anymore.
- mattjaynes 6y ago
  Not often, I only remember a few times in the last couple of years. If you're just hosting backups, you're unlikely to even notice. But if you're serving live production data from B2, it can bring down your whole service, which is quite painful especially if you have a large customer base.
- atYevP 6y ago
  Yev from Backblaze here -> typically not - we've built most of our systems to be keep data up and flowing during those maintenance windows - if we anticipate longer windows or are doing things that can impact performance we typically announce it on our blog and twitter!
  - ericvanular 6y ago
    Thanks for building Backblaze. If you're able to share some feedback with the team - this is an extremely important factor. If you can ensure downtime is avoided during maintenance windows, it will make your service much more viable for production systems
    - CameronNemo 6y ago
      Can anyone be 100% sure that downtime is avoided during maintenance windows?
    - xchaotic 6y ago
      If you have some umm other useful tips for that 100% uptime let the rest of the world know. I am more happy with realistic and upfront statements from B2 than some wishful thinking from potential users.
    - barkingcat 6y ago
truetraveller 6y ago
Side question: Apart from this maintenance window, is B2 Cloud reliable? I've heard of problems with the S3 API. Is the "native" API more stable? Would love to know your insight, it will potentially save me a lot of time!
- mattjaynes 6y ago
  Have had live production data on there for a couple of years and it's been very solid outside the maintenance windows. (The exception being the recent outages GoDaddy caused for them, but since they've moved to using Cloudflare as their registrar, I don't anticipate further issues there: https://news.ycombinator.com/item?id=26119619 https://news.ycombinator.com/item?id=26119619 )
  - truetraveller 6y ago
    Okay, that's great to hear.
- tedivm 6y ago
  Several years ago Backblaze lost all of my wife's data. Their dashboard said it was all there, and we trusted their systems to be accurate. When attempting to download the data it turned out that none of it was there. When my wife contacted support they tried to blame her. Obviously this was a few years ago, but a backup provider failing at their one job and then blaming the customer left a really bad impression that keeps me from using them.
  - jiofih 6y ago
    You were probably a victim of their 30 day deletion policy. If for any reason (firewall, etc) you did not connect to the backup servers your data would just be purged without a grace period. For that reason I built my own backup sync using B2 directly instead of their backup service (and it’s a lot cheaper).
    - tedivm 6y ago
preommr 6y ago
Digital Ocean has spaces with unlimited uploads, 250gB + (0.02/gb), 1TB of outbound transfer all at $5/mo. That seems way cheaper than this. For backups and large, long term storage, AWS has Glacier, that's really really cheap.
- jjeaff 6y ago
  $0.02/gb compared to $0.005 with b2. So not really in the same ballpark as far as price.
  - preommr 6y ago
    That's just the storage cost per GB, if you have any kind of serious throughput, then bandwith costs make it more expensive.
    - brightball 6y ago
      Aren't they in that free bandwidth alliance with Cloudflare?
      - divbzero 6y ago
        Yes, it looks like Backblaze is part of that free bandwidth alliance. [1] Thank you, I was not aware of this policy. [1]: https://www.cloudflare.com/bandwidth-alliance/backblaze/ https://www.cloudflare.com/bandwidth-alliance/backblaze/
        preommr 6y ago
        Yea, that changes things, that's a pretty good deal then. Especially because Cloudflare's pricing is "smoother" and detached from any one service.
      - atYevP 6y ago
        Yev from Backblaze here -> yes we are!
qw3rty01 6y ago
Also keep in mind all the data is in a single datacenter, so if something like OVH happens to backblaze, you'll probably lose the data I mainly only use them for backups instead of production data EDIT: As of 2019 there are multiple datacenters, but it doesn't seem like the data is stored redundantly across them
- jiofih 6y ago
  They have four datacenters, three in the US and one in the EU. Details are not given regarding how the 20 shards that comprise your data are distributed geographically, but they state eight 9s of reliability.
  - qw3rty01 6y ago
    Oh how new is that? Last time I was looking through they just said their datacenter was in a bunker so nothing outside of a major natural disaster would affect it.
    - jiofih 6y ago
      Since 2019 it seems: https://help.backblaze.com/hc/en-us/articles/217667468-B2-Security-and-Redundancy?mobile_site=true https://help.backblaze.com/hc/en-us/articles/217667468-B2-Se...
  - spondyl 6y ago
    Just to clarify, Backblaze states eight 9s of durability, not reliability. Durability refers to the idea that your data will still be retrievable (ie no corruption) similar to S3's claim of eleven 9s of durability. Reliability however would be say; the actual availability of the Web UI or API server that you download your data through. If it were down, that wouldn't impact the actual integrity of the data itself. For anyone interested, any more than about five 9s of reliability is basically impossible anyway when it comes to human intervention. As an example, 6 nines of reliability would allow you 3 seconds of unavailability a year so 250 milliseconds of unavailability a month. From a users point of view, being "unavailable" includes everything from going through a tunnel and having your mobile connection drop out to a shark biting one of the undersea cables in the middle of the ocean. As you might imagine, with a human involved, they couldn't even get acknowledge an alert fast enough to meet that deadline let alone actually going about doing any repairs and diagnosis :) It could be spread over multiple instances and redundant hardware as well but as with any system being touched by humans, it's near guaranteed that something will go wrong eventually.
vishnumohandas 6y ago
B2 claims to offer 99.9% uptime[1], and seems to have SLAs[2] similar to that of AWS S3[3]. [1]: https://www.backblaze.com/blog/cloud-storage-durability-vs-availability/ https://www.backblaze.com/blog/cloud-storage-durability-vs-a... [2]: https://www.backblaze.com/company/sla.html https://www.backblaze.com/company/sla.html [3]: https://aws.amazon.com/s3/sla/ https://aws.amazon.com/s3/sla/
- mattjaynes 6y ago
  In practical terms, 99.9% uptime means no more than ~44 minutes per month of downtime (~9 hours per year).
busymom0 6y ago
A limitation I ran across when using B2 was that their pre-defined url generation doesn't allow you to set file-size limits nor does it allow you to set the file name in the pre-defined url. It simply gives you a url to upload it to. So if you are using b2 for storage for lets say image uploads from browser, some malicious user has the ability to modify the network request with whatever file name or file size they want. Next thing you know, you have a 5gb sized image uploads happening.... This pretty much prevents me from using B2 for now.
- SmellTheGlove 6y ago
  I ran into the same limitation! IIRC, there also wasn't a way to expire a signed upload URL sooner than whatever the default was, which was hours or maybe a day. I had the exact use case you mentioned, too - image uploads bypassing my backend server. I didn't want the generation of a signed url to, say, upload a profile photo, give carte blanche to create a hidden image host when combined with the limitation that you highlighted. All sorts of bad things could come of that. I ended up just going back to S3 - costs more, but still worth it.
fukmbas 6y ago
Lol who has a maintenance window in the middle of the day?

tedmiston 6y ago

What a strange time for a company with lots of US customers to schedule a maintenance window.

Twisol 6y ago
I'm not so sure. It's better than a late night window in that it forces their customers to actually deal with the engineering window, rather than just cross their fingers.
- aequitas 6y ago
  Also, your office is full of awake engineers at that time. Which is better than a handful of on-call sleepy engineers and crossing your fingers the rest of the team wakes up when you call them if something goes completely haywire.
johnjboren 6y ago
Don't push to production on a Friday, but Thursday at the latest, sure...
- ericbarrett 6y ago
  Thursday is the best maintenance day. If it goes haywire, you have a full business day to fix before losing a weekend. And if you can't, well, you've given your coworkers a "free Friday," which is far less likely to result in complaints than screwing up M-Th.
  - lupire 6y ago
    What's a free Friday?
    - ericbarrett 6y ago
      Ever go into work on Friday and "the system is down"? You can't get anything done, because the tools you use to do your job aren't working, and the fix is out of your hands. Your coworkers are all affected, too. First people are frustrated, because they have tasks deadlines, and those tasks won't get done and the deadlines won't be met. A few are really freaked out and start calling bosses and getting VPs to yell at other VPs. But soon, most people in the company realize that everybody else is in the same boat, and nobody will be meeting their OKRs this week, and the status reports probably won't be filed. Then they relax. If they're in offices, they group up, maybe in the break room. A rousing game of ping-pong breaks out. Remote coworkers ping each other on Slack. Maybe a few start a round of Among Us. Bread dough is kneaded. Kids get a little more help with their schoolwork. Everybody takes a very long lunch. By 2pm, people realize the entire day is gone. Almost everybody has left or signed off by 3. Some roll out to bars; others go home to their kids, or to their gardens or garages or battlestations. Everybody beats the traffic. Come Monday, the system is fixed. People are a little stressed out, since there's so much catching-up to do, status reports to be filed, widgets to be tracked and poked. But everybody agrees that was an amazing couple of days, and they got lots of rest, and it sure was nice. And hey, I had this great idea over the weekend—
      - monsieurbanana 6y ago
        Sometimes I miss working in big companies.
bretpiatt 6y ago
With the B2 target use case of backup and archive data I suspect it's actually a good time to do it for their customer base (and it also then happens to be good for their engineering team too, awake and alert!).

truetraveller 6y ago

Ouch. Does B2 Cloud actually go down during this time. That's a major show-stopper. Was actually counting on B2. Not sure anymore.

mattjaynes 6y ago
Not often, I only remember a few times in the last couple of years. If you're just hosting backups, you're unlikely to even notice. But if you're serving live production data from B2, it can bring down your whole service, which is quite painful especially if you have a large customer base.
atYevP 6y ago
Yev from Backblaze here -> typically not - we've built most of our systems to be keep data up and flowing during those maintenance windows - if we anticipate longer windows or are doing things that can impact performance we typically announce it on our blog and twitter!
- ericvanular 6y ago
  Thanks for building Backblaze. If you're able to share some feedback with the team - this is an extremely important factor. If you can ensure downtime is avoided during maintenance windows, it will make your service much more viable for production systems
  - CameronNemo 6y ago
    Can anyone be 100% sure that downtime is avoided during maintenance windows?
  - xchaotic 6y ago
    If you have some umm other useful tips for that 100% uptime let the rest of the world know. I am more happy with realistic and upfront statements from B2 than some wishful thinking from potential users.
  - barkingcat 6y ago

truetraveller 6y ago

Side question: Apart from this maintenance window, is B2 Cloud reliable? I've heard of problems with the S3 API. Is the "native" API more stable? Would love to know your insight, it will potentially save me a lot of time!

mattjaynes 6y ago
Have had live production data on there for a couple of years and it's been very solid outside the maintenance windows. (The exception being the recent outages GoDaddy caused for them, but since they've moved to using Cloudflare as their registrar, I don't anticipate further issues there: https://news.ycombinator.com/item?id=26119619 https://news.ycombinator.com/item?id=26119619 )
- truetraveller 6y ago
  Okay, that's great to hear.
tedivm 6y ago
Several years ago Backblaze lost all of my wife's data. Their dashboard said it was all there, and we trusted their systems to be accurate. When attempting to download the data it turned out that none of it was there. When my wife contacted support they tried to blame her. Obviously this was a few years ago, but a backup provider failing at their one job and then blaming the customer left a really bad impression that keeps me from using them.
- jiofih 6y ago
  You were probably a victim of their 30 day deletion policy. If for any reason (firewall, etc) you did not connect to the backup servers your data would just be purged without a grace period. For that reason I built my own backup sync using B2 directly instead of their backup service (and it’s a lot cheaper).
  - tedivm 6y ago

preommr 6y ago

Digital Ocean has spaces with unlimited uploads, 250gB + (0.02/gb), 1TB of outbound transfer all at $5/mo. That seems way cheaper than this. For backups and large, long term storage, AWS has Glacier, that's really really cheap.

jjeaff 6y ago
$0.02/gb compared to $0.005 with b2. So not really in the same ballpark as far as price.
- preommr 6y ago
  That's just the storage cost per GB, if you have any kind of serious throughput, then bandwith costs make it more expensive.
  - brightball 6y ago
    Aren't they in that free bandwidth alliance with Cloudflare?
    - divbzero 6y ago
      Yes, it looks like Backblaze is part of that free bandwidth alliance. [1] Thank you, I was not aware of this policy. [1]: https://www.cloudflare.com/bandwidth-alliance/backblaze/ https://www.cloudflare.com/bandwidth-alliance/backblaze/
      - preommr 6y ago
        Yea, that changes things, that's a pretty good deal then. Especially because Cloudflare's pricing is "smoother" and detached from any one service.
    - atYevP 6y ago
      Yev from Backblaze here -> yes we are!

qw3rty01 6y ago

Also keep in mind all the data is in a single datacenter, so if something like OVH happens to backblaze, you'll probably lose the data I mainly only use them for backups instead of production data EDIT: As of 2019 there are multiple datacenters, but it doesn't seem like the data is stored redundantly across them

jiofih 6y ago
They have four datacenters, three in the US and one in the EU. Details are not given regarding how the 20 shards that comprise your data are distributed geographically, but they state eight 9s of reliability.
- qw3rty01 6y ago
  Oh how new is that? Last time I was looking through they just said their datacenter was in a bunker so nothing outside of a major natural disaster would affect it.
  - jiofih 6y ago
    Since 2019 it seems: https://help.backblaze.com/hc/en-us/articles/217667468-B2-Security-and-Redundancy?mobile_site=true https://help.backblaze.com/hc/en-us/articles/217667468-B2-Se...
- spondyl 6y ago
  Just to clarify, Backblaze states eight 9s of durability, not reliability. Durability refers to the idea that your data will still be retrievable (ie no corruption) similar to S3's claim of eleven 9s of durability. Reliability however would be say; the actual availability of the Web UI or API server that you download your data through. If it were down, that wouldn't impact the actual integrity of the data itself. For anyone interested, any more than about five 9s of reliability is basically impossible anyway when it comes to human intervention. As an example, 6 nines of reliability would allow you 3 seconds of unavailability a year so 250 milliseconds of unavailability a month. From a users point of view, being "unavailable" includes everything from going through a tunnel and having your mobile connection drop out to a shark biting one of the undersea cables in the middle of the ocean. As you might imagine, with a human involved, they couldn't even get acknowledge an alert fast enough to meet that deadline let alone actually going about doing any repairs and diagnosis :) It could be spread over multiple instances and redundant hardware as well but as with any system being touched by humans, it's near guaranteed that something will go wrong eventually.

vishnumohandas 6y ago

B2 claims to offer 99.9% uptime[1], and seems to have SLAs[2] similar to that of AWS S3[3]. [1]: https://www.backblaze.com/blog/cloud-storage-durability-vs-availability/ https://www.backblaze.com/blog/cloud-storage-durability-vs-a... [2]: https://www.backblaze.com/company/sla.html https://www.backblaze.com/company/sla.html [3]: https://aws.amazon.com/s3/sla/ https://aws.amazon.com/s3/sla/

mattjaynes 6y ago
In practical terms, 99.9% uptime means no more than ~44 minutes per month of downtime (~9 hours per year).

busymom0 6y ago

A limitation I ran across when using B2 was that their pre-defined url generation doesn't allow you to set file-size limits nor does it allow you to set the file name in the pre-defined url. It simply gives you a url to upload it to. So if you are using b2 for storage for lets say image uploads from browser, some malicious user has the ability to modify the network request with whatever file name or file size they want. Next thing you know, you have a 5gb sized image uploads happening.... This pretty much prevents me from using B2 for now.

SmellTheGlove 6y ago
I ran into the same limitation! IIRC, there also wasn't a way to expire a signed upload URL sooner than whatever the default was, which was hours or maybe a day. I had the exact use case you mentioned, too - image uploads bypassing my backend server. I didn't want the generation of a signed url to, say, upload a profile photo, give carte blanche to create a hidden image host when combined with the limitation that you highlighted. All sorts of bad things could come of that. I ended up just going back to S3 - costs more, but still worth it.

fukmbas 6y ago

Lol who has a maintenance window in the middle of the day?