6 ms·

Last I checked, Backblaze still stores most data in 1 location, no? So, durability of data (which to be fair doesn't matter for most s3 use cases), and interop

by cheeze 7y ago

Last I checked, Backblaze still stores most data in 1 location, no? So, durability of data (which to be fair doesn't matter for most s3 use cases), and interop with literally everything else in AWS Intelligent data tiering Actual access control Pre signed URLs

chocolatkey 7y ago
I've combined cloudflare workers with backblaze to implement etags, signed URLs, etc. Backblaze is part of CF's bandwidth alliance so your bandwidth fee is zero. This makes for a very low monthly cost
- SergeAx 7y ago
  Can you elaborate further about this setup? Is there an article or a FAQ topic about it?
  - atYevP 7y ago
    Yev from Backblaze here -> the Nodecraft folks did a great job with that project -> https://jross.me/free-personal-image-hosting-with-backblaze-b2-and-cloudflare-workers/ https://jross.me/free-personal-image-hosting-with-backblaze-...
    - SergeAx 7y ago
      Hi, Yev! Thanks a lot. I am fancying the idea to move our CDN from AWS stack to B2 + CF, thanks to Bandwidth Alliance. There's at least one thing stopping me: for simple schema of hosting static content out of bucket we should deploy Workers just for URL rewriting. Guys from CF recommending that way and not URL rewrites by simple rules[1]. But it puts us in a weak position of raising cost twice: for increased edge trafic AND for increased number of requests. Can anything be done on BackBlaze side to address the problem, like custom domains for buckets? Like https://f001.backblazeb2.com/file/bucket-name/file.jpg https://f001.backblazeb2.com/file/bucket-name/file.jpg => https://bucket-name.f001.backblazeb2.com/file.jpg https://bucket-name.f001.backblazeb2.com/file.jpg ? [1] https://community.cloudflare.com/t/page-rule-setting-request-rewrite/45491/4 https://community.cloudflare.com/t/page-rule-setting-request...
  - chocolatkey 7y ago
    Hi, I haven't had time to write up about this, however, I have dumped the majority of the related code here for you and others who are interested in this solution: https://gist.github.com/chocolatkey/a7ef0364e357629e9875521ddc8bc03a https://gist.github.com/chocolatkey/a7ef0364e357629e9875521d.... That should help you get started. It includes HMACSHA256 shared secret URL signatures based on IP, expiry, and optional path scope restriction, caching, ETAGs, sentry error reports, access to non-B2 data from a server w/ basic auth, and more... URLs look like this: https://example.com/delivery/UNIQUE_ID/p-001.jpg?token=16fb4112744c20a1f71f2730072f9457612a90ce00cdb6a1ccdf639e4b31966e&expires=5e450200&scope=%2Fdelivery%2FUNIQUE_ID%2F https://example.com/delivery/UNIQUE_ID/p-001.jpg?token=16fb4... . My B2 bucket is public, however the requested path is also hmac'd with a secret known only to the CF worker to derive the path of the resources in the bucket. It is optimized for my use case of serving EPUB data. I do not guarantee it to be free of flaws, but it's worked well so far.
    - SergeAx 7y ago
      Thanks a lot!
shiftpgdn 7y ago
S3 is single zone unless you specifically request multi zone durability (most don't.)
- kevinchen 7y ago
  I don’t think this is accurate. Within a single region, S3 stores in multiple availability zones by default. https://aws.amazon.com/s3/faqs/ https://aws.amazon.com/s3/faqs/
  - shiftpgdn 7y ago
    You're completely right. I guess I was working on old info.
brianwski 7y ago
Disclaimer: I work at Backblaze. > Last I checked, Backblaze still stores most data in 1 location, no? Backblaze now has multiple regions! One in Europe (Netherlands) and one is called "US-West". Quietly the US-West is actually three separate data centers, but your data will only really land in 1 datacenter somewhere in US-West based on a few internal factors. To be absolutely clear, if you only upload and store and pay for 1 copy of your Backblaze B2 data, it is living in one region. To get a copy in two locations you have to pay twice as much and take some actions. So if this kind of redundancy is important to you for mission critical reasons Backblaze B2 would only be half as expensive as one copy in Amazon S3, not 1/4 as expensive. In the one copy in one region in Backblaze B2, any file is "sharded" across 20 different servers in 20 different racks in 20 different locations inside that datacenter. This helps insulate against failures like if one rack loses power (like if a power strip goes bad or a circuit breaker blows). But if a meteor hits that 1 datacenter and wipes out all of the equipment in a 1 mile blast radius, you won't be getting that data back unless you have a backup somewhere else.

by cheeze 7y ago

chocolatkey 7y ago

I've combined cloudflare workers with backblaze to implement etags, signed URLs, etc. Backblaze is part of CF's bandwidth alliance so your bandwidth fee is zero. This makes for a very low monthly cost

SergeAx 7y ago
Can you elaborate further about this setup? Is there an article or a FAQ topic about it?
- atYevP 7y ago
  Yev from Backblaze here -> the Nodecraft folks did a great job with that project -> https://jross.me/free-personal-image-hosting-with-backblaze-b2-and-cloudflare-workers/ https://jross.me/free-personal-image-hosting-with-backblaze-...
  - SergeAx 7y ago
    Hi, Yev! Thanks a lot. I am fancying the idea to move our CDN from AWS stack to B2 + CF, thanks to Bandwidth Alliance. There's at least one thing stopping me: for simple schema of hosting static content out of bucket we should deploy Workers just for URL rewriting. Guys from CF recommending that way and not URL rewrites by simple rules[1]. But it puts us in a weak position of raising cost twice: for increased edge trafic AND for increased number of requests. Can anything be done on BackBlaze side to address the problem, like custom domains for buckets? Like https://f001.backblazeb2.com/file/bucket-name/file.jpg https://f001.backblazeb2.com/file/bucket-name/file.jpg => https://bucket-name.f001.backblazeb2.com/file.jpg https://bucket-name.f001.backblazeb2.com/file.jpg ? [1] https://community.cloudflare.com/t/page-rule-setting-request-rewrite/45491/4 https://community.cloudflare.com/t/page-rule-setting-request...
- chocolatkey 7y ago
  Hi, I haven't had time to write up about this, however, I have dumped the majority of the related code here for you and others who are interested in this solution: https://gist.github.com/chocolatkey/a7ef0364e357629e9875521ddc8bc03a https://gist.github.com/chocolatkey/a7ef0364e357629e9875521d.... That should help you get started. It includes HMACSHA256 shared secret URL signatures based on IP, expiry, and optional path scope restriction, caching, ETAGs, sentry error reports, access to non-B2 data from a server w/ basic auth, and more... URLs look like this: https://example.com/delivery/UNIQUE_ID/p-001.jpg?token=16fb4112744c20a1f71f2730072f9457612a90ce00cdb6a1ccdf639e4b31966e&expires=5e450200&scope=%2Fdelivery%2FUNIQUE_ID%2F https://example.com/delivery/UNIQUE_ID/p-001.jpg?token=16fb4... . My B2 bucket is public, however the requested path is also hmac'd with a secret known only to the CF worker to derive the path of the resources in the bucket. It is optimized for my use case of serving EPUB data. I do not guarantee it to be free of flaws, but it's worked well so far.
  - SergeAx 7y ago
    Thanks a lot!

shiftpgdn 7y ago

S3 is single zone unless you specifically request multi zone durability (most don't.)

kevinchen 7y ago
I don’t think this is accurate. Within a single region, S3 stores in multiple availability zones by default. https://aws.amazon.com/s3/faqs/ https://aws.amazon.com/s3/faqs/
- shiftpgdn 7y ago
  You're completely right. I guess I was working on old info.

brianwski 7y ago

Disclaimer: I work at Backblaze. > Last I checked, Backblaze still stores most data in 1 location, no? Backblaze now has multiple regions! One in Europe (Netherlands) and one is called "US-West". Quietly the US-West is actually three separate data centers, but your data will only really land in 1 datacenter somewhere in US-West based on a few internal factors. To be absolutely clear, if you only upload and store and pay for 1 copy of your Backblaze B2 data, it is living in one region. To get a copy in two locations you have to pay twice as much and take some actions. So if this kind of redundancy is important to you for mission critical reasons Backblaze B2 would only be half as expensive as one copy in Amazon S3, not 1/4 as expensive. In the one copy in one region in Backblaze B2, any file is "sharded" across 20 different servers in 20 different racks in 20 different locations inside that datacenter. This helps insulate against failures like if one rack loses power (like if a power strip goes bad or a circuit breaker blows). But if a meteor hits that 1 datacenter and wipes out all of the equipment in a 1 mile blast radius, you won't be getting that data back unless you have a backup somewhere else.