12 ms·
Backblaze is now a Terraform provider
- majormjr 6y agoI was just looking at this new Terraform provider yesterday how timely. Nice to see the quickstart guide this will be helpful for managing the buckets and application keys.
- atYevP 6y agoYev from Backblaze here -> That's great! Glad the quick-start makes it simple.
- vbsteven 6y agoTerraform is an amazing tool and it’s always great to see more infrastructure providers jumping on the bandwagon.
- atYevP 6y agoYev from Backblaze here -> Yea, it's pretty great! They're a great tool and we're glad to be involved!
- deleted 6y ago[deleted]
- OJFord 6y agoBut unfortunately plugins cannot provide remote backends, and it's a little clunky to use the S3 one: https://github.com/hashicorp/terraform/issues/27304 https://github.com/hashicorp/terraform/issues/27304
- javajosh 6y agoIt's interesting because it feels like a small turf war. The "Amazon Provider Team" wants to own the s3 provider backend, and you want to make it pluggable. Is this to avoid an explicit "b2" provider? I can see how it might be confusing to use terraform and see an s3 provider and then have it...not use s3. :)
- ArchOversight 6y agoTerraform doesn't allow any backends other than the ones built-in. So even if you wanted to provide a "b2" backend, it'd have to be built into to terraform. Using the S3 backend with B2 should work fine since B2 makes a S3 compatible API available, but its made more difficult because the "Amazon Provider Team" is the one that maintains the "s3" backend for Terraform, and they want to do additional validation that matches AWS's expectations.
- renewiltord 6y agoVery cool! Great job, guys. Do you use a similar technique to https://poweruser.blog/embedding-python-in-go-338c0399f3d5 https://poweruser.blog/embedding-python-in-go-338c0399f3d5 to embed the Python SDK? Also, was there a reason it's not against the B2 API? Not a judgment, just curious about the design tradeoffs in a professional-talking-to-professional sense.
- AYBABTME 6y agoThat design choice prevents their provider from working with official Terraform container images. They should make the API calls directly in Go, it's just weird to do it via Python... They have very few resources exposed on their API so it's not like writing a Go client would be all that hard.
- renewiltord 6y agoWhy would the official container images not work? The Python interpreter and libs are bundled into the (massively fat) Go plugin binary, right? And appropriately linked?
- mattjaynes 6y agoIf you'll be using B2 for live production data, a couple points: First, B2's pricing is pretty amazing, especially compared to S3 and similar competitors: https://www.backblaze.com/b2/cloud-storage-pricing.html https://www.backblaze.com/b2/cloud-storage-pricing.html Second, be aware those savings come with some downsides. The major one for us has been their maintenance window every Thursday from 2:00-3:00pm Pacific Time. Usually there's no outage, but sometimes there is. There's no warning - it's just down sometimes during that window. So, if uptime is important for your data, consider the cost of also implementing a fallback solution to cover your production use during those maintenance windows. https://www.backblaze.com/scheduled-maintenance.html https://www.backblaze.com/scheduled-maintenance.html
- tedmiston 6y agoWhat a strange time for a company with lots of US customers to schedule a maintenance window.
- Twisol 6y agoI'm not so sure. It's better than a late night window in that it forces their customers to actually deal with the engineering window, rather than just cross their fingers.
- aequitas 6y agoAlso, your office is full of awake engineers at that time. Which is better than a handful of on-call sleepy engineers and crossing your fingers the rest of the team wakes up when you call them if something goes completely haywire.
- johnjboren 6y agoDon't push to production on a Friday, but Thursday at the latest, sure...
- ericbarrett 6y ago
- nikisweeting 6y agoI love B2's free ingress and egress when you use CloudFlare, they utterly destroy the competition on price. But my true dream would be for backblaze to someday offer ZFS as a service. I want to `zfs send -i my_pool@2020-03-11 | b2zfs recv some_bucket_id`, then be able to view my snapshots and files within in the backblaze web UI, and restore with `b2zfs send -i some_bucket_id/my_pool@2020-03-11 | zfs recv my_pool`. You can already mount B2 as a FUSE filesystem with something like ExpanDrive, then write ZFS raw file vdevs to the B2 FUSE mount, but it's horrifically slow and probably too janky for any real use.
- ptomato 6y agorsync.net's zfs as a service is really great for that, though a bit more expensive than b2 is.
- bpye 6y agoRsync.net looks great but the storage cost is 5x B2 unfortunately EDIT: As mentioned below this is for the most expensive (lowest capacity) tier. I’d been comparing for my own home use and so I would be unlikely to exceed 10TB but if you’re looking at higher capacity then maybe the calculus is different.
- rsync 6y ago... for small quantities, yes. For large quantities, it is 3x, and actually less since there are no charges for ingress/egress. Further questions/comments over email, please, since this is BBs HN thread and I don't want to butt in.
- kgog 6y agomaybe you need borg with borgbase.
- mhio 6y agoWithout b2 adding anything zfs specific, that should be possible with a regular large file upload if you can deal with each upload part being buffered in a file/memory locally before upload.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- rattray 6y agoI sure wish this post had included a few code snippets, just to make it extra-clear what using Backblaze through Terraform might look like.
- vbsteven 6y agoInside the article is a section "How to get started using Backblaze B2 in Terraform" with a link [1] to the getting started guide which should have everything you need. I can see why they put it on a separate page to not clutter up the article. [1] https://help.backblaze.com/hc/en-us/articles/1260803375989 https://help.backblaze.com/hc/en-us/articles/1260803375989
- rattray 6y agoNice, thanks! I was less interested in the code sample as a "how-to" and more for skimmability – rather than reading a bunch of words I just wanted to see what it would look like. Eg, from that link: terraform { required_version = ">= 0.13" required_providers { b2 = { source = "Backblaze/b2" version = "~> 0.2" } } } provider "b2" { } resource "b2_application_key" "example" { key_name = "test-b2-tfp-0000000000000000000" capabilities = ["readFiles"] } data "b2_application_key" "example" { key_name = b2_application_key.example.key_name } output "application_key" { value = data.b2_application_key.example }
- snicker7 6y agoTerraform is great, but I wish more cloud providers had Guix/Nix integration.
- CameronNemo 6y agoAre there any providers that to have such integration? I have only lightly used nix. Having a hard time understanding what a cloud provider integration would look like.
- ethnt 6y agoYou could use NixOps[0] for Nix but I'm not sure you can directly compare Terraform and Guix/Nix? My set up involves Terraform for infrastructure and Nix for provisioning, and it's working for me so far. [0] https://github.com/NixOS/nixops https://github.com/NixOS/nixops
- emptysongglass 6y agoYou can deploy and manage NixOS on Big Cloud using Terraform: https://nix.dev/tutorials/deploying-nixos-using-terraform.html https://nix.dev/tutorials/deploying-nixos-using-terraform.ht...
- aseipp 6y agoI think this too, but at the same time the reality is a tool like Terraform is really complicated to implement well, and, importantly, it always has to work. All the time. There are really high standards for this, and in my personal experience, alternative solutions like NixOps don't quite stack up in reliability or broad utility versus Terraform. The design is good in theory, but it needs just a huge amount of work to be trusted. For the most part, I provision things with Terraform and then instantiate the servers with NixOS/Nix itself, and this mostly works. For bonus points you can use Nix to generate the HCL that Terraform reads in (because Nix can write JSON, and HCL is just JSON in a trenchcoat) if you want to put some veneer on it.
- 1vuio0pswjnm7 6y agoFor large downloads, does BB support the Range header. If the user is on a connection that that is not suitable for long downloads, could the Range header be used to download a large file in several parts.
- 0xbkt 6y agoThey support byte serving.
- deleted 6y ago[deleted]
- satyrnein 6y agoAnyone have any experience with the Backblaze compute partners? I want to put data up on B2, but then I still need to do things with it.
- philsnow 6y agoThe wording is a little clunky; the "is" in "Backblaze is now a Terraform provider" makes me think that BB is competing with Terraform Cloud. "Backblaze now has a Terraform provider" or "Backblaze released a Terraform provider" makes more sense to me.
- solatic 6y ago> Terraform is an open-source infrastructure as code (IaC) tool Maybe it was in the beginning, but Terraform is far more powerful than that now. Terraform is a monad that neatly separates pure declarative configuration from the I/O (side effects) that are factored out into providers. Terraform used at its most powerful is not limited to infrastructure, it also sets up the platforms and applications running on that infrastructure for you, by separating the configuration of the platforms and applications from the generic API calls that apply that configuration. Terraform's dependency graph ensures that the calls are made in the right order, no matter if they are made to infrastructure APIs, platform APIs, or APIs belonging to layers further up the stack
- nhoughto 6y agoPopped in to say backblaze built out a tf provider at our request and they were great about it! Got a quick early build out to test, GA build a few months later and are very responsive to feedback. Pleasure to deal with
- atYevP 6y agoYev from Backblaze here -> That's really nice of you, glad we could work together!