4 ms·

On a similar but slightly different topic, I've long wondered why someone hasn't created a "Universal Terraform Provider" that can work across the major cloud s

On a similar but slightly different topic, I've long wondered why someone hasn't created a "Universal Terraform Provider" that can work across the major cloud services. Resources such as "instance" and network are pretty general, an while there would definitely be gotchas, it would probably be helpful in the same way that cross platform GUI frameworks have seen a rise in popularity.

MuffinFlavored 4y ago
On a similar but slightly different topic, I used to think Terraform was the be-all end-all until I realized you don't manage Kubernetes with it. To my knowledge, https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs https://registry.terraform.io/providers/hashicorp/kubernetes... less people use this than use Helm charts, Kustomize, or some other Kubernetes-.yaml formatter Then I get confused on where Vagrant provisioning fits into all of this. Or Ansible/Chef/Puppet. I just spin up a DigitalOcean droplet (I think I usually do it in the GUI to be honest, lol). Then I SSH into it, git clone a repo, docker compose up or (sometimes terraform apply) and I'm done. Where does it grow from there? I feel like based on the comments here, some people would even argue Docker is overkill there, or anything that isn't a monolith (small services glued together) is overkill.
- joseph 4y ago
  I do manage Kubernetes with Terraform. I'm not a fan of Helm, however it is useful due to the community of available charts. Since I use Terraform to manage most everything else, I typically wrap the chart in a Terraform module too. This lets me do other things I usually need, such as creating IAM roles or other associated resources that the chart requires. Sometimes this means using the Kubernetes provider to add something needed by the chart, for example pulling a password from SSM or somewhere and creating a Kubernetes Secret from it. The benefit is being able to build full stacks with only Terraform without needing to orchestrate multiple tools together. Building the cluster itself is done with Terraform, as is deploying the charts and other resources I need to build a "base system", all from a single root module. This also simplifies CI/CD as I only need a simple template to run IaC jobs and they all follow the same pattern of just running Terraform and not much else.
  - MuffinFlavored 4y ago
    Could you show me an example somewhere on Github on how you do Kubernetes resources with Terraform? To be clear, there is no YAML that you are writing? I found this: https://github.com/hashicorp/learn-terraform-helm/blob/main/helm_release.tf https://github.com/hashicorp/learn-terraform-helm/blob/main/...
    - joseph 4y ago
      I made a small example at https://gist.github.com/rjosephwright/f1485cddcc12dd651ec96558ac7601c6 https://gist.github.com/rjosephwright/f1485cddcc12dd651ec965.... There is no YAML, I use HCL for values as they are easier to validate and less error prone, then convert them with yamlencode().
      - MuffinFlavored 4y ago
        Thank you. What are the advantages to this over a light shell script that uses `kubectl + helm` CLI tools in your opinion? You can just `terraform apply` and that's it? My concern is it adds a "third" technology/abstraction layer (Terraform) to another underlying 2 (Helm + Kubernetes, which are also "abstraction layers" in themself, right?)
        joseph 4y ago
        For me there is a benefit in having a single abstraction layer, the Terraform root module, to make IaC changes, and specifically one that is in the form of data rather than a script. The data incidentally contains everything required for reproducibility (all dependency versions are pinned, etc), and it always calls only versioned modules that are published independently. There is one Terraform wrapper script that runs from CI/CD and it is the only script, no matter what kind of change is being made, Kubernetes or otherwise. Teaching someone how to make a change is easy, they just need to make a change to the data and apply it. If the data layer doesn't support the necessary change, then work can be done to add the capability to one of the modules and then publish a new version. For simple needs this might be overkill, but in my environment it helps to keep things from getting out of hand. Helm and Kubernetes are abstraction layers, but very specific ones, whereas Terraform is a generic one that ties it all together.
redeux 4y ago
While it’s true that the cloud providers offer the same primitives, storage/network/compute, their implementations are different enough that it matters when you’re building any complex system. A Terraform provider that attempted to provide an abstraction layer over the major could services would end up having to either encode assumptions about how the user wants their infrastructure or it would have to be overly verbose in its configuration. In practice users want neither of those. Terraform also aim to create a single workflow for all clouds, rather than provide a common abstraction over the clouds. So, it’s a combination of that’s not what the tool was designed to do, and that’s not what people want in practice, given the current state of the major clouds.
jerf 4y ago
If you want to take the time to deeply understand the answer to your question, grab one of those resources and set the Azure, Google, and AWS versions to each other. The VM instance equivalent is a reasonable starting point. The S3 equivalent isn't too bad either. Start lining up their similarities. Figure out what you would do to harmonize them. At least glance at the transitive dependencies, like networking... you don't need to fully work them out but you need to at least think about how that would be harmonized too. If you do this honestly, and with some experience of having to tweak this semi-obscure setting and fiddle with that semi-obscure setting a few times, and you look at them honestly and realize that this is one of those "Microsoft Word" cases ("everyone uses 10% of Microsoft Word but no one uses the same 10%"), and all those settings are there for a reason and there isn't hardly one you can just drop without consequence, you'll understand. At a 10,000 foot view it looks incredibly easy. When you actually try, you discover the lowest common denominator is so low that it would barely function. I'm pretty sure you can put an instance on "the internet" and probably load it with a common OS (though your harmonization layer is probably already doing some modestly heavy lifting just trying to have a coherent view of OS distributions) and get it to serve something, probably get some very simple port blocking done, but you're going to start being in serious trouble the moment you need more than that. The details will explode in your face, repeatedly. But, I don't just mean this rhetorically. I'm serious. If you're really interested in this, this is a good exercise and you will learn something from the experience worth the time. Good for tuning the intuition.
BaconPackets 4y ago
There is already a couple of options https://github.com/multycloud/multy https://github.com/multycloud/multy But a multicloud wrapper creates a lot of abstractions.

MuffinFlavored 4y ago

On a similar but slightly different topic, I used to think Terraform was the be-all end-all until I realized you don't manage Kubernetes with it. To my knowledge, https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs https://registry.terraform.io/providers/hashicorp/kubernetes... less people use this than use Helm charts, Kustomize, or some other Kubernetes-.yaml formatter Then I get confused on where Vagrant provisioning fits into all of this. Or Ansible/Chef/Puppet. I just spin up a DigitalOcean droplet (I think I usually do it in the GUI to be honest, lol). Then I SSH into it, git clone a repo, docker compose up or (sometimes terraform apply) and I'm done. Where does it grow from there? I feel like based on the comments here, some people would even argue Docker is overkill there, or anything that isn't a monolith (small services glued together) is overkill.

joseph 4y ago
I do manage Kubernetes with Terraform. I'm not a fan of Helm, however it is useful due to the community of available charts. Since I use Terraform to manage most everything else, I typically wrap the chart in a Terraform module too. This lets me do other things I usually need, such as creating IAM roles or other associated resources that the chart requires. Sometimes this means using the Kubernetes provider to add something needed by the chart, for example pulling a password from SSM or somewhere and creating a Kubernetes Secret from it. The benefit is being able to build full stacks with only Terraform without needing to orchestrate multiple tools together. Building the cluster itself is done with Terraform, as is deploying the charts and other resources I need to build a "base system", all from a single root module. This also simplifies CI/CD as I only need a simple template to run IaC jobs and they all follow the same pattern of just running Terraform and not much else.
- MuffinFlavored 4y ago
  Could you show me an example somewhere on Github on how you do Kubernetes resources with Terraform? To be clear, there is no YAML that you are writing? I found this: https://github.com/hashicorp/learn-terraform-helm/blob/main/helm_release.tf https://github.com/hashicorp/learn-terraform-helm/blob/main/...
  - joseph 4y ago
    I made a small example at https://gist.github.com/rjosephwright/f1485cddcc12dd651ec96558ac7601c6 https://gist.github.com/rjosephwright/f1485cddcc12dd651ec965.... There is no YAML, I use HCL for values as they are easier to validate and less error prone, then convert them with yamlencode().
    - MuffinFlavored 4y ago
      Thank you. What are the advantages to this over a light shell script that uses `kubectl + helm` CLI tools in your opinion? You can just `terraform apply` and that's it? My concern is it adds a "third" technology/abstraction layer (Terraform) to another underlying 2 (Helm + Kubernetes, which are also "abstraction layers" in themself, right?)
      - joseph 4y ago
        For me there is a benefit in having a single abstraction layer, the Terraform root module, to make IaC changes, and specifically one that is in the form of data rather than a script. The data incidentally contains everything required for reproducibility (all dependency versions are pinned, etc), and it always calls only versioned modules that are published independently. There is one Terraform wrapper script that runs from CI/CD and it is the only script, no matter what kind of change is being made, Kubernetes or otherwise. Teaching someone how to make a change is easy, they just need to make a change to the data and apply it. If the data layer doesn't support the necessary change, then work can be done to add the capability to one of the modules and then publish a new version. For simple needs this might be overkill, but in my environment it helps to keep things from getting out of hand. Helm and Kubernetes are abstraction layers, but very specific ones, whereas Terraform is a generic one that ties it all together.

redeux 4y ago

While it’s true that the cloud providers offer the same primitives, storage/network/compute, their implementations are different enough that it matters when you’re building any complex system. A Terraform provider that attempted to provide an abstraction layer over the major could services would end up having to either encode assumptions about how the user wants their infrastructure or it would have to be overly verbose in its configuration. In practice users want neither of those. Terraform also aim to create a single workflow for all clouds, rather than provide a common abstraction over the clouds. So, it’s a combination of that’s not what the tool was designed to do, and that’s not what people want in practice, given the current state of the major clouds.

jerf 4y ago

If you want to take the time to deeply understand the answer to your question, grab one of those resources and set the Azure, Google, and AWS versions to each other. The VM instance equivalent is a reasonable starting point. The S3 equivalent isn't too bad either. Start lining up their similarities. Figure out what you would do to harmonize them. At least glance at the transitive dependencies, like networking... you don't need to fully work them out but you need to at least think about how that would be harmonized too. If you do this honestly, and with some experience of having to tweak this semi-obscure setting and fiddle with that semi-obscure setting a few times, and you look at them honestly and realize that this is one of those "Microsoft Word" cases ("everyone uses 10% of Microsoft Word but no one uses the same 10%"), and all those settings are there for a reason and there isn't hardly one you can just drop without consequence, you'll understand. At a 10,000 foot view it looks incredibly easy. When you actually try, you discover the lowest common denominator is so low that it would barely function. I'm pretty sure you can put an instance on "the internet" and probably load it with a common OS (though your harmonization layer is probably already doing some modestly heavy lifting just trying to have a coherent view of OS distributions) and get it to serve something, probably get some very simple port blocking done, but you're going to start being in serious trouble the moment you need more than that. The details will explode in your face, repeatedly. But, I don't just mean this rhetorically. I'm serious. If you're really interested in this, this is a good exercise and you will learn something from the experience worth the time. Good for tuning the intuition.

BaconPackets 4y ago

There is already a couple of options https://github.com/multycloud/multy https://github.com/multycloud/multy But a multicloud wrapper creates a lot of abstractions.