5 ms·

this is great, but I would argue the biggest issue with infrastructure as code is this: the structure and syntax for AWS is entirely different from Azure is en

this is great, but I would argue the biggest issue with infrastructure as code is this: the structure and syntax for AWS is entirely different from Azure is entirely different from GCP. Instead of abstracting to CSS, I would argue modeling what Bytecode did in java for multi-operating system, we should do for infrastructure of code. That way, you could easily replicate in different environments, free yourself from vendor lock, and have readability/re-usability all in one. This is what I want from infrastructure as code and I have yet to see it.

firesteelrain 3y ago
You can do that through abstraction. You “include” your Terraform Azure Provider or Terraform AWS Provider. At the end of the day, your module needs to know what it’s interacting with but not the higher level of abstraction. We have done it at my work to make it cloud agnostic just in case we need to go to another CSP
- firesteelrain 3y ago
  Meaning we made our own wrapper for the terraform azure provider. The higher level does not expose those technical details. It just says virtual machine.
- drewcoo 3y ago
  That's a lot like peppering your C code with #ifdef/#ifndef to write code that's "platform independent." Just like in the bad old days.
  - firesteelrain 3y ago
    No, it’s object oriented like
- mattpallissard 3y ago
  This hould be done even if you don't care about being agnostic. Lay out your declarations in the format you'd like to interact with, then transform them to whatever they need to be.
- datahead 3y ago
  How did you approach testing this? Anything to watch out for when defining abstractions? I'm following Crossplane's abstraction model.
  - firesteelrain 3y ago
    So we start with terraform fmt -check, terraform validate, and terraform plan. Contract tests are used Then we have test resources that get created, validated then instantly destroyed. If we don’t have access to the CSP, then we can only go as far as contract tests. I can’t integrate with it live such as GCP. Starting to look into Terratest.
- m_0x 3y ago
  > We have done it at my work to make it cloud agnostic just in case we need to go to another CSP Have you actually tested migrating Cloud Providers with your cloud-agnostic infrastructure?
  - firesteelrain 3y ago
    Yes we tested it with Azure Stack Hub and AWS Outposts
jerf 3y ago
"This is what I want from infrastructure as code and I have yet to see it." If you sit down with the terraform specifications for an AWS instance, a GCP instance, and an Azure instance, and start trying to write that harmonization, you will rapidly discover why for yourself. Even just trying to specify a network setup and putting an instance on the public internet is impossible to harmonize, without making something so lowest-common-denominator it is almost useless, let alone anything complicated.
- whoomp12342 3y ago
  I never said it was easy. I would expect complicated things to suffer, just as bytecode binary is not as efficient as a c++ binary.
- substation13 3y ago
  Exactly. If you want portability across clouds, Terraform ain't it. The only way I know of to achieve that right now (for any reasonably complex architecture) is: - Minimal amount of Terraform to deploy Kubernetes (which is different for each cloud provider) - Helm (or similar) for deploying to Kubernetes But then you have Kubernets to deal with.
drewcoo 3y ago
> he structure and syntax for AWS is entirely different from Azure is entirely different from GCP This! We're essentially writing locked-in vendorscript. What we want is an actual infrastructure language. One that lets us write once and deploy anywhere (nods to Java). That would also allow us to standardize the way additional tooling (monitoring, logging, etc.) hooks into everything. It would allow us to easily deploy to new environment types as they become available (I keep hearing about the wonders of WASM). It would allow standardized ways of doing ops testing.
- mike_hearn 3y ago
  You could also just use Java (or Kotlin Scripting, which is a bit more ergonomic for such use cases). I know it's super unpopular in these parts but a lot of what the article asks for is satisfied by conventional statically typed OOP (with inheritance), and other features common in full blown programming languages. You don't have to actually do the setup imperatively, but you can construct the data structures describing what you want with such tools and let some engine bring your real infrastructure into compliance. That's why the author is experimenting with TypeScript. Obviously, full languages allow you to eliminate repetition in many different ways. Java interfaces with default methods are very close to 'traits', Kotlin has the same features with real properties and convenient syntaxes. It also supports builder DSLs which would be ideal for this. The main reason to use such languages is that you get full IDE features out of the box, for instance, "show me everything that uses this trait/interface/abstract base class" is one click away. Refactoring is easy. But there are other reasons. You will often want to define some behavior that can be composed, but which leaves "slots" that the user has to fill out. Other times you need to say "something just like that, but adjusted a bit". OOP type systems are good at this sort of thing. The DevOps world seems to continually go through cycles where someone says, full languages are too powerful, we need a declarative subset! And then it's too limited and leads to too much repetition, so you start getting stuff like if statements encoded into YAML and other nonsense. And then people get dissatisfied and try to invent another minimal declarative markup, and the cycle repeats.
spc476 3y ago
You don't want vendor lock-in. The vendor, however, has different incentives---why would they want to prevent lock-in to their service?
goncalo-r 3y ago
This is what Multy (https://github.com/multycloud/multy https://github.com/multycloud/multy) aims to do

firesteelrain 3y ago

You can do that through abstraction. You “include” your Terraform Azure Provider or Terraform AWS Provider. At the end of the day, your module needs to know what it’s interacting with but not the higher level of abstraction. We have done it at my work to make it cloud agnostic just in case we need to go to another CSP

firesteelrain 3y ago
Meaning we made our own wrapper for the terraform azure provider. The higher level does not expose those technical details. It just says virtual machine.
drewcoo 3y ago
That's a lot like peppering your C code with #ifdef/#ifndef to write code that's "platform independent." Just like in the bad old days.
- firesteelrain 3y ago
  No, it’s object oriented like
mattpallissard 3y ago
This hould be done even if you don't care about being agnostic. Lay out your declarations in the format you'd like to interact with, then transform them to whatever they need to be.
datahead 3y ago
How did you approach testing this? Anything to watch out for when defining abstractions? I'm following Crossplane's abstraction model.
- firesteelrain 3y ago
  So we start with terraform fmt -check, terraform validate, and terraform plan. Contract tests are used Then we have test resources that get created, validated then instantly destroyed. If we don’t have access to the CSP, then we can only go as far as contract tests. I can’t integrate with it live such as GCP. Starting to look into Terratest.
m_0x 3y ago
> We have done it at my work to make it cloud agnostic just in case we need to go to another CSP Have you actually tested migrating Cloud Providers with your cloud-agnostic infrastructure?
- firesteelrain 3y ago
  Yes we tested it with Azure Stack Hub and AWS Outposts

jerf 3y ago

"This is what I want from infrastructure as code and I have yet to see it." If you sit down with the terraform specifications for an AWS instance, a GCP instance, and an Azure instance, and start trying to write that harmonization, you will rapidly discover why for yourself. Even just trying to specify a network setup and putting an instance on the public internet is impossible to harmonize, without making something so lowest-common-denominator it is almost useless, let alone anything complicated.

whoomp12342 3y ago
I never said it was easy. I would expect complicated things to suffer, just as bytecode binary is not as efficient as a c++ binary.
substation13 3y ago
Exactly. If you want portability across clouds, Terraform ain't it. The only way I know of to achieve that right now (for any reasonably complex architecture) is: - Minimal amount of Terraform to deploy Kubernetes (which is different for each cloud provider) - Helm (or similar) for deploying to Kubernetes But then you have Kubernets to deal with.

drewcoo 3y ago

> he structure and syntax for AWS is entirely different from Azure is entirely different from GCP This! We're essentially writing locked-in vendorscript. What we want is an actual infrastructure language. One that lets us write once and deploy anywhere (nods to Java). That would also allow us to standardize the way additional tooling (monitoring, logging, etc.) hooks into everything. It would allow us to easily deploy to new environment types as they become available (I keep hearing about the wonders of WASM). It would allow standardized ways of doing ops testing.

mike_hearn 3y ago
You could also just use Java (or Kotlin Scripting, which is a bit more ergonomic for such use cases). I know it's super unpopular in these parts but a lot of what the article asks for is satisfied by conventional statically typed OOP (with inheritance), and other features common in full blown programming languages. You don't have to actually do the setup imperatively, but you can construct the data structures describing what you want with such tools and let some engine bring your real infrastructure into compliance. That's why the author is experimenting with TypeScript. Obviously, full languages allow you to eliminate repetition in many different ways. Java interfaces with default methods are very close to 'traits', Kotlin has the same features with real properties and convenient syntaxes. It also supports builder DSLs which would be ideal for this. The main reason to use such languages is that you get full IDE features out of the box, for instance, "show me everything that uses this trait/interface/abstract base class" is one click away. Refactoring is easy. But there are other reasons. You will often want to define some behavior that can be composed, but which leaves "slots" that the user has to fill out. Other times you need to say "something just like that, but adjusted a bit". OOP type systems are good at this sort of thing. The DevOps world seems to continually go through cycles where someone says, full languages are too powerful, we need a declarative subset! And then it's too limited and leads to too much repetition, so you start getting stuff like if statements encoded into YAML and other nonsense. And then people get dissatisfied and try to invent another minimal declarative markup, and the cycle repeats.

spc476 3y ago

You don't want vendor lock-in. The vendor, however, has different incentives---why would they want to prevent lock-in to their service?

goncalo-r 3y ago

This is what Multy (https://github.com/multycloud/multy https://github.com/multycloud/multy) aims to do