7 ms·

This exists because terraform DSL, yaml, and Json are not programming languages and are thus tragic choices for making declarative statements

BaronSamedi 6y ago
After many years working with these tools I agree. Declarative and template DSLs are OK for basic things but you quickly bump into their limitations when you want to do advanced configurations. At this point administrators should know how to code so being able to use a normal programming language with IaC modules should be the standard. Amazon's CDK for AWS is a step in the right direction. Microsoft's new Bicep for Azure misses the mark in my view because it is yet another DSL and not a real programming language.
hardwaresofton 6y ago
I have been saying this for a while now -- as much as I respect Hashicorp and admire their engineering, HCL is the biggest problem with the tools they have that it's used in -- biggest of which being Terraform. Don't use a DSL where a full programming language is what you need. I can see that they probably wanted to be able to skip writing 2/3 language CDKs out of the box, and DSLs probably hit on the need for simplicity really early, but infrastructure code is somewhere you really want to be able to drop into the full expressive power of a programming language. I use Pulumi[0] for my projects, and while I doubt people will be able to get it as much adoption as I think it should have in corporate environments, Terraform has introduced the CDK[1] which is a similar approach. Have a talk with your engineers about complexity (and how not to strangle the rest of the team with it) and stop using DSLs in places you need more expressive power. I think a good indication of when you need to stop using DSLs or when you should maybe realize a DSL wasn't the right way to go is when you start doing things like writing control flow. You could can get away with DSLs if you were using s-expressions, but that is an exception that proves the rule (because there happens to a family of languages that treat s-expressions as syntax). [0]: https://www.pulumi.com https://www.pulumi.com [1]: https://www.hashicorp.com/blog/cdk-for-terraform-enabling-python-and-typescript-support https://www.hashicorp.com/blog/cdk-for-terraform-enabling-py...
- cyberpunk 6y ago
  I’m super curious about pulumi — I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway, is it mature enough for a reasonably standard aws/eks setup?
  - hardwaresofton 6y ago
    As much as I like Pulumi, no one gets fired for choosing Terraform. You're going to be able to find the most help, discussion, and stuff for Terraform. Terraform and it's ecosystem is the safer choice, especially when the CDK is an option. On the other hand I can say that a bunch of Pulumi's stuff is built on terraform underneath the covers so I'm not sure how far behind they are but it probably isn't by much. In my limited use (for example deploying SES stuff) I found it a pleasure to use and didn't find anything that terraform did that it didn't (again, likely due to pulumi being able to utilize terraform providers under the covers). > I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway Hope you're really sure this is where you want to use your effort tokens -- I'm not sure how much of a mess with it, but these tools are so new, it might be worth seeing if you can de-clutter it without a complete rewrite. Or maybe it's small enough that a complete rewrite is relatively low friction... Either way, there be dragons.
- busterarm 6y ago
  The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. Terraform should merely provision things and in that role I find it fine as is. Preferred, even.
  - hardwaresofton 6y ago
    > The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. Declarative infrastructure shouldn't be pursued for it's own sake -- what I want is efficient, and simple to manage infrastructure automation. The declarative nature is awesome, but once you start doing plumbing of variables and complexity from one static script from another, the cognitive load of keeping this all in line is better managed with a programming language in my opinion, you're just choosing bash/jq/awk/etc instead of a different language. I think "the way the declarations are made must be static files" is dogmatic or at least limiting for me. Yes it is absolutely the simplest way to view what's present, but the problem is when someone goes into change any of this they will be dealing with your bolted-together complexity (even if it's not very complex). > I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. > Terraform should merely provision things and in that role I find it fine as is. Preferred, even. I can't argue with the efficiency and efficacy of your setup, but I don't think much of this has to do with what we were discussing -- Pulumi does not seek to do the jobs of those other tools -- it's not going to build your VM images or do provisioning (unless you use it that way like with terraform[0]). Here's a concrete example of a benefit I got form using Pulumi over terraform recently, in some code working with SES: import * as fs from "fs"; // ... more imports and other lines // Email Access key const emailAccessKey = new aws.iam.AccessKey( `${stack}-ses-access-key`, {user: emailUser.name} ); export const emailUserSMTPPassword = emailAccessKey.sesSmtpPasswordV4; export const emailUserSecret = emailAccessKey.encryptedSecret; // Write the smtp username and password out to a local secret file const apiSecretsDir = path.join(__dirname, "secrets", "api", stack); const smtpUsernameFilePath = path.resolve(path.join(apiSecretsDir, "SES_USERNAME.secret")); const smtpPasswordFilePath = path.resolve(path.join(apiSecretsDir, "SES_PASSWORD.secret")); emailAccessKey.sesSmtpPasswordV4.apply(password => { console.log(`Writing SES SMTP username to [${smtpUsernameFilePath}]`); fs.writeFileSync(smtpUsernameFilePath, emailUsername); console.log(`Writing SES SMTP password to [${smtpPasswordFilePath}]`); fs.writeFileSync(smtpPasswordFilePath, password); }); I wanted to write information out to a file... So I just did, and that was it. No need to reach for the stack output later and pipe it anywhere -- any time pulumi runs it will update that variable if/when it changes, and the next tool (which requires the file at that path to be present) will continue on without knowing a thing. I can't say that this is perfect Pulumi code (ex. I could have defined a custom Resource to do this for me), but I have saved myself having to do the plumbing with bash scripts and terraform output awk-ing, and the information goes just where I want it (NOTE: the secrets folder is encrypted with git-crypt[1]). When someone comes to this file (ses.ts), they're going to be able to easily trace where these values where generated -- similar with bash scripts, but now they don't have to be a bash/awk/jq master to manipulate information. There are definitely some gotchas to using Pulumi (like the `.apply` there), but in the end, I'd prefer to make changes like this in a consistent language I like (Typescript). My toolkit looks very similar to you, except I basically only use make + kubectl + pulumi + ansible (rarely, because of the kind of servers I rent). [0]: https://www.terraform.io/docs/provisioners/ https://www.terraform.io/docs/provisioners/ [1]: https://www.agwa.name/projects/git-crypt/ https://www.agwa.name/projects/git-crypt/
zerubeus 6y ago
I love you
eat_veggies 6y ago
It seems like Nix and NixOps are designed for this type of thing, since they're built on a functional language instead of yaml. But I haven't played with them too much yet -- anyone have experience using them?

BaronSamedi 6y ago

After many years working with these tools I agree. Declarative and template DSLs are OK for basic things but you quickly bump into their limitations when you want to do advanced configurations. At this point administrators should know how to code so being able to use a normal programming language with IaC modules should be the standard. Amazon's CDK for AWS is a step in the right direction. Microsoft's new Bicep for Azure misses the mark in my view because it is yet another DSL and not a real programming language.

hardwaresofton 6y ago

I have been saying this for a while now -- as much as I respect Hashicorp and admire their engineering, HCL is the biggest problem with the tools they have that it's used in -- biggest of which being Terraform. Don't use a DSL where a full programming language is what you need. I can see that they probably wanted to be able to skip writing 2/3 language CDKs out of the box, and DSLs probably hit on the need for simplicity really early, but infrastructure code is somewhere you really want to be able to drop into the full expressive power of a programming language. I use Pulumi[0] for my projects, and while I doubt people will be able to get it as much adoption as I think it should have in corporate environments, Terraform has introduced the CDK[1] which is a similar approach. Have a talk with your engineers about complexity (and how not to strangle the rest of the team with it) and stop using DSLs in places you need more expressive power. I think a good indication of when you need to stop using DSLs or when you should maybe realize a DSL wasn't the right way to go is when you start doing things like writing control flow. You could can get away with DSLs if you were using s-expressions, but that is an exception that proves the rule (because there happens to a family of languages that treat s-expressions as syntax). [0]: https://www.pulumi.com https://www.pulumi.com [1]: https://www.hashicorp.com/blog/cdk-for-terraform-enabling-python-and-typescript-support https://www.hashicorp.com/blog/cdk-for-terraform-enabling-py...

cyberpunk 6y ago
I’m super curious about pulumi — I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway, is it mature enough for a reasonably standard aws/eks setup?
- hardwaresofton 6y ago
  As much as I like Pulumi, no one gets fired for choosing Terraform. You're going to be able to find the most help, discussion, and stuff for Terraform. Terraform and it's ecosystem is the safer choice, especially when the CDK is an option. On the other hand I can say that a bunch of Pulumi's stuff is built on terraform underneath the covers so I'm not sure how far behind they are but it probably isn't by much. In my limited use (for example deploying SES stuff) I found it a pleasure to use and didn't find anything that terraform did that it didn't (again, likely due to pulumi being able to utilize terraform providers under the covers). > I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway Hope you're really sure this is where you want to use your effort tokens -- I'm not sure how much of a mess with it, but these tools are so new, it might be worth seeing if you can de-clutter it without a complete rewrite. Or maybe it's small enough that a complete rewrite is relatively low friction... Either way, there be dragons.
busterarm 6y ago
The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. Terraform should merely provision things and in that role I find it fine as is. Preferred, even.
- hardwaresofton 6y ago
  > The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. Declarative infrastructure shouldn't be pursued for it's own sake -- what I want is efficient, and simple to manage infrastructure automation. The declarative nature is awesome, but once you start doing plumbing of variables and complexity from one static script from another, the cognitive load of keeping this all in line is better managed with a programming language in my opinion, you're just choosing bash/jq/awk/etc instead of a different language. I think "the way the declarations are made must be static files" is dogmatic or at least limiting for me. Yes it is absolutely the simplest way to view what's present, but the problem is when someone goes into change any of this they will be dealing with your bolted-together complexity (even if it's not very complex). > I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. > Terraform should merely provision things and in that role I find it fine as is. Preferred, even. I can't argue with the efficiency and efficacy of your setup, but I don't think much of this has to do with what we were discussing -- Pulumi does not seek to do the jobs of those other tools -- it's not going to build your VM images or do provisioning (unless you use it that way like with terraform[0]). Here's a concrete example of a benefit I got form using Pulumi over terraform recently, in some code working with SES: import * as fs from "fs"; // ... more imports and other lines // Email Access key const emailAccessKey = new aws.iam.AccessKey( `${stack}-ses-access-key`, {user: emailUser.name} ); export const emailUserSMTPPassword = emailAccessKey.sesSmtpPasswordV4; export const emailUserSecret = emailAccessKey.encryptedSecret; // Write the smtp username and password out to a local secret file const apiSecretsDir = path.join(__dirname, "secrets", "api", stack); const smtpUsernameFilePath = path.resolve(path.join(apiSecretsDir, "SES_USERNAME.secret")); const smtpPasswordFilePath = path.resolve(path.join(apiSecretsDir, "SES_PASSWORD.secret")); emailAccessKey.sesSmtpPasswordV4.apply(password => { console.log(`Writing SES SMTP username to [${smtpUsernameFilePath}]`); fs.writeFileSync(smtpUsernameFilePath, emailUsername); console.log(`Writing SES SMTP password to [${smtpPasswordFilePath}]`); fs.writeFileSync(smtpPasswordFilePath, password); }); I wanted to write information out to a file... So I just did, and that was it. No need to reach for the stack output later and pipe it anywhere -- any time pulumi runs it will update that variable if/when it changes, and the next tool (which requires the file at that path to be present) will continue on without knowing a thing. I can't say that this is perfect Pulumi code (ex. I could have defined a custom Resource to do this for me), but I have saved myself having to do the plumbing with bash scripts and terraform output awk-ing, and the information goes just where I want it (NOTE: the secrets folder is encrypted with git-crypt[1]). When someone comes to this file (ses.ts), they're going to be able to easily trace where these values where generated -- similar with bash scripts, but now they don't have to be a bash/awk/jq master to manipulate information. There are definitely some gotchas to using Pulumi (like the `.apply` there), but in the end, I'd prefer to make changes like this in a consistent language I like (Typescript). My toolkit looks very similar to you, except I basically only use make + kubectl + pulumi + ansible (rarely, because of the kind of servers I rent). [0]: https://www.terraform.io/docs/provisioners/ https://www.terraform.io/docs/provisioners/ [1]: https://www.agwa.name/projects/git-crypt/ https://www.agwa.name/projects/git-crypt/

zerubeus 6y ago

I love you

eat_veggies 6y ago

It seems like Nix and NixOps are designed for this type of thing, since they're built on a functional language instead of yaml. But I haven't played with them too much yet -- anyone have experience using them?