6 ms·

I have been saying this for a while now -- as much as I respect Hashicorp and admire their engineering, HCL is the biggest problem with the tools they have that

I have been saying this for a while now -- as much as I respect Hashicorp and admire their engineering, HCL is the biggest problem with the tools they have that it's used in -- biggest of which being Terraform. Don't use a DSL where a full programming language is what you need. I can see that they probably wanted to be able to skip writing 2/3 language CDKs out of the box, and DSLs probably hit on the need for simplicity really early, but infrastructure code is somewhere you really want to be able to drop into the full expressive power of a programming language. I use Pulumi[0] for my projects, and while I doubt people will be able to get it as much adoption as I think it should have in corporate environments, Terraform has introduced the CDK[1] which is a similar approach. Have a talk with your engineers about complexity (and how not to strangle the rest of the team with it) and stop using DSLs in places you need more expressive power. I think a good indication of when you need to stop using DSLs or when you should maybe realize a DSL wasn't the right way to go is when you start doing things like writing control flow. You could can get away with DSLs if you were using s-expressions, but that is an exception that proves the rule (because there happens to a family of languages that treat s-expressions as syntax). [0]: https://www.pulumi.com https://www.pulumi.com [1]: https://www.hashicorp.com/blog/cdk-for-terraform-enabling-python-and-typescript-support https://www.hashicorp.com/blog/cdk-for-terraform-enabling-py...

cyberpunk 6y ago
I’m super curious about pulumi — I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway, is it mature enough for a reasonably standard aws/eks setup?
- hardwaresofton 6y ago
  As much as I like Pulumi, no one gets fired for choosing Terraform. You're going to be able to find the most help, discussion, and stuff for Terraform. Terraform and it's ecosystem is the safer choice, especially when the CDK is an option. On the other hand I can say that a bunch of Pulumi's stuff is built on terraform underneath the covers so I'm not sure how far behind they are but it probably isn't by much. In my limited use (for example deploying SES stuff) I found it a pleasure to use and didn't find anything that terraform did that it didn't (again, likely due to pulumi being able to utilize terraform providers under the covers). > I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway Hope you're really sure this is where you want to use your effort tokens -- I'm not sure how much of a mess with it, but these tools are so new, it might be worth seeing if you can de-clutter it without a complete rewrite. Or maybe it's small enough that a complete rewrite is relatively low friction... Either way, there be dragons.
busterarm 6y ago
The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. Terraform should merely provision things and in that role I find it fine as is. Preferred, even.
- hardwaresofton 6y ago
  > The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. Declarative infrastructure shouldn't be pursued for it's own sake -- what I want is efficient, and simple to manage infrastructure automation. The declarative nature is awesome, but once you start doing plumbing of variables and complexity from one static script from another, the cognitive load of keeping this all in line is better managed with a programming language in my opinion, you're just choosing bash/jq/awk/etc instead of a different language. I think "the way the declarations are made must be static files" is dogmatic or at least limiting for me. Yes it is absolutely the simplest way to view what's present, but the problem is when someone goes into change any of this they will be dealing with your bolted-together complexity (even if it's not very complex). > I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. > Terraform should merely provision things and in that role I find it fine as is. Preferred, even. I can't argue with the efficiency and efficacy of your setup, but I don't think much of this has to do with what we were discussing -- Pulumi does not seek to do the jobs of those other tools -- it's not going to build your VM images or do provisioning (unless you use it that way like with terraform[0]). Here's a concrete example of a benefit I got form using Pulumi over terraform recently, in some code working with SES: import * as fs from "fs"; // ... more imports and other lines // Email Access key const emailAccessKey = new aws.iam.AccessKey( `${stack}-ses-access-key`, {user: emailUser.name} ); export const emailUserSMTPPassword = emailAccessKey.sesSmtpPasswordV4; export const emailUserSecret = emailAccessKey.encryptedSecret; // Write the smtp username and password out to a local secret file const apiSecretsDir = path.join(__dirname, "secrets", "api", stack); const smtpUsernameFilePath = path.resolve(path.join(apiSecretsDir, "SES_USERNAME.secret")); const smtpPasswordFilePath = path.resolve(path.join(apiSecretsDir, "SES_PASSWORD.secret")); emailAccessKey.sesSmtpPasswordV4.apply(password => { console.log(`Writing SES SMTP username to [${smtpUsernameFilePath}]`); fs.writeFileSync(smtpUsernameFilePath, emailUsername); console.log(`Writing SES SMTP password to [${smtpPasswordFilePath}]`); fs.writeFileSync(smtpPasswordFilePath, password); }); I wanted to write information out to a file... So I just did, and that was it. No need to reach for the stack output later and pipe it anywhere -- any time pulumi runs it will update that variable if/when it changes, and the next tool (which requires the file at that path to be present) will continue on without knowing a thing. I can't say that this is perfect Pulumi code (ex. I could have defined a custom Resource to do this for me), but I have saved myself having to do the plumbing with bash scripts and terraform output awk-ing, and the information goes just where I want it (NOTE: the secrets folder is encrypted with git-crypt[1]). When someone comes to this file (ses.ts), they're going to be able to easily trace where these values where generated -- similar with bash scripts, but now they don't have to be a bash/awk/jq master to manipulate information. There are definitely some gotchas to using Pulumi (like the `.apply` there), but in the end, I'd prefer to make changes like this in a consistent language I like (Typescript). My toolkit looks very similar to you, except I basically only use make + kubectl + pulumi + ansible (rarely, because of the kind of servers I rent). [0]: https://www.terraform.io/docs/provisioners/ https://www.terraform.io/docs/provisioners/ [1]: https://www.agwa.name/projects/git-crypt/ https://www.agwa.name/projects/git-crypt/
WatchDog 6y ago
Oh wow, I had no idea that terraform had implemented a CDK frontend. That's awesome. I've been writing a bunch of regular CDK code, which is AWS specific, I loathe HCL, and I was a bit disappointed by the quality of pulumi when I evaluated it. Il have to check out the terraform CDK support.
- temikus 6y ago
  Care to share your thoughts on Pulumi quality? I’m asking since I’m currently looking at their TypeScript implementation to use in prod and would love to know any issues that may pop up.
  - WatchDog 6y ago
    I didn't really get that far with it, but the first issue I ran into, was that the program wouldn't work at all, just errored out with no logging, even after passing all available debug flags. Spent hours digging through the code trying to figure out what the issue was, figured it out, reported it, even provided the solution, and the github issue remains open over a year later[0]. The other thing I didn't particularly like about it, is that it doesn't really seem feasible to develop using it, without it having access/credentials to your cloud environment. The great thing about CDK, is that I can synthesize templates, write unit tests, and have a reasonable amount of confidence in my changes, without actually running it against an environment. Don't get me wrong CDK/Cloudformation still fails plenty at deployment time, but it's better than nothing. [0] https://github.com/pulumi/pulumi/issues/3590 https://github.com/pulumi/pulumi/issues/3590
mixedCase 6y ago
> but infrastructure code is somewhere you really want to be able to drop into the full expressive power of a programming language. Please expand on what this expressive power you need is. I see things like Pulumi and I can only think of IaC codebases that end up being turing tarpits in the hands of developers that don't get what the whole point of the declarative model is, putting a bunch of untestable arbitrary IO and high cyclomatic complexity in the middle of determining what is going to be deployed. And this is definitely not an argument against having better languages than HCL, Dhall would certainly be a step up that doesn't give the developer an arsenal of footguns to move fast and wreak havoc on the underlying foundations of a business. But I want to know what needs you have that you feel are worth the risk of putting a bull in the china shop.
- hardwaresofton 6y ago
  for_each[0] was a good example, though it's been in there for a long time. Pulumi was around at the same time and wouldn't have suffered from this same problem, though this was 2 years ago. [0]: https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each https://www.hashicorp.com/blog/hashicorp-terraform-0-12-prev...

cyberpunk 6y ago

I’m super curious about pulumi — I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway, is it mature enough for a reasonably standard aws/eks setup?

hardwaresofton 6y ago
As much as I like Pulumi, no one gets fired for choosing Terraform. You're going to be able to find the most help, discussion, and stuff for Terraform. Terraform and it's ecosystem is the safer choice, especially when the CDK is an option. On the other hand I can say that a bunch of Pulumi's stuff is built on terraform underneath the covers so I'm not sure how far behind they are but it probably isn't by much. In my limited use (for example deploying SES stuff) I found it a pleasure to use and didn't find anything that terraform did that it didn't (again, likely due to pulumi being able to utilize terraform providers under the covers). > I just inherited a mess of terragrunt at a new gig and was plannning to rewrite it anyway Hope you're really sure this is where you want to use your effort tokens -- I'm not sure how much of a mess with it, but these tools are so new, it might be worth seeing if you can de-clutter it without a complete rewrite. Or maybe it's small enough that a complete rewrite is relatively low friction... Either way, there be dragons.

busterarm 6y ago

The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. Terraform should merely provision things and in that role I find it fine as is. Preferred, even.

hardwaresofton 6y ago
> The moment you need control flow to define your resources, I'd argue that you're verging away from the realm of declarative infrastructure. Declarative infrastructure shouldn't be pursued for it's own sake -- what I want is efficient, and simple to manage infrastructure automation. The declarative nature is awesome, but once you start doing plumbing of variables and complexity from one static script from another, the cognitive load of keeping this all in line is better managed with a programming language in my opinion, you're just choosing bash/jq/awk/etc instead of a different language. I think "the way the declarations are made must be static files" is dogmatic or at least limiting for me. Yes it is absolutely the simplest way to view what's present, but the problem is when someone goes into change any of this they will be dealing with your bolted-together complexity (even if it's not very complex). > I'm using Terraform to manage 10^4 machines in combination with sane CI/CD, Bash/JQ (for dealing with Terraform outputs), Packer and Ansible. Everytime I see somebody reaching to a full programming language to define their infrastructure, they seem to be doing too much with one tool. > Terraform should merely provision things and in that role I find it fine as is. Preferred, even. I can't argue with the efficiency and efficacy of your setup, but I don't think much of this has to do with what we were discussing -- Pulumi does not seek to do the jobs of those other tools -- it's not going to build your VM images or do provisioning (unless you use it that way like with terraform[0]). Here's a concrete example of a benefit I got form using Pulumi over terraform recently, in some code working with SES: import * as fs from "fs"; // ... more imports and other lines // Email Access key const emailAccessKey = new aws.iam.AccessKey( `${stack}-ses-access-key`, {user: emailUser.name} ); export const emailUserSMTPPassword = emailAccessKey.sesSmtpPasswordV4; export const emailUserSecret = emailAccessKey.encryptedSecret; // Write the smtp username and password out to a local secret file const apiSecretsDir = path.join(__dirname, "secrets", "api", stack); const smtpUsernameFilePath = path.resolve(path.join(apiSecretsDir, "SES_USERNAME.secret")); const smtpPasswordFilePath = path.resolve(path.join(apiSecretsDir, "SES_PASSWORD.secret")); emailAccessKey.sesSmtpPasswordV4.apply(password => { console.log(`Writing SES SMTP username to [${smtpUsernameFilePath}]`); fs.writeFileSync(smtpUsernameFilePath, emailUsername); console.log(`Writing SES SMTP password to [${smtpPasswordFilePath}]`); fs.writeFileSync(smtpPasswordFilePath, password); }); I wanted to write information out to a file... So I just did, and that was it. No need to reach for the stack output later and pipe it anywhere -- any time pulumi runs it will update that variable if/when it changes, and the next tool (which requires the file at that path to be present) will continue on without knowing a thing. I can't say that this is perfect Pulumi code (ex. I could have defined a custom Resource to do this for me), but I have saved myself having to do the plumbing with bash scripts and terraform output awk-ing, and the information goes just where I want it (NOTE: the secrets folder is encrypted with git-crypt[1]). When someone comes to this file (ses.ts), they're going to be able to easily trace where these values where generated -- similar with bash scripts, but now they don't have to be a bash/awk/jq master to manipulate information. There are definitely some gotchas to using Pulumi (like the `.apply` there), but in the end, I'd prefer to make changes like this in a consistent language I like (Typescript). My toolkit looks very similar to you, except I basically only use make + kubectl + pulumi + ansible (rarely, because of the kind of servers I rent). [0]: https://www.terraform.io/docs/provisioners/ https://www.terraform.io/docs/provisioners/ [1]: https://www.agwa.name/projects/git-crypt/ https://www.agwa.name/projects/git-crypt/

WatchDog 6y ago

Oh wow, I had no idea that terraform had implemented a CDK frontend. That's awesome. I've been writing a bunch of regular CDK code, which is AWS specific, I loathe HCL, and I was a bit disappointed by the quality of pulumi when I evaluated it. Il have to check out the terraform CDK support.

temikus 6y ago
Care to share your thoughts on Pulumi quality? I’m asking since I’m currently looking at their TypeScript implementation to use in prod and would love to know any issues that may pop up.
- WatchDog 6y ago
  I didn't really get that far with it, but the first issue I ran into, was that the program wouldn't work at all, just errored out with no logging, even after passing all available debug flags. Spent hours digging through the code trying to figure out what the issue was, figured it out, reported it, even provided the solution, and the github issue remains open over a year later[0]. The other thing I didn't particularly like about it, is that it doesn't really seem feasible to develop using it, without it having access/credentials to your cloud environment. The great thing about CDK, is that I can synthesize templates, write unit tests, and have a reasonable amount of confidence in my changes, without actually running it against an environment. Don't get me wrong CDK/Cloudformation still fails plenty at deployment time, but it's better than nothing. [0] https://github.com/pulumi/pulumi/issues/3590 https://github.com/pulumi/pulumi/issues/3590

mixedCase 6y ago

> but infrastructure code is somewhere you really want to be able to drop into the full expressive power of a programming language. Please expand on what this expressive power you need is. I see things like Pulumi and I can only think of IaC codebases that end up being turing tarpits in the hands of developers that don't get what the whole point of the declarative model is, putting a bunch of untestable arbitrary IO and high cyclomatic complexity in the middle of determining what is going to be deployed. And this is definitely not an argument against having better languages than HCL, Dhall would certainly be a step up that doesn't give the developer an arsenal of footguns to move fast and wreak havoc on the underlying foundations of a business. But I want to know what needs you have that you feel are worth the risk of putting a bull in the china shop.

hardwaresofton 6y ago
for_each[0] was a good example, though it's been in there for a long time. Pulumi was around at the same time and wouldn't have suffered from this same problem, though this was 2 years ago. [0]: https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each https://www.hashicorp.com/blog/hashicorp-terraform-0-12-prev...