3 ms·

What a long winded article to say "it depends", I liked the history though. It got me thinking, here at amazon, we deliver "infrastructure as code" using the C

What a long winded article to say "it depends", I liked the history though. It got me thinking, here at amazon, we deliver "infrastructure as code" using the Cloud Development Kit: https://aws.amazon.com/cdk/ https://aws.amazon.com/cdk/ We expect engineers (not devops) to define their infrastructure in typescript and configure it through code. That code gets turned into cloudformation scripts and stands up the how cloud system for the api you're building. I think this is a great hybrid approach. Knowing what you want is different than knowing all the intricate details of defining, say an API gateway. But the CDK lets me stand up an API Gateway and configure it with a swagger and security policy and be done. This lowers the barrier for devs to do devops work, and lets teams own and move fast when making changes.

brodouevencode 4y ago
The layers of AWSCDK leads to a lot of brittleness, and that was a huge turn-off for us. I like the thought of building systems in a functional way but the tooling just isn't there yet. I haven't dived too deep into Terraform CDK yet, and Pulumi just had too many problems.
- nailer 4y ago
  What were your issues with Pulumi?
  - yarosh 4y ago
    tldr; it wraps Terraform providers poorly and can fail building proper infra diffs from time to time... https://github.com/pulumi/pulumi-aws-native https://github.com/pulumi/pulumi-aws-native is nowhere near GA state, just scroll through the issues... and the respective terraform wrapper https://github.com/pulumi/terraform-provider-aws https://github.com/pulumi/terraform-provider-aws is somewhat neglected in favour of the tf2pulumi native port above. Crossplane, on the other hand, does better with the Terrajet codegen, and all the infra drifts are a part of the reconciliation cycle, which is very handy on simpler deployments but doesn't work with more complex ones due to excessive drift polling model.
- theideaofcoffee 4y ago
  I am finding the same. Even a relatively simple deployment built on CDK has no end to issues and headaches, ranging from rollbacks that don't completely roll back to a previous state, dangling resources that aren't cleaned up properly, and the issues go on and on. This is mainly due to CDK depending on cloudformation which is, in my most humble of opinions, a non-starter for starting up anything more complex than a single ec2 instance. Had we built this out in terraform, state cleanup and tracking would have been more robust, the ability to retry resource creation would have been more stable, the project overall would have been much more of a pleasure to use. The functional/declarative aspect of terraform in relation to cloudformation is so much more polished. Edit: declarative
- cutthegrass2 4y ago
  Wondering what your issues were with Pulumi also? I'm leading the engineering at a startup based in the UK and we actively chose Pulumi over CDK, TF etc.... Been going 12months now and we're really pleased with the decision so far.
  - yarosh 4y ago
    1. both pulumi and crossplane just wraps the Terraform providers as is on many occasions, and quite poorly. There are a lot of pending issues with the dependency graphs, state refresh and proper state diffs. Although a lot of the most troubling issues had been resolved, it's still a mine field run. 2. Both TFCDK and dagger.io can be used for multistage TF deployments, although I prefer dagger myself... Terraform has a major state management design flaw that had been ignored by hashicorp to force TFE upsales. It's impossible to perform multi stage deployments with a single `terraform apply`. You have to manually identify the deplyoment targets for every stage, terraform providers do not support `depends_on` block and they are not a part of the resource dependency resolution graph. i.e. You can't deploy Vault than configure it with the respective provider - terraform will try to perform both deployment and configuration simultaneously and will fail. 3. This is due to strong Sales Opinion that a Single Plan is of a Positive Product Value for Terraform. While in practice it turned out to be False, the actual Product Value of Terraform is in Single Consolidated Infrastructure state, which can be analyzed by the respective static analyzers (infracost, tfsec, checkov, inframap, driftctl etc). And it's a strong pro compared to both Pulumi and Crossplane... Having a single state is a blessing for large companies with a tight operational schedule - having multiple states with a single lock can cause conflicts quite often, with volatile outcomes. Yet again, an upsale point for TFE. Even though Terraform "has more providers" you have to be able to support 'em all by yourself, HashiCorp does not provide a Viable Support Plan for the existing Official Terraform providers (on my xp - maybe someone was more lucky). That's why I'm often saying that DevOps is not a title, it's a methodology... and every DevSecOps guy should be well versed in golang to be able to support, test and extend the respective tools and operators (k8s automation). * answered in a bit more detail below https://news.ycombinator.com/item?id=32405064 https://news.ycombinator.com/item?id=32405064
danwee 4y ago
As a SWE, I don't think it's great. I'm more interested in writing product features and fixing bugs. I'm not that interested in writing CDK stacks to create databases or what not.
- malfist 4y ago
  That's fine if that's what you want. But I'd suggest that your career might be improved by being more willing to take on harder challenges and grow as an engineer. Product features and fixing bugs are intricately linked to infrastructure, and limiting yourself to only code is going to limit your ability to debug hard problems. Early in my career, I worked for a business that still used mainframes, and we had this random bug that caused processes not to communicate and drop messages. Because I was willing to dig into the intricacies of server administration, I was able to diagnose the problem as the IPC queues size being set to the kernal default, and the default only allowed for a few seconds of messages to back up. It was a quick fix, deploy a new kernal parameter to allow for bigger IPC buffers, but if I'd refused to do "devops" work, we'd never have found the problem. As my old boss said, you're not paid to do only the easy stuff. You're paid to do the hard stuff too.
  - danwee 4y ago
    > But I'd suggest that your career might be improved by being more willing to take on harder challenges and grow as an engineer. Indeed. That's what I have done (I do infra stuff + product features). I don't like it, but I do it. > As my old boss said, you're not paid to do only the easy stuff. You're paid to do the hard stuff too. This intrinsically states "product features == easy", "infra struff == hard". While I think that certain topics related to infrastructure are hard, certain topics related to product feature are hard as well. What I think your boss wanted to say is "I don't want to pay an extra paycheck to an infra engineer. So, I'll pay you 20% more so you do the infra stuff instead. And you get to 'grow as an engineer'. It's a win-win!".
- moduspol 4y ago
  If your features don't involve utilizing stuff like S3, SNS, SQS, Lambda, DynamoDB, CloudWatch Metrics / Alarms, you're probably missing out on a lot of value. Nobody wants to spend time "writing CDK stacks to create databases or what not," but implementing tough features without these services can easily involve exponentially more work / maintenance / expertise.
jcims 4y ago
The challenge with this is that many orgs have hundreds if not thousands of standards that must be adhered to in this situation. So if a engineer wishes to use a database, they can stand one up with the CDK no problem. But it's going to fail every audit unless they are aware of these requirements. And even if they are aware of these requirements, doing the integration for things like privileged access management, billing and security monitoring are a pain in the ass that provide no incremental value. On the flipside, having a single platform team write the IaC components that do all of this grunt work tends to reduce the degrees of freedom that the engineer has to build the application architecture exactly how they want it.
erikerikson 4y ago
The CDK seemed to be an attempt to supplant the Serverless framework and Pulumi even. Are those options barred for use internally or just under promoted?
- malfist 4y ago
  Amazon loves dogfooding. I don't think they're barred, but fall under the "nobody ever got fired for using CDK" domain.
syntheticcdo 4y ago
The Amazon CDK is a breath of fresh air and I love working with it, coming from a CloudFormation background. For myself, I can just `cdk synth` and take a peek at the output to conclude "yes, that's what I would have written and the TypeScript saved me hours of reading CF documentation". However, for others on my team coming in without the CF background, it feels a little like voodoo and as soon as they tread off the "standard path" I find myself getting pulled in to do the "hard stuff".

brodouevencode 4y ago

The layers of AWSCDK leads to a lot of brittleness, and that was a huge turn-off for us. I like the thought of building systems in a functional way but the tooling just isn't there yet. I haven't dived too deep into Terraform CDK yet, and Pulumi just had too many problems.

nailer 4y ago
What were your issues with Pulumi?
- yarosh 4y ago
  tldr; it wraps Terraform providers poorly and can fail building proper infra diffs from time to time... https://github.com/pulumi/pulumi-aws-native https://github.com/pulumi/pulumi-aws-native is nowhere near GA state, just scroll through the issues... and the respective terraform wrapper https://github.com/pulumi/terraform-provider-aws https://github.com/pulumi/terraform-provider-aws is somewhat neglected in favour of the tf2pulumi native port above. Crossplane, on the other hand, does better with the Terrajet codegen, and all the infra drifts are a part of the reconciliation cycle, which is very handy on simpler deployments but doesn't work with more complex ones due to excessive drift polling model.
theideaofcoffee 4y ago
I am finding the same. Even a relatively simple deployment built on CDK has no end to issues and headaches, ranging from rollbacks that don't completely roll back to a previous state, dangling resources that aren't cleaned up properly, and the issues go on and on. This is mainly due to CDK depending on cloudformation which is, in my most humble of opinions, a non-starter for starting up anything more complex than a single ec2 instance. Had we built this out in terraform, state cleanup and tracking would have been more robust, the ability to retry resource creation would have been more stable, the project overall would have been much more of a pleasure to use. The functional/declarative aspect of terraform in relation to cloudformation is so much more polished. Edit: declarative
cutthegrass2 4y ago
Wondering what your issues were with Pulumi also? I'm leading the engineering at a startup based in the UK and we actively chose Pulumi over CDK, TF etc.... Been going 12months now and we're really pleased with the decision so far.
- yarosh 4y ago
  1. both pulumi and crossplane just wraps the Terraform providers as is on many occasions, and quite poorly. There are a lot of pending issues with the dependency graphs, state refresh and proper state diffs. Although a lot of the most troubling issues had been resolved, it's still a mine field run. 2. Both TFCDK and dagger.io can be used for multistage TF deployments, although I prefer dagger myself... Terraform has a major state management design flaw that had been ignored by hashicorp to force TFE upsales. It's impossible to perform multi stage deployments with a single `terraform apply`. You have to manually identify the deplyoment targets for every stage, terraform providers do not support `depends_on` block and they are not a part of the resource dependency resolution graph. i.e. You can't deploy Vault than configure it with the respective provider - terraform will try to perform both deployment and configuration simultaneously and will fail. 3. This is due to strong Sales Opinion that a Single Plan is of a Positive Product Value for Terraform. While in practice it turned out to be False, the actual Product Value of Terraform is in Single Consolidated Infrastructure state, which can be analyzed by the respective static analyzers (infracost, tfsec, checkov, inframap, driftctl etc). And it's a strong pro compared to both Pulumi and Crossplane... Having a single state is a blessing for large companies with a tight operational schedule - having multiple states with a single lock can cause conflicts quite often, with volatile outcomes. Yet again, an upsale point for TFE. Even though Terraform "has more providers" you have to be able to support 'em all by yourself, HashiCorp does not provide a Viable Support Plan for the existing Official Terraform providers (on my xp - maybe someone was more lucky). That's why I'm often saying that DevOps is not a title, it's a methodology... and every DevSecOps guy should be well versed in golang to be able to support, test and extend the respective tools and operators (k8s automation). * answered in a bit more detail below https://news.ycombinator.com/item?id=32405064 https://news.ycombinator.com/item?id=32405064

danwee 4y ago

As a SWE, I don't think it's great. I'm more interested in writing product features and fixing bugs. I'm not that interested in writing CDK stacks to create databases or what not.

malfist 4y ago
That's fine if that's what you want. But I'd suggest that your career might be improved by being more willing to take on harder challenges and grow as an engineer. Product features and fixing bugs are intricately linked to infrastructure, and limiting yourself to only code is going to limit your ability to debug hard problems. Early in my career, I worked for a business that still used mainframes, and we had this random bug that caused processes not to communicate and drop messages. Because I was willing to dig into the intricacies of server administration, I was able to diagnose the problem as the IPC queues size being set to the kernal default, and the default only allowed for a few seconds of messages to back up. It was a quick fix, deploy a new kernal parameter to allow for bigger IPC buffers, but if I'd refused to do "devops" work, we'd never have found the problem. As my old boss said, you're not paid to do only the easy stuff. You're paid to do the hard stuff too.
- danwee 4y ago
  > But I'd suggest that your career might be improved by being more willing to take on harder challenges and grow as an engineer. Indeed. That's what I have done (I do infra stuff + product features). I don't like it, but I do it. > As my old boss said, you're not paid to do only the easy stuff. You're paid to do the hard stuff too. This intrinsically states "product features == easy", "infra struff == hard". While I think that certain topics related to infrastructure are hard, certain topics related to product feature are hard as well. What I think your boss wanted to say is "I don't want to pay an extra paycheck to an infra engineer. So, I'll pay you 20% more so you do the infra stuff instead. And you get to 'grow as an engineer'. It's a win-win!".
moduspol 4y ago
If your features don't involve utilizing stuff like S3, SNS, SQS, Lambda, DynamoDB, CloudWatch Metrics / Alarms, you're probably missing out on a lot of value. Nobody wants to spend time "writing CDK stacks to create databases or what not," but implementing tough features without these services can easily involve exponentially more work / maintenance / expertise.

jcims 4y ago

The challenge with this is that many orgs have hundreds if not thousands of standards that must be adhered to in this situation. So if a engineer wishes to use a database, they can stand one up with the CDK no problem. But it's going to fail every audit unless they are aware of these requirements. And even if they are aware of these requirements, doing the integration for things like privileged access management, billing and security monitoring are a pain in the ass that provide no incremental value. On the flipside, having a single platform team write the IaC components that do all of this grunt work tends to reduce the degrees of freedom that the engineer has to build the application architecture exactly how they want it.

erikerikson 4y ago

The CDK seemed to be an attempt to supplant the Serverless framework and Pulumi even. Are those options barred for use internally or just under promoted?

malfist 4y ago
Amazon loves dogfooding. I don't think they're barred, but fall under the "nobody ever got fired for using CDK" domain.

syntheticcdo 4y ago

The Amazon CDK is a breath of fresh air and I love working with it, coming from a CloudFormation background. For myself, I can just `cdk synth` and take a peek at the output to conclude "yes, that's what I would have written and the TypeScript saved me hours of reading CF documentation". However, for others on my team coming in without the CF background, it feels a little like voodoo and as soon as they tread off the "standard path" I find myself getting pulled in to do the "hard stuff".