13 ms·

On a related note, CDK for Terraform allows DevOps practitioners to use a variety of programming languages instead of HCL. I've really enjoyed modeling my AWS e

On a related note, CDK for Terraform allows DevOps practitioners to use a variety of programming languages instead of HCL. I've really enjoyed modeling my AWS environments with Python using Terraform only as the engine. More info here: https://github.com/hashicorp/terraform-cdk https://github.com/hashicorp/terraform-cdk

justin_oaks 5y ago
Are all variables, conditionals, templating, and loops done in Python? Or is some of that still needed on the Terraform side?
- diroussel 5y ago
  Basically you create the desired state DAG in procedural code, rather than the TF DSL. Blithe diffing and applying are the same.
  - ben0x539 5y ago
    Can you inspect inputs from terraform resource attributes or data sources in the procedural evaluation?
    - cmclaughlin 5y ago
      > Can you inspect inputs from terraform resource attributes or data sources in the procedural evaluation? No... the high level programming language really just serves as a bridge or translation layer to a Terraform compatible JSON file. Those sorts of evaluations don’t happen to the actually plan/apply. However, you may find it useful to make direct API calls to your cloud provider in cdktf stacks. For instance, I mostly use data lookups but if I want to perform string operations on that sort of data I would use boto3 instead.
wooly_bully 5y ago
The 'not using HCL' bit is really the only positive I've found so far because everything else has been more difficult than just using TF directly. I think my goals were slightly off from the beginning, because this is really just replacing one CLI with another for me at this point. What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Ideally, I had intended to distribute some npm-installable packages which would run this stuff.
- dastx 5y ago
  > What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Maybe not node/python, but I'm pretty sure you can use terraform as a package in go. If not, there is always the "make temp dir, write/download files necessary tf files, run terraform apply"
  - wooly_bully 5y ago
    That's good to know at least; will give the go API a look. The latter option you're recommending is essentially what I went with (Node bin script that shells out to run cdktf commands).
    - deleted 5y ago
      [deleted]
- lukehoban 5y ago
  > What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Ideally, I had intended to distribute some npm-installable packages which would run this stuff. This is actually exactly the use case we’ve designed the Pulumi Automation API (https://www.pulumi.com/blog/automation-api/ https://www.pulumi.com/blog/automation-api/) to support. Allowing modern IaC technology (like Pulumi or Terraform) to be easily embedded into custom software solutions, instead of just being something humans work with directly, is a huge potential enabler for the next wave of cloud infrastructure management tooling.
theopsguy 5y ago
If you like CDK, then i highly recommend pulumi
- k__ 5y ago
  How do they differ?
  - xyzzy123 5y ago
    So regular CDK is basically a program-driven CFN generator. Pulumi has a similar model where you build a resource graph at runtime BUT it's also got the execution engine built-in to the tool. What this means in practice is that you can create resources (like a kube cluster) and then use them as providers (e.g provision state tracked resources with kube api) all in the same operation. You can also (in your infracode or an importable module) define "dynamic providers", meaning you can easily extend the execution engine to do custom things related to your workload. As an example, imagine you want to create a cluster, deploy an app, then provision (state tracked) some app resources like an admin user and group via the app's REST API. You can do that without too much fuss. Neither terraform nor CDK can really do those things very well. TF is not powerful enough language-wise, and in CDK the execution phase is locked away from you.
    - bogomipz 5y ago
      >"So regular CDK is basically a program-driven CFN generator." What is CFN here? Cloudformation?
      - count 5y ago
        Yes.
    - k__ 5y ago
      I don't think I 100% understand. Could you elaborate a bit more? Let's say I have an app with API Gateway, Lambda, and DynamoDB. I would provision them with one of those tools CDK or Pulumi. How would these tools differ in their provisioning steps?
erik_seaberg 5y ago
I really wish CDK supported writing a Terraform plugin in any of those languages.
- throwaway894345 5y ago
  I really wish they would support CDK in Go!
  - iamsureshk 5y ago
    Getting started with the AWS Cloud Development Kit and Go >https://aws.amazon.com/blogs/developer/getting-started-with-the-aws-cloud-development-kit-and-go/ https://aws.amazon.com/blogs/developer/getting-started-with-...
    - throwaway894345 5y ago
      We were talking about Terraform CDK here, not AWS CDK. It is cool that AWS CDK supports Go though!
bogomipz 5y ago
>"On a related note, CDK for Terraform allows DevOps practitioners to use a variety of programming languages instead of HCL." Out of curiosity what is it that people generally don't like about HCL?
- throwaway894345 5y ago
  Bizarre syntax, crazy resource paths, whole directories just to define one function, etc etc etc.
- cies 5y ago
  Yet one more thing to learn. No types (hints in IDEs).
  - sofixa 5y ago
    v2 has types, and my IDE ( VS Code ) does hints
    - jen20 5y ago
      The IntelliJ HCL plugin has full refactoring support, auto completion etc. It’s a shame it’s not open source, since there must be a Kotlin implementation of an HCL2 parser underpinning it.
    - cies 5y ago
      Thanks for this, I was unaware!
Guthur 5y ago
Why "practitioner"? it really does not feel like the correct term.
- viraptor 5y ago
  It's a philosophy not a title https://www.visualscript.com/devops/devops-is-not-a-title-it-is-a-philosophy.htm https://www.visualscript.com/devops/devops-is-not-a-title-it...
  - Guthur 5y ago
    But that still does not align with the use of the word practitioner, it's usually a formal profession with licensing such as medicine or law.
    - viraptor 5y ago
      That's just one way to use it. Practitioners practice something. For example you can find https://en.wikipedia.org/wiki/Martial_arts https://en.wikipedia.org/wiki/Martial_arts using "practitioner" for people training martial arts. There are practitioners of religion, techniques, etc.
dkdk8283 5y ago
There is little value in abstracting HCL al la troposphere. just write native
ygouzerh 5y ago
Thanks for the share, didn't heard about it before, I see the advantage in a use-case that we have : We have are actually working on : "How to manage Terraform resources". We ended up having a conflict dev <-> ops where dev teams are messaging the terraform guys to create resources. For example, to create a database, it will take only 1 hour writing HCL, but 2 weeks of emails to align on the specs. We are currently building something on top, to have resources that can be created in a self-service mode by the devs themselves. (Behind the scenes, it uses Terraform modules to generate resources that will comply with the company policies). For the moment, it's a bunch of Jenkins pipelines. Having a CDK can actually help us a lot there. (Can plug it to a CMDB database, have a UI on top, etc)
74d-fe6-2c6 5y ago
Why would you use CDK with Terraform instead of CDK with CloudFormation? The latter seems like a more reasonable choice as it is native to AWS.
- cmclaughlin 5y ago
  CloudFormation is really slow and lags behind in supporting all AWS features. Terraform is super fast, has great docs and vast coverage for managing just about any type of cloud resource.

justin_oaks 5y ago

Are all variables, conditionals, templating, and loops done in Python? Or is some of that still needed on the Terraform side?

diroussel 5y ago
Basically you create the desired state DAG in procedural code, rather than the TF DSL. Blithe diffing and applying are the same.
- ben0x539 5y ago
  Can you inspect inputs from terraform resource attributes or data sources in the procedural evaluation?
  - cmclaughlin 5y ago
    > Can you inspect inputs from terraform resource attributes or data sources in the procedural evaluation? No... the high level programming language really just serves as a bridge or translation layer to a Terraform compatible JSON file. Those sorts of evaluations don’t happen to the actually plan/apply. However, you may find it useful to make direct API calls to your cloud provider in cdktf stacks. For instance, I mostly use data lookups but if I want to perform string operations on that sort of data I would use boto3 instead.

wooly_bully 5y ago

The 'not using HCL' bit is really the only positive I've found so far because everything else has been more difficult than just using TF directly. I think my goals were slightly off from the beginning, because this is really just replacing one CLI with another for me at this point. What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Ideally, I had intended to distribute some npm-installable packages which would run this stuff.

dastx 5y ago
> What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Maybe not node/python, but I'm pretty sure you can use terraform as a package in go. If not, there is always the "make temp dir, write/download files necessary tf files, run terraform apply"
- wooly_bully 5y ago
  That's good to know at least; will give the go API a look. The latter option you're recommending is essentially what I went with (Node bin script that shells out to run cdktf commands).
  - deleted 5y ago
    [deleted]
lukehoban 5y ago
> What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Ideally, I had intended to distribute some npm-installable packages which would run this stuff. This is actually exactly the use case we’ve designed the Pulumi Automation API (https://www.pulumi.com/blog/automation-api/ https://www.pulumi.com/blog/automation-api/) to support. Allowing modern IaC technology (like Pulumi or Terraform) to be easily embedded into custom software solutions, instead of just being something humans work with directly, is a huge potential enabler for the next wave of cloud infrastructure management tooling.

theopsguy 5y ago

If you like CDK, then i highly recommend pulumi

k__ 5y ago
How do they differ?
- xyzzy123 5y ago
  So regular CDK is basically a program-driven CFN generator. Pulumi has a similar model where you build a resource graph at runtime BUT it's also got the execution engine built-in to the tool. What this means in practice is that you can create resources (like a kube cluster) and then use them as providers (e.g provision state tracked resources with kube api) all in the same operation. You can also (in your infracode or an importable module) define "dynamic providers", meaning you can easily extend the execution engine to do custom things related to your workload. As an example, imagine you want to create a cluster, deploy an app, then provision (state tracked) some app resources like an admin user and group via the app's REST API. You can do that without too much fuss. Neither terraform nor CDK can really do those things very well. TF is not powerful enough language-wise, and in CDK the execution phase is locked away from you.
  - bogomipz 5y ago
    >"So regular CDK is basically a program-driven CFN generator." What is CFN here? Cloudformation?
    - count 5y ago
      Yes.
  - k__ 5y ago
    I don't think I 100% understand. Could you elaborate a bit more? Let's say I have an app with API Gateway, Lambda, and DynamoDB. I would provision them with one of those tools CDK or Pulumi. How would these tools differ in their provisioning steps?

erik_seaberg 5y ago

I really wish CDK supported writing a Terraform plugin in any of those languages.

throwaway894345 5y ago
I really wish they would support CDK in Go!
- iamsureshk 5y ago
  Getting started with the AWS Cloud Development Kit and Go >https://aws.amazon.com/blogs/developer/getting-started-with-the-aws-cloud-development-kit-and-go/ https://aws.amazon.com/blogs/developer/getting-started-with-...
  - throwaway894345 5y ago
    We were talking about Terraform CDK here, not AWS CDK. It is cool that AWS CDK supports Go though!

bogomipz 5y ago

>"On a related note, CDK for Terraform allows DevOps practitioners to use a variety of programming languages instead of HCL." Out of curiosity what is it that people generally don't like about HCL?

throwaway894345 5y ago
Bizarre syntax, crazy resource paths, whole directories just to define one function, etc etc etc.
cies 5y ago
Yet one more thing to learn. No types (hints in IDEs).
- sofixa 5y ago
  v2 has types, and my IDE ( VS Code ) does hints
  - jen20 5y ago
    The IntelliJ HCL plugin has full refactoring support, auto completion etc. It’s a shame it’s not open source, since there must be a Kotlin implementation of an HCL2 parser underpinning it.
  - cies 5y ago
    Thanks for this, I was unaware!

Guthur 5y ago

Why "practitioner"? it really does not feel like the correct term.

viraptor 5y ago
It's a philosophy not a title https://www.visualscript.com/devops/devops-is-not-a-title-it-is-a-philosophy.htm https://www.visualscript.com/devops/devops-is-not-a-title-it...
- Guthur 5y ago
  But that still does not align with the use of the word practitioner, it's usually a formal profession with licensing such as medicine or law.
  - viraptor 5y ago
    That's just one way to use it. Practitioners practice something. For example you can find https://en.wikipedia.org/wiki/Martial_arts https://en.wikipedia.org/wiki/Martial_arts using "practitioner" for people training martial arts. There are practitioners of religion, techniques, etc.

dkdk8283 5y ago

There is little value in abstracting HCL al la troposphere. just write native

ygouzerh 5y ago

Thanks for the share, didn't heard about it before, I see the advantage in a use-case that we have : We have are actually working on : "How to manage Terraform resources". We ended up having a conflict dev <-> ops where dev teams are messaging the terraform guys to create resources. For example, to create a database, it will take only 1 hour writing HCL, but 2 weeks of emails to align on the specs. We are currently building something on top, to have resources that can be created in a self-service mode by the devs themselves. (Behind the scenes, it uses Terraform modules to generate resources that will comply with the company policies). For the moment, it's a bunch of Jenkins pipelines. Having a CDK can actually help us a lot there. (Can plug it to a CMDB database, have a UI on top, etc)

74d-fe6-2c6 5y ago

Why would you use CDK with Terraform instead of CDK with CloudFormation? The latter seems like a more reasonable choice as it is native to AWS.

cmclaughlin 5y ago
CloudFormation is really slow and lags behind in supporting all AWS features. Terraform is super fast, has great docs and vast coverage for managing just about any type of cloud resource.