8 ms·

Converting Full Terraform Programs to Pulumi

79 points by hekul 3y ago

dmitrygr 3y ago
https://pixelastic.github.io/pokemonorbigdata/ https://pixelastic.github.io/pokemonorbigdata/
s_Hogg 3y ago
Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.
- rjbwork 3y ago
  I'm pretty sure they do, which is a feature, IMO. We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. We also just recently started setting up a periodic drift detection build to help identify and address drift. Could you elaborate on the circumstances that have caused your state sync issues?
  - s_Hogg 3y ago
    Yeah we have a 10Mb state file and it already takes 30-45 min to deploy. The main way it happens is if someone cancels a CI/CD job because they're realised something is wrong and don't want to wait that long to try again. That's not an issue if there isn't a state file, but yeah I can see how there would be a tradeoff possibly associated.
    - thedougd 3y ago
      IME the plan times explode because the refreshes hit the cloud provider API rate limits.
    - Aeolun 3y ago
      Pulumi is pretty good about keeping track of in-progress state. If you cancel and restart a new job, it’ll first ask you what to do with the in-progress resources.
    - dronacharya 3y ago
      That does seem like quite a large state file. Would you be able to shed some light on the sorts of of resources that you're provisioning? I've found it to be useful to split dependencies between cloud resources, and to add links via data lookups. At any reasonable scale, I've found splitting Terraform state to be crucial. For example: network resources and Kubernetes clusters are created separately, and networking is a required dependency for Kubernetes clusters. Resources are associated with environments and regions, and modules take in environments and regions as parameters. In my setup, we have 200+ state files, and the largest is around 80KB.
      - rjbwork 3y ago
        How do you manage the meta DAG between all of that stuff in the event of, e.g. complete disaster recovery?
  - throwdbaaway 3y ago
    > We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. I don't think that's necessary true. Most cloud API's actually can return hundreds of records with 1 API calls, e.g. https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_DescribeTargetGroups.html https://docs.aws.amazon.com/elasticloadbalancing/latest/APIR... has a maximum page size of 400. If I manage the cloud resources via some custom tools and/or with some ansible-fu, I can decide to batch the API calls when it makes sense. With terraform, it is not possible to do so (https://github.com/hashicorp/terraform-plugin-sdk/issues/66 https://github.com/hashicorp/terraform-plugin-sdk/issues/66, https://github.com/hashicorp/terraform-provider-aws/issues/2553 https://github.com/hashicorp/terraform-provider-aws/issues/2...).
- nl 3y ago
  Yes they do. You can refresh the state file quite easily using the `pulimi refresh` command. I'm not really sure how you'd do it without a state file though? State drift is very common and without an idea of the current state it's pretty difficult for any framework to work out what updates need to be applied.
- esafak 3y ago
  Does pulumi let you write a state reader, so you can add schemas or some kind of validation?
- paulgb 3y ago
  I’ve been pretty happy with Pulumi but have learned that state management can be a pain. I got myself into a situation recently that I could only get out of by using the CLI interactively. I wound up with multiple copies of each resource, which shared a URN, so when I tried to delete them from the CLI it would always prompt me for which instance of that URN to delete. I ended up spending much of a day writing a program to call their CLI and then interact with it programmatically because I had hundreds of resources to delete. Since then I’ve been doing more with CloudFormation directly and am tempted to switch.
- fodkodrasz 3y ago
  The claimed pro of being able to use a normal programming language fades away quickly as you are forced into non-idiomatic boilerplate patterns to get pulumi to work. State is a problem and have had just the same kind of hardships when had to manually modify it on some changes as I would have had on terraform. (transfer resources between state stores as a business function is handed over within company, with zero downtime, import docs are not always correct/up2date just like in case of terraform, as if the example codes were generated by chatgpt) Overall pulumi couldn't convince me, it wasn't a bit more convenient than terraform, yet I could bump into its bugs for pretty basic aws features (around lambdas/apigw, but was more than a year ago, might be fixed already) which did work just fine with terraform ootb, and the pulumi code felt bad to look at despite being in a language I generally like. It did not bring the effect management wanted that developers with no terraform knowledge can be onboarded to infra, as of course infra knowledge is needed and is harder to pick up than learning terraform syntax. Despite my dislike for terraform and its syntax's limitations in expressiveness, I'm still back to TF, and is more productive than pulumi was.
tjasdfhwd___ 3y ago
I've been happy with Hashicorp Terraform Cloud and HCL except for the very recent price hikes which may force us to leave them so we are keeping our eyes out for alternatives. The AI assistant seen here is a very good idea. I will actually share this with my team. I imagine the terraform conversion doesn't work when you use in-house written providers (in golang), is this correct?
- jaxxstorm 3y ago
  You would need to bridge the Terraform provider to a Pulumi provider first. If you've written a Terraform provider, bridging to a Pulumi provider should be straightforward. If you need help with this, please reach out to me on the Pulumi community slack at https://slack.pulumi.com https://slack.pulumi.com and happy to help
- tedivm 3y ago
  Fortunately there are a lot of other terraform deployment and state management systems, such as Spacelift and Scalr.
- e12e 3y ago
  We're building our new infrastructure with Terraform - and experimenting with postgresql for state. So far so good, but we have only a handful of resources. Bit of a catch-22 to setup managed postgres for state, tailscale network with a vm as exit-node (to access postgres via VPN/lan - not Internet) - via Terraform - and then switch state from local (initial setup) to postgres. But nice to be able to eg: interact with a different provider for DNS, supplying IPs from other resources.
- cube2222 3y ago
  Hey! I recommend checking out Spacelift[0]. It's a CI/CD system specialized for Infra as Code, including Terraform and Pulumi. We have both a ton of additional features, as well as a much more reasonable pricing model, so we've recently been talking a lot to folks wanting to migrate from TFC due to the changes you've mentioned. You can easily reach out to us on the website by scheduling a demo, using the chat widget, or just sending me an email (see my profile). [0]: https://spacelift.io https://spacelift.io Disclaimer: Software Engineering Team Lead at Spacelift, so take this with a healthy grain of salt, but I do legitimately think it's a great product.
nl 3y ago
I've used Pulumi for a few small projects. I quite like it and I can see a Terraform importer being useful (for example GCP provides comprehensive Terraform support and being able to convert those to Pulumi quickly would speed things up a lot).
x3n0ph3n3 3y ago
My biggest issue with Pulumi is that it doesn't have all of the guardrails/tools (like tfsec, tflint, and pavlov) to prevent newbies from doing bad things. I highly considered it for my team's project, but they are not mature enough developers for me to trust them not to massively screw things up or spend forever doing fairly simple things.
- __jem 3y ago
  One of the big benefits of using an actual programming language rather than HCL is that it allows you to define abstractions / use the type system to enforce these invariants. I find this massively preferable to something like a linter in terms of being able to provide a nice API to engineers who might need a little more help defining their resources.
  - x3n0ph3n3 3y ago
    Oh, I agree! I use pulumi for my own personal projects, but I don't have the time to build a DSL on top of it pulumi just to establish the same guardrails my team needs that I'd otherwise get for free.
  - fodkodrasz 3y ago
    Or you could also use the cloud provider's api from a real programming language, or generate terraform code from it. After having tried Pulumi I felt it does not bring the claimed pros, you need to write code in a non-idiomatic way, and coming up with abstractions for the representation of the stuff you talk about takes more time than just write proper terraform, and perform proper thorough reviews, which also facilitate knowledge sharing in the organization. Also then you'd have 3 layers as sources of bugs, instead of just one: your code (you tested it properly, didn't you! How much time saved as opposed to just doing proper reviews!), pulumi itself (has faced its bugs on such a simple task as setting up an aws lambda), and terraform it uses under the hood. In my experience pulumi doesn't solve problems for the business, but lets engineers waste time and money on playing around with their NIH abstractions. We tried it, and dumped it, and went back to terraform, which just works.
  - 3y ago
esafak 3y ago
Is there a clear winner between pulumi and cdktf for new projects?
preommr 3y ago
I really dislike pulumi's pricing structure. I absolutely hate the idea of paying credits for each resource that I create - it creates resistance to experiment and just create a bunch of resources that don't cost anything on the cloud provider yet somehow costs me money for the tool to manage them. It's funny because I believe they used to have a per seat licensing and then people complained that they were paying too much for large orgs with a few resources. Anyways, nowadays I just use aws-cdk. I like the freedom of scripts and the extrme power that comes with them. It is vendor specific, but nothing is really cloud-agnostic, there's always some requirement to have some understanding of the platform's terminology and nuances so this point is pretty moot. It also requires a little bit more planning than just writing something delcarative and letting something figure out the incremental changes, but I like it because it lets me better understand what's happening without runaway complexity.
- __jem 3y ago
  You can use Pulumi for free. Also, the incremental cost for each resource is minuscule. Unless your experiments require thousands of resources, it really shouldn't be a big deterrent. Just clean up when you're done experimenting.
  - Volundr 3y ago
    Sort of. You can use Pulumi for free as an individual. As a team you'd have to use a backend for your state other than Pulumi cloud, and your responsible for your own concurrency control [1]. In a team environment that's such a big foot gun I don't really consider it usable. That limitation makes Pulumi a nonstarter for me. Even if I am paying for it, this is super sensitive data and I want control over how it's stored. I'm sure Pulumi cloud is excellent, but I hate it's the only real option. [1] https://www.pulumi.com/docs/concepts/state/ https://www.pulumi.com/docs/concepts/state/
    - renewiltord 3y ago
      The Terraform equivalent is pretty good. You can just use DynamoDB for concurrency control, S3 for state control, and it's very easy to set up. I imagine Pulumi is not very different on this front. Very little boiler plate and you're off to the races.
irjustin 3y ago
Honest question, if you're locked in to AWS why not just use the CDK? I've used Terraform and Pulumi in the past and the "platform agnostic" is only true for trivial builds. Once you get into medium sized infra you're writing so much AWS specific code it stands better to go 1st party. I can't speak for GCP, but this is what happened to us. We kept fighting 3rd party code we ended up going CDK. While there are still issues, there were less issues. Cloudformation manages the state under the hood anyway so we're all just stuck wrapping that sucker, even in the CDK.
- latchkey 3y ago
  CDK is the correct answer.
- fcarletti 3y ago
  It's not about being vendor agnostic, but being able to manage resources outside of AWS. in my use cases, our platform integrated GitHub, Auth0, AWS, kubernetes and so on. Everything managed via Pulumi, what would not be feasible if we were just on Cloudformation. When trying to achieve a cloud agnostic approach, there are some other tools focused on that (such as serverless framework).
- anuraaga 3y ago
  I have found Terraform or Pulumi to be very convenient even with a single cloud because they offer many cloud-agnostic features as well. - random provider to generate a db password, cloud provider to provision db and admin user, random and mysql providers to provision additional non-admin users, k8s provider to upload credentials to secrets - tls provider to create ca, k8s provider to create namespace, create certs for each k8s namespace, upload to k8s secrets - Cloud provider to issue service account key, GitHub provider to upload to GHA (don't do this anymore since oidc is supported, before it was rather important) While I haven't used CDK much, I believe it is still basically about provisioning aws resources and would not have such cross cutting configs, though I think they may have had a mechanism for importing tf providers. Terraform can have its issues but overall being able to provision such diverse resources with a single command has been great for onboarding/reducing human error. I'm sure there are plenty of cases where CDK is easier too, these are just to demonstrate why you may use TF even when locked in.
jryan49 3y ago
Isn't pulumi aws just terraform under the hood still? Why wouldn't I just use terraform's CDKTF instead?
- mzi 3y ago
  Yes, pulumi is just wrapping terraform[1]. So you need to understand both the quirks of that and the quirks of pulumi. And I'm lazy so I just want to deal with one quirk at the time. [1] https://github.com/pulumi/pulumi-terraform-bridge https://github.com/pulumi/pulumi-terraform-bridge
- re-thc 3y ago
  CDKTF feels like a giant hack (yes, used it). No everything is available vs TF HCL, it uses CDK from AWS CDK and tries to bridge things to TF have codegen. There’s a lot of weird corner cases and inconveniences. It even uses React under the hood. Pulumi at least takes care of it better. Maybe not perfectly but better.
- sigwinch28 3y ago
  >Isn't pulumi aws just terraform under the hood still? It depends. The AWS "Classic" provider uses the terraform provider [1]. The AWS "Native" provider does not, and instead uses the AWS Cloud Control API [2]. [1]: https://github.com/pulumi/pulumi-aws https://github.com/pulumi/pulumi-aws [2]: https://github.com/pulumi/pulumi-aws-native https://github.com/pulumi/pulumi-aws-native
madduci 3y ago
Does anyone know if there's the inverse process available, from Pulumi to Terraform?
worldsayshi 3y ago
I love the idea of IAC as actual code. Too many times I've looked at yet another over engineered templating engine and thought, why couldn't this just be code?
- rbanffy 3y ago
  The reason for not doing a full programming language for that is that you can prove the infrastructure is finite with a simple template language, but you can’t if your template language can do conditional branching. Or something like that. My CS-fu is admittedly weak. OTOH, I think Lisp would be the perfect choice for IaC.
  - worldsayshi 3y ago
    It's a valid point. Although I think most of these templating languages are probably Turing complete anyway. We should make better use of Turing limited languages. I think reinventing the wheel every time a for loop is needed in a resource spec makes things unnecessarily tedious for everyone.
    - rbanffy 3y ago
      IIRC, Terraform was not Turing-complete at the start, but users demanded features and that’s how it ended. The idea to start from a programming language is that, since you are going to end up Turing-complete anyway, at least you make sure you don’t end up with a programming language with an awkward syntax.
rbanffy 3y ago
I really like the idea of using a proper programming language to describe infrastructure, but Pulumi isn’t quite what I thought it could be.
el-duderino42 3y ago
If devs write and maintain their own IAC then Pulumi is a good choice since they can write it in a familiar language. If however a dedicated devops team has to manage it eventually, and there are multiple dev teams who all use different languages, then Terraform is better, since it’s practically impossible to learn all these languages. Terraform will give you a dumbed down enough lingua franca. Also Terraform gives you the advantage to keep code complexity to a minimum. No chance to find some “clever” self referencing function or other language specific crazy stuff to solve a problem in it, it forces you to keep it simple.
- re-thc 3y ago
  > If however a dedicated devops team has to manage it eventually, and there are multiple dev teams who all use different languages How is this a disadvantage? If you can mandate Terraform and HCL across the org you can mandate Pulumi and a language eg Typescript. What’s the difference? You can also use Terraform with a programming language ie CDKTF so the Terraform choice doesn’t make it simpler.
  - el-duderino42 3y ago
    Under the case that “there are multiple dev teams who all use different languages” as stated, they have the advantage with TF that it’s an easier to learn DSL than Typescript. CDKTF is totally optional and besides the point. Could as well say you can use Pulumi with Makefiles so the Pulumi choice doesn’t make it simpler.
    - re-thc 3y ago
      > CDKTF is totally optional and besides the point. Eh? Then Terraform is optional too? You can do the reverse and say HCL is optional. Who decides? You're thinking from a Terraform 1st perspective. > Could as well say you can use Pulumi with Makefiles so the Pulumi choice doesn’t make it simpler. Except Pulumi doesn't support makefiles? It does support yaml. So yaml is a simpler DSL that people know than HSL, so it still wins by your logic? Makes no sense. UI is simpler to learn than Terraform. Might as well do ClickOps? > “there are multiple dev teams who all use different languages” as stated And so you can't pick a language that is used more often? It doesn't have to be Typescript. So if >30% of the devs use 1 language it already has that advantage over HCL.
Cloudef 3y ago
I use terranix to manage AWS https://terranix.org/ https://terranix.org/
- mdaniel 3y ago
  cute: https://github.com/terranix/terranix-examples/blob/main/aws-nixos-server/shell.nix#L5-L8 https://github.com/terranix/terranix-examples/blob/main/aws-... Also: don't use static AWS creds, kids
pachico 3y ago
We use Terraform but mostly because of its ecosystem. I was always tempted to use Pulumi, especially because they got something right from the beginning: pure declarative code is an illusion. You always need loops, control structures, etc. and this is automatically a feature in Pulumi as it is in the language you use it with. Terraform users hack it all the time (think of the "count" as if/else, for instance) precisely because it's never a purely a declarative config what you need, and the level of dynamism you need is not been semantically supported by Terraform. So, we are stuck with hacks...

79 points by hekul 3y ago

https://pixelastic.github.io/pokemonorbigdata/ https://pixelastic.github.io/pokemonorbigdata/

Do these guys need a state file like terraform does? It's great that you can use a normal programming language, but that's nowhere near as big a pain point as state files getting out of sync. Not even close.

rjbwork 3y ago
I'm pretty sure they do, which is a feature, IMO. We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. We also just recently started setting up a periodic drift detection build to help identify and address drift. Could you elaborate on the circumstances that have caused your state sync issues?
- s_Hogg 3y ago
  Yeah we have a 10Mb state file and it already takes 30-45 min to deploy. The main way it happens is if someone cancels a CI/CD job because they're realised something is wrong and don't want to wait that long to try again. That's not an issue if there isn't a state file, but yeah I can see how there would be a tradeoff possibly associated.
  - thedougd 3y ago
    IME the plan times explode because the refreshes hit the cloud provider API rate limits.
  - Aeolun 3y ago
    Pulumi is pretty good about keeping track of in-progress state. If you cancel and restart a new job, it’ll first ask you what to do with the in-progress resources.
  - dronacharya 3y ago
    That does seem like quite a large state file. Would you be able to shed some light on the sorts of of resources that you're provisioning? I've found it to be useful to split dependencies between cloud resources, and to add links via data lookups. At any reasonable scale, I've found splitting Terraform state to be crucial. For example: network resources and Kubernetes clusters are created separately, and networking is a required dependency for Kubernetes clusters. Resources are associated with environments and regions, and modules take in environments and regions as parameters. In my setup, we have 200+ state files, and the largest is around 80KB.
    - rjbwork 3y ago
      How do you manage the meta DAG between all of that stuff in the event of, e.g. complete disaster recovery?
- throwdbaaway 3y ago
  > We're coming up on 10000 resources in our main Terraform repository and while there is definitely some friction, it's overall much better than having to hit the cloud API's to gather each of those states which would probably take at least an order of magnitude longer. I don't think that's necessary true. Most cloud API's actually can return hundreds of records with 1 API calls, e.g. https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_DescribeTargetGroups.html https://docs.aws.amazon.com/elasticloadbalancing/latest/APIR... has a maximum page size of 400. If I manage the cloud resources via some custom tools and/or with some ansible-fu, I can decide to batch the API calls when it makes sense. With terraform, it is not possible to do so (https://github.com/hashicorp/terraform-plugin-sdk/issues/66 https://github.com/hashicorp/terraform-plugin-sdk/issues/66, https://github.com/hashicorp/terraform-provider-aws/issues/2553 https://github.com/hashicorp/terraform-provider-aws/issues/2...).
nl 3y ago
Yes they do. You can refresh the state file quite easily using the `pulimi refresh` command. I'm not really sure how you'd do it without a state file though? State drift is very common and without an idea of the current state it's pretty difficult for any framework to work out what updates need to be applied.
esafak 3y ago
Does pulumi let you write a state reader, so you can add schemas or some kind of validation?
paulgb 3y ago
I’ve been pretty happy with Pulumi but have learned that state management can be a pain. I got myself into a situation recently that I could only get out of by using the CLI interactively. I wound up with multiple copies of each resource, which shared a URN, so when I tried to delete them from the CLI it would always prompt me for which instance of that URN to delete. I ended up spending much of a day writing a program to call their CLI and then interact with it programmatically because I had hundreds of resources to delete. Since then I’ve been doing more with CloudFormation directly and am tempted to switch.
fodkodrasz 3y ago
The claimed pro of being able to use a normal programming language fades away quickly as you are forced into non-idiomatic boilerplate patterns to get pulumi to work. State is a problem and have had just the same kind of hardships when had to manually modify it on some changes as I would have had on terraform. (transfer resources between state stores as a business function is handed over within company, with zero downtime, import docs are not always correct/up2date just like in case of terraform, as if the example codes were generated by chatgpt) Overall pulumi couldn't convince me, it wasn't a bit more convenient than terraform, yet I could bump into its bugs for pretty basic aws features (around lambdas/apigw, but was more than a year ago, might be fixed already) which did work just fine with terraform ootb, and the pulumi code felt bad to look at despite being in a language I generally like. It did not bring the effect management wanted that developers with no terraform knowledge can be onboarded to infra, as of course infra knowledge is needed and is harder to pick up than learning terraform syntax. Despite my dislike for terraform and its syntax's limitations in expressiveness, I'm still back to TF, and is more productive than pulumi was.

tjasdfhwd___ 3y ago

I've been happy with Hashicorp Terraform Cloud and HCL except for the very recent price hikes which may force us to leave them so we are keeping our eyes out for alternatives. The AI assistant seen here is a very good idea. I will actually share this with my team. I imagine the terraform conversion doesn't work when you use in-house written providers (in golang), is this correct?

jaxxstorm 3y ago
You would need to bridge the Terraform provider to a Pulumi provider first. If you've written a Terraform provider, bridging to a Pulumi provider should be straightforward. If you need help with this, please reach out to me on the Pulumi community slack at https://slack.pulumi.com https://slack.pulumi.com and happy to help
tedivm 3y ago
Fortunately there are a lot of other terraform deployment and state management systems, such as Spacelift and Scalr.
e12e 3y ago
We're building our new infrastructure with Terraform - and experimenting with postgresql for state. So far so good, but we have only a handful of resources. Bit of a catch-22 to setup managed postgres for state, tailscale network with a vm as exit-node (to access postgres via VPN/lan - not Internet) - via Terraform - and then switch state from local (initial setup) to postgres. But nice to be able to eg: interact with a different provider for DNS, supplying IPs from other resources.
cube2222 3y ago
Hey! I recommend checking out Spacelift[0]. It's a CI/CD system specialized for Infra as Code, including Terraform and Pulumi. We have both a ton of additional features, as well as a much more reasonable pricing model, so we've recently been talking a lot to folks wanting to migrate from TFC due to the changes you've mentioned. You can easily reach out to us on the website by scheduling a demo, using the chat widget, or just sending me an email (see my profile). [0]: https://spacelift.io https://spacelift.io Disclaimer: Software Engineering Team Lead at Spacelift, so take this with a healthy grain of salt, but I do legitimately think it's a great product.

nl 3y ago

I've used Pulumi for a few small projects. I quite like it and I can see a Terraform importer being useful (for example GCP provides comprehensive Terraform support and being able to convert those to Pulumi quickly would speed things up a lot).

x3n0ph3n3 3y ago

My biggest issue with Pulumi is that it doesn't have all of the guardrails/tools (like tfsec, tflint, and pavlov) to prevent newbies from doing bad things. I highly considered it for my team's project, but they are not mature enough developers for me to trust them not to massively screw things up or spend forever doing fairly simple things.

__jem 3y ago
One of the big benefits of using an actual programming language rather than HCL is that it allows you to define abstractions / use the type system to enforce these invariants. I find this massively preferable to something like a linter in terms of being able to provide a nice API to engineers who might need a little more help defining their resources.
- x3n0ph3n3 3y ago
  Oh, I agree! I use pulumi for my own personal projects, but I don't have the time to build a DSL on top of it pulumi just to establish the same guardrails my team needs that I'd otherwise get for free.
- fodkodrasz 3y ago
  Or you could also use the cloud provider's api from a real programming language, or generate terraform code from it. After having tried Pulumi I felt it does not bring the claimed pros, you need to write code in a non-idiomatic way, and coming up with abstractions for the representation of the stuff you talk about takes more time than just write proper terraform, and perform proper thorough reviews, which also facilitate knowledge sharing in the organization. Also then you'd have 3 layers as sources of bugs, instead of just one: your code (you tested it properly, didn't you! How much time saved as opposed to just doing proper reviews!), pulumi itself (has faced its bugs on such a simple task as setting up an aws lambda), and terraform it uses under the hood. In my experience pulumi doesn't solve problems for the business, but lets engineers waste time and money on playing around with their NIH abstractions. We tried it, and dumped it, and went back to terraform, which just works.
- 3y ago

esafak 3y ago

Is there a clear winner between pulumi and cdktf for new projects?

preommr 3y ago

I really dislike pulumi's pricing structure. I absolutely hate the idea of paying credits for each resource that I create - it creates resistance to experiment and just create a bunch of resources that don't cost anything on the cloud provider yet somehow costs me money for the tool to manage them. It's funny because I believe they used to have a per seat licensing and then people complained that they were paying too much for large orgs with a few resources. Anyways, nowadays I just use aws-cdk. I like the freedom of scripts and the extrme power that comes with them. It is vendor specific, but nothing is really cloud-agnostic, there's always some requirement to have some understanding of the platform's terminology and nuances so this point is pretty moot. It also requires a little bit more planning than just writing something delcarative and letting something figure out the incremental changes, but I like it because it lets me better understand what's happening without runaway complexity.

__jem 3y ago
You can use Pulumi for free. Also, the incremental cost for each resource is minuscule. Unless your experiments require thousands of resources, it really shouldn't be a big deterrent. Just clean up when you're done experimenting.
- Volundr 3y ago
  Sort of. You can use Pulumi for free as an individual. As a team you'd have to use a backend for your state other than Pulumi cloud, and your responsible for your own concurrency control [1]. In a team environment that's such a big foot gun I don't really consider it usable. That limitation makes Pulumi a nonstarter for me. Even if I am paying for it, this is super sensitive data and I want control over how it's stored. I'm sure Pulumi cloud is excellent, but I hate it's the only real option. [1] https://www.pulumi.com/docs/concepts/state/ https://www.pulumi.com/docs/concepts/state/
  - renewiltord 3y ago
    The Terraform equivalent is pretty good. You can just use DynamoDB for concurrency control, S3 for state control, and it's very easy to set up. I imagine Pulumi is not very different on this front. Very little boiler plate and you're off to the races.

irjustin 3y ago

Honest question, if you're locked in to AWS why not just use the CDK? I've used Terraform and Pulumi in the past and the "platform agnostic" is only true for trivial builds. Once you get into medium sized infra you're writing so much AWS specific code it stands better to go 1st party. I can't speak for GCP, but this is what happened to us. We kept fighting 3rd party code we ended up going CDK. While there are still issues, there were less issues. Cloudformation manages the state under the hood anyway so we're all just stuck wrapping that sucker, even in the CDK.

latchkey 3y ago
CDK is the correct answer.
fcarletti 3y ago
It's not about being vendor agnostic, but being able to manage resources outside of AWS. in my use cases, our platform integrated GitHub, Auth0, AWS, kubernetes and so on. Everything managed via Pulumi, what would not be feasible if we were just on Cloudformation. When trying to achieve a cloud agnostic approach, there are some other tools focused on that (such as serverless framework).
anuraaga 3y ago
I have found Terraform or Pulumi to be very convenient even with a single cloud because they offer many cloud-agnostic features as well. - random provider to generate a db password, cloud provider to provision db and admin user, random and mysql providers to provision additional non-admin users, k8s provider to upload credentials to secrets - tls provider to create ca, k8s provider to create namespace, create certs for each k8s namespace, upload to k8s secrets - Cloud provider to issue service account key, GitHub provider to upload to GHA (don't do this anymore since oidc is supported, before it was rather important) While I haven't used CDK much, I believe it is still basically about provisioning aws resources and would not have such cross cutting configs, though I think they may have had a mechanism for importing tf providers. Terraform can have its issues but overall being able to provision such diverse resources with a single command has been great for onboarding/reducing human error. I'm sure there are plenty of cases where CDK is easier too, these are just to demonstrate why you may use TF even when locked in.

jryan49 3y ago

Isn't pulumi aws just terraform under the hood still? Why wouldn't I just use terraform's CDKTF instead?

mzi 3y ago
Yes, pulumi is just wrapping terraform[1]. So you need to understand both the quirks of that and the quirks of pulumi. And I'm lazy so I just want to deal with one quirk at the time. [1] https://github.com/pulumi/pulumi-terraform-bridge https://github.com/pulumi/pulumi-terraform-bridge
re-thc 3y ago
CDKTF feels like a giant hack (yes, used it). No everything is available vs TF HCL, it uses CDK from AWS CDK and tries to bridge things to TF have codegen. There’s a lot of weird corner cases and inconveniences. It even uses React under the hood. Pulumi at least takes care of it better. Maybe not perfectly but better.
sigwinch28 3y ago
>Isn't pulumi aws just terraform under the hood still? It depends. The AWS "Classic" provider uses the terraform provider [1]. The AWS "Native" provider does not, and instead uses the AWS Cloud Control API [2]. [1]: https://github.com/pulumi/pulumi-aws https://github.com/pulumi/pulumi-aws [2]: https://github.com/pulumi/pulumi-aws-native https://github.com/pulumi/pulumi-aws-native

madduci 3y ago

Does anyone know if there's the inverse process available, from Pulumi to Terraform?

worldsayshi 3y ago

I love the idea of IAC as actual code. Too many times I've looked at yet another over engineered templating engine and thought, why couldn't this just be code?

rbanffy 3y ago
The reason for not doing a full programming language for that is that you can prove the infrastructure is finite with a simple template language, but you can’t if your template language can do conditional branching. Or something like that. My CS-fu is admittedly weak. OTOH, I think Lisp would be the perfect choice for IaC.
- worldsayshi 3y ago
  It's a valid point. Although I think most of these templating languages are probably Turing complete anyway. We should make better use of Turing limited languages. I think reinventing the wheel every time a for loop is needed in a resource spec makes things unnecessarily tedious for everyone.
  - rbanffy 3y ago
    IIRC, Terraform was not Turing-complete at the start, but users demanded features and that’s how it ended. The idea to start from a programming language is that, since you are going to end up Turing-complete anyway, at least you make sure you don’t end up with a programming language with an awkward syntax.

rbanffy 3y ago

I really like the idea of using a proper programming language to describe infrastructure, but Pulumi isn’t quite what I thought it could be.

el-duderino42 3y ago

If devs write and maintain their own IAC then Pulumi is a good choice since they can write it in a familiar language. If however a dedicated devops team has to manage it eventually, and there are multiple dev teams who all use different languages, then Terraform is better, since it’s practically impossible to learn all these languages. Terraform will give you a dumbed down enough lingua franca. Also Terraform gives you the advantage to keep code complexity to a minimum. No chance to find some “clever” self referencing function or other language specific crazy stuff to solve a problem in it, it forces you to keep it simple.

re-thc 3y ago
> If however a dedicated devops team has to manage it eventually, and there are multiple dev teams who all use different languages How is this a disadvantage? If you can mandate Terraform and HCL across the org you can mandate Pulumi and a language eg Typescript. What’s the difference? You can also use Terraform with a programming language ie CDKTF so the Terraform choice doesn’t make it simpler.
- el-duderino42 3y ago
  Under the case that “there are multiple dev teams who all use different languages” as stated, they have the advantage with TF that it’s an easier to learn DSL than Typescript. CDKTF is totally optional and besides the point. Could as well say you can use Pulumi with Makefiles so the Pulumi choice doesn’t make it simpler.
  - re-thc 3y ago
    > CDKTF is totally optional and besides the point. Eh? Then Terraform is optional too? You can do the reverse and say HCL is optional. Who decides? You're thinking from a Terraform 1st perspective. > Could as well say you can use Pulumi with Makefiles so the Pulumi choice doesn’t make it simpler. Except Pulumi doesn't support makefiles? It does support yaml. So yaml is a simpler DSL that people know than HSL, so it still wins by your logic? Makes no sense. UI is simpler to learn than Terraform. Might as well do ClickOps? > “there are multiple dev teams who all use different languages” as stated And so you can't pick a language that is used more often? It doesn't have to be Typescript. So if >30% of the devs use 1 language it already has that advantage over HCL.

Cloudef 3y ago

I use terranix to manage AWS https://terranix.org/ https://terranix.org/

mdaniel 3y ago
cute: https://github.com/terranix/terranix-examples/blob/main/aws-nixos-server/shell.nix#L5-L8 https://github.com/terranix/terranix-examples/blob/main/aws-... Also: don't use static AWS creds, kids

pachico 3y ago

We use Terraform but mostly because of its ecosystem. I was always tempted to use Pulumi, especially because they got something right from the beginning: pure declarative code is an illusion. You always need loops, control structures, etc. and this is automatically a feature in Pulumi as it is in the language you use it with. Terraform users hack it all the time (think of the "count" as if/else, for instance) precisely because it's never a purely a declarative config what you need, and the level of dynamism you need is not been semantically supported by Terraform. So, we are stuck with hacks...