4 ms·

I ditched Terraform years ago and just interact with the raw cloud provider SDKs now. It's much easier to long-term evolve actual code and deal with weird edgec

I ditched Terraform years ago and just interact with the raw cloud provider SDKs now. It's much easier to long-term evolve actual code and deal with weird edgecases that come up when you're not in beholden to the straight jacket that is configuration masquerading as code. Oh yea, and we can write tests for all that provisioning logic too.

plmpsu 2y ago
How are you handling creating multiple resources in parallel? or rolling back changes after an unsuccessful run?
- gorgoiler 2y ago
  Not OP, but for rolling back we just… revert the change to the setup_k8s_stuff.py script ! In practice it’s a module that integrates with quite a large number of things in the monolith because that’s one of the advantages of Infrastructure as Actual Code: symbols and enums and functions that have meaningful semantics in your business logic are frequently useful in your infrastructure logic too. The Apples API runs on the Apples tier, the Oranges API runs on the Oranges tier, etc. etc. People call me old fashioned (“it’s not the 1990s any more”) but when I deploy something it’s a brand new set of instances to which traffic gets migrated. We don’t modify in place with anything clever and I imagine reverting changes in a mutable environment is indeed quite hard to get right (and what you are hinting at?)
  - solatic 2y ago
    > I imagine reverting changes in a mutable environment is indeed quite hard to get right (and what you are hinting at?) I guess you're not managing any databases then? Because you can't just treat those immutably, you have to manage the database in-place.
    - jiggawatts 2y ago
      One thing that annoys me is the inconsistency between mutable "data" resources and everything else. Something that would be nice would be the rough equivalent of the deployment slots used in Azure App Service, but for everything else too. So you could provision a "whole new resource" and then atomically switch traffic over to it.
      - solatic 2y ago
        You can express this in Terraform, it's just a little more contrived. You release your changes as Terraform modules (a module in and of itself doesn't do anything, it's like a library/package), then your Terraform workspace instantiates both a "blue" module and a "green" module, at different versions, with DNS / load balancing resources depending on both modules and switching between either blue or green.
  - michaelmior 2y ago
    > revert the change to the setup_k8s_stuff.py script What about resources that were created by the code you reverted?
    - CoolCold 2y ago
      it's not fair to ask such questions
- inopinatus 2y ago
  A very small shell script.
beacon294 2y ago
I agree that the SDK is better for many use cases. I do like terraform for static resources like aws vpc, networking, s3 buckets, etc.
solatic 2y ago
Terraform added tests somewhat recently: https://developer.hashicorp.com/terraform/language/tests https://developer.hashicorp.com/terraform/language/tests
kikimora 2y ago
I’ve been thinking about this for a long time. But doesn’t it brings a host of other issues? For example, I need to update instance RAM from 4 to 8 Gb but how do I know if the instance exists or should be created? I need to make a small change, how do I know what parts of my scripts to run?
- voidfunc 2y ago
  You write code to do these things? If there's a requirement for you to be able to do such a thing make it a feature, implement it with tests and voila, no different than any other feature or bug you work on is it?
- diggan 2y ago
  > For example, I need to update instance RAM from 4 to 8 Gb but how do I know if the instance exists or should be created? let front_id = if instance_exists("front_balancer") { return fetch_instance("front_balancer").id } else { return create_new_instance("front_balancer", front_balancer_opts).id } Or however else you would manage that sort of thing in your favorite programming language. > I need to make a small change, how do I know what parts of my scripts to run? Either just re-run the parts you know you've changed (manually or based on git diffs), or even better, make the entire thing idempotent and you won't have to care, re-run the entire program after each change and it'll automagically work.
  - michaelmior 2y ago
    > Either just re-run the parts you know you've changed (manually or based on git diffs) This is exactly the sort of thing Terraform is designed to avoid because it can obviously get quite messy. Agreed that making things idempotent solves that problem, but it's not always obvious/easy how to do so.
  - kikimora 2y ago
    I get the idea but I don't think it addresses the issue. There has to be a function that a) checks if instance exists b) checks if instance state is what I want (e.g. it has 8 GB ram) c) if not it updates the instance. Ideally it also locks environment while doing this to prevent race conditions. It can be written but seems to be quite cumbersome. Complexity of this code and also time it takes to run it what concerns me most. I guess this is why terraform state is there. IMHO state is IaaC biggest weakness because you have to keep it consistent with actual cloud state. If we can just query state from the cloud and make it performant + be able to automatically (or just fast enough) select resources to be update it would be ideal.
imp0cat 2y ago
And eventually, you end up with your own in-house Terraform.
evantbyrne 2y ago
I went through the same evolution, even built a PaaS for AWS, but I kept going and now just deploy my own stuff to VMs with Swarm via one command in Rove. It's great. And yes I know kubernetes I use it at work. It's an unnecessary waste of time.
- dijksterhuis 2y ago
  > Swarm docker swarm is so simple and easy compared to the utter behemoth that is k8s, and basically is all you need for CRUD webapps 80-90% of the time. add an RDS instance and you’re set. i will always pick swarm in a small company* whenever possible until k8s or ECS makes sense because something has changed and it’s needed. dont start with complexity. * - bigger companies have different needs.
  - evantbyrne 2y ago
    People have really been sleeping on Swarm. I sometimes even see people trying to recreate Swarm features with Compose. Wish more devs knew about it.

plmpsu 2y ago

How are you handling creating multiple resources in parallel? or rolling back changes after an unsuccessful run?

gorgoiler 2y ago
Not OP, but for rolling back we just… revert the change to the setup_k8s_stuff.py script ! In practice it’s a module that integrates with quite a large number of things in the monolith because that’s one of the advantages of Infrastructure as Actual Code: symbols and enums and functions that have meaningful semantics in your business logic are frequently useful in your infrastructure logic too. The Apples API runs on the Apples tier, the Oranges API runs on the Oranges tier, etc. etc. People call me old fashioned (“it’s not the 1990s any more”) but when I deploy something it’s a brand new set of instances to which traffic gets migrated. We don’t modify in place with anything clever and I imagine reverting changes in a mutable environment is indeed quite hard to get right (and what you are hinting at?)
- solatic 2y ago
  > I imagine reverting changes in a mutable environment is indeed quite hard to get right (and what you are hinting at?) I guess you're not managing any databases then? Because you can't just treat those immutably, you have to manage the database in-place.
  - jiggawatts 2y ago
    One thing that annoys me is the inconsistency between mutable "data" resources and everything else. Something that would be nice would be the rough equivalent of the deployment slots used in Azure App Service, but for everything else too. So you could provision a "whole new resource" and then atomically switch traffic over to it.
    - solatic 2y ago
      You can express this in Terraform, it's just a little more contrived. You release your changes as Terraform modules (a module in and of itself doesn't do anything, it's like a library/package), then your Terraform workspace instantiates both a "blue" module and a "green" module, at different versions, with DNS / load balancing resources depending on both modules and switching between either blue or green.
- michaelmior 2y ago
  > revert the change to the setup_k8s_stuff.py script What about resources that were created by the code you reverted?
  - CoolCold 2y ago
    it's not fair to ask such questions
inopinatus 2y ago
A very small shell script.

beacon294 2y ago

I agree that the SDK is better for many use cases. I do like terraform for static resources like aws vpc, networking, s3 buckets, etc.

solatic 2y ago

Terraform added tests somewhat recently: https://developer.hashicorp.com/terraform/language/tests https://developer.hashicorp.com/terraform/language/tests

kikimora 2y ago

I’ve been thinking about this for a long time. But doesn’t it brings a host of other issues? For example, I need to update instance RAM from 4 to 8 Gb but how do I know if the instance exists or should be created? I need to make a small change, how do I know what parts of my scripts to run?

voidfunc 2y ago
You write code to do these things? If there's a requirement for you to be able to do such a thing make it a feature, implement it with tests and voila, no different than any other feature or bug you work on is it?
diggan 2y ago
> For example, I need to update instance RAM from 4 to 8 Gb but how do I know if the instance exists or should be created? let front_id = if instance_exists("front_balancer") { return fetch_instance("front_balancer").id } else { return create_new_instance("front_balancer", front_balancer_opts).id } Or however else you would manage that sort of thing in your favorite programming language. > I need to make a small change, how do I know what parts of my scripts to run? Either just re-run the parts you know you've changed (manually or based on git diffs), or even better, make the entire thing idempotent and you won't have to care, re-run the entire program after each change and it'll automagically work.
- michaelmior 2y ago
  > Either just re-run the parts you know you've changed (manually or based on git diffs) This is exactly the sort of thing Terraform is designed to avoid because it can obviously get quite messy. Agreed that making things idempotent solves that problem, but it's not always obvious/easy how to do so.
- kikimora 2y ago
  I get the idea but I don't think it addresses the issue. There has to be a function that a) checks if instance exists b) checks if instance state is what I want (e.g. it has 8 GB ram) c) if not it updates the instance. Ideally it also locks environment while doing this to prevent race conditions. It can be written but seems to be quite cumbersome. Complexity of this code and also time it takes to run it what concerns me most. I guess this is why terraform state is there. IMHO state is IaaC biggest weakness because you have to keep it consistent with actual cloud state. If we can just query state from the cloud and make it performant + be able to automatically (or just fast enough) select resources to be update it would be ideal.

imp0cat 2y ago

And eventually, you end up with your own in-house Terraform.

evantbyrne 2y ago

I went through the same evolution, even built a PaaS for AWS, but I kept going and now just deploy my own stuff to VMs with Swarm via one command in Rove. It's great. And yes I know kubernetes I use it at work. It's an unnecessary waste of time.

dijksterhuis 2y ago
> Swarm docker swarm is so simple and easy compared to the utter behemoth that is k8s, and basically is all you need for CRUD webapps 80-90% of the time. add an RDS instance and you’re set. i will always pick swarm in a small company* whenever possible until k8s or ECS makes sense because something has changed and it’s needed. dont start with complexity. * - bigger companies have different needs.
- evantbyrne 2y ago
  People have really been sleeping on Swarm. I sometimes even see people trying to recreate Swarm features with Compose. Wish more devs knew about it.