7 ms·

It's not about the specific resources for the most part, it's about knowing what needs replacing and why things are where they are, all while keeping your migra

It's not about the specific resources for the most part, it's about knowing what needs replacing and why things are where they are, all while keeping your migration under version control. Also to be fair, with good enough module abstractions and generic enough problems you can completely replace their implementation with few changes to their call sites for things like: - A module that fully sets up a static site with DNS, CDN on an S3-compatible service - A module for setting up an OCI image repo, and spinning resources that runs an image from it as a daemon somewhere. - A module for setting up an OCI image repo, and sets up a "cron" job to run a binary off of it. Sure the migration is unlikely to be as clean as terra[grunt|form] down && terra[grunt|form] up but it can get close.

icedchai 4y ago
This sounds good in theory. No terraform code I've seen has had that level of abstraction. It's basically copy-pasta everywhere! Maybe I've just had bad luck taking over poorly done, pre-existing projects. Could you point me to a well written example?
- baobob 4y ago
  There is a lot to be said for just having 100% Terraform defs for everything, sure you rewrite the file from scratch to move to a different cloud, but you know what needs to be in the file
- mixedCase 4y ago
  I've written code like that for clients (and specifically executed a transition from bare metal+shell scripts to EKS for a "generic backend" module once), but I can't publish any of it for unfortunate obvious reasons. But FWIW I think it's inherent to IaC to involve a lot of copy-pasta, sometimes your best abstraction will just lead to shifting the copy-pasta from code to call-sites, which tends to be more maintainable but still. What I can recommend is to treat your modules for what they are: Public library functions. Meaning common sense advice for those applies: - Approach writing modules by actually designing an API for it. Don't treat it as a folder to dump resources in. - Make your API as small as you can get away with. - Its variables and outputs should be, whenever possible, describe a generic black box that could have its internals swapped. - Make modules composable where you can. Being able to pass around entire module instances in variables because they have clean, stable APIs is very nice. - Accept some things will inevitably leak but put in the effort to minimize them. This blog post and the book from the authors (Terraform Up & Running) has some good ideas that got me started with the basics, but there's not much written with regards to more involved usecases: https://blog.gruntwork.io/how-to-create-reusable-infrastructure-with-terraform-modules-25526d65f73d https://blog.gruntwork.io/how-to-create-reusable-infrastruct...
  - icedchai 4y ago
    thank you. That link looks interesting!
  - shyn3 4y ago
    I like your advice but this misses the MOAT of the cloud which is being able to utilize the IaaS/Papas they provide. For example if you are building in azure today going with SQL doesn't make sense because Cosmos and service bus/event grids handle that. This locks you into the vendor bad makes it near impossible to leave.
    - icedchai 4y ago
      That's the real cost of cloud: lock in. It's cheap to get in, expensive to get out. Example: You get into data into DynamoDB, it's going to be very difficult to migrate out. Consider all your code is likely using AWS-specific APIs, unless your devs were smart enough to build some sort of abstraction. (I've only seen that done right once.)
      - shyn3 4y ago
        I don't think it makes sense to use the abstraction. The cloud is not worth it for that IMO. If you are going to do that using a traditional data center model makes more sense otherwise you can't realize the full value. Also all the providers will have different services or lack of. Even worst, you use a mix of SaaS to avoid the reliance. Better to pay the operations cost.
        icedchai 4y ago
        As always, it depends. boto3's DynamoDB APIs, for example, feel a bit "meh." It is probably worth wrapping them just for your own sanity.
        shyn3 4y ago
        That's a great point. I look at it from an ops perspective and not the headache the devs deal with. I see where you are coming from. Thanks for the insight.

icedchai 4y ago

This sounds good in theory. No terraform code I've seen has had that level of abstraction. It's basically copy-pasta everywhere! Maybe I've just had bad luck taking over poorly done, pre-existing projects. Could you point me to a well written example?

baobob 4y ago
There is a lot to be said for just having 100% Terraform defs for everything, sure you rewrite the file from scratch to move to a different cloud, but you know what needs to be in the file
mixedCase 4y ago
I've written code like that for clients (and specifically executed a transition from bare metal+shell scripts to EKS for a "generic backend" module once), but I can't publish any of it for unfortunate obvious reasons. But FWIW I think it's inherent to IaC to involve a lot of copy-pasta, sometimes your best abstraction will just lead to shifting the copy-pasta from code to call-sites, which tends to be more maintainable but still. What I can recommend is to treat your modules for what they are: Public library functions. Meaning common sense advice for those applies: - Approach writing modules by actually designing an API for it. Don't treat it as a folder to dump resources in. - Make your API as small as you can get away with. - Its variables and outputs should be, whenever possible, describe a generic black box that could have its internals swapped. - Make modules composable where you can. Being able to pass around entire module instances in variables because they have clean, stable APIs is very nice. - Accept some things will inevitably leak but put in the effort to minimize them. This blog post and the book from the authors (Terraform Up & Running) has some good ideas that got me started with the basics, but there's not much written with regards to more involved usecases: https://blog.gruntwork.io/how-to-create-reusable-infrastructure-with-terraform-modules-25526d65f73d https://blog.gruntwork.io/how-to-create-reusable-infrastruct...
- icedchai 4y ago
  thank you. That link looks interesting!
- shyn3 4y ago
  I like your advice but this misses the MOAT of the cloud which is being able to utilize the IaaS/Papas they provide. For example if you are building in azure today going with SQL doesn't make sense because Cosmos and service bus/event grids handle that. This locks you into the vendor bad makes it near impossible to leave.
  - icedchai 4y ago
    That's the real cost of cloud: lock in. It's cheap to get in, expensive to get out. Example: You get into data into DynamoDB, it's going to be very difficult to migrate out. Consider all your code is likely using AWS-specific APIs, unless your devs were smart enough to build some sort of abstraction. (I've only seen that done right once.)
    - shyn3 4y ago
      I don't think it makes sense to use the abstraction. The cloud is not worth it for that IMO. If you are going to do that using a traditional data center model makes more sense otherwise you can't realize the full value. Also all the providers will have different services or lack of. Even worst, you use a mix of SaaS to avoid the reliance. Better to pay the operations cost.
      - icedchai 4y ago
        As always, it depends. boto3's DynamoDB APIs, for example, feel a bit "meh." It is probably worth wrapping them just for your own sanity.
        shyn3 4y ago
        That's a great point. I look at it from an ops perspective and not the headache the devs deal with. I see where you are coming from. Thanks for the insight.