4 ms·

Terraform Cloud costs are an absolute joke. I was actually the decision maker a couple of years ago when we chose our IAC stack and decided not to go with Terra

Terraform Cloud costs are an absolute joke. I was actually the decision maker a couple of years ago when we chose our IAC stack and decided not to go with Terraform Cloud despite thinking the product is strong, and it was entirely due to the business model and cost. For us (and I'm guessing for many VC funded companies), SAAS managed IAC is a nice to have, but certainly not a MUST-have and certainly not something in the same "willing to spend money range" as your monitoring or your main cloud hosting costs. I see these kind of tools as a tier below your monitoring tools like DataDog/Splunk etc. And these tools are a tier below your AWS/GCP costs. If your IAC or CI costs are approaching or overtaking your monitoring costs, something weird is afoot. Likewise, if your Datadog costs are approaching your AWS bill, this is obviously wrong. Hashicorp, in my opinion, thinks their tool is more mission-critical and more of a value-add than in reality it actually is, and I think they also don't understand that in the current high interest rate environment, companies are FAR more willing to put in engineering time to do migrations or money saving projects. My own company put in 100s of hours of Engineering time to reduce the Datadog bill by roughly 40-50%.

mathverse 3y ago
I think Terraform Cloud (and most Hashi's enterprise offerings) aim at absolutel behemoths of deployments with many, many infra teams where the complexity comes from scale and the companies are not Google or Facebook and therefore this problem can't be solved through talent. For many such enterprises it's easier to throw money at the problem.
- candiddevmike 3y ago
  Then they should price them with smaller margins and make it up with volume.
  - ghaff 3y ago
    Although it's certainly not always the right strategy, it's certainly a valid strategy to serve a smaller number of relatively price-insensitive customers (and provide them with ancillary services/high-touch sales/etc. that they want). The fact that you may not personally be on the market for that sort of thing doesn't mean there isn't a market.
    - bandrami 3y ago
      In particular think major regional non-technical companies, like grocery store chains or logistics companies or hospital networks. Beyond basic help desk support none of their IT is in-house and they don't want it to be in-house any more than an IT company wants to run an MRI machine for its employees, so they have dedicated a block of money to making that problem go away and don't really care how much it all costs as long as it comes in under that line.
- noirbot 3y ago
  You would think that, but having just went through trying to implement Terraform Cloud at work, its tools for doing anything that approaches behemoth deployment are abysmal. Permissions management is a nightmare and any sort of orchestration between different layers is barely there.
- tedivm 3y ago
  Except those same companies then run into hidden limits that exist on Terraform Cloud and hadn't been previously mentioned. Honestly, even bigger companies are better off going with Spacelift as it can scale extremely well, has reasonable support, and is much more feature rich. As a disclaiming I'm currently writing a book on Terraform, and I've been interacting with a lot of the people in the space. Up until a few years ago I was a huge fan of Hashicorp, but their price changes and lack of support were what made it so I couldn't recommend Terraform Cloud anymore. The license changes they made were the icing on that cake.
nprateem 3y ago
It is mission critical, just not the SaaS version.
teaearlgraycold 3y ago
I personally like what render.com is doing with IaC through their blueprints format. It's definitely a kid's toy version of IaC, but it's a really nice step up from Heroku.
- latchkey 3y ago
  It appears to be proprietary.
  - teaearlgraycold 3y ago
    Yes. Still a great tool for small dev shops
    - latchkey 3y ago
      Why would you pick a proprietary solution for a small dev shop?
      - teaearlgraycold 3y ago
        Less people available to own self-hosted solutions. Proprietary also can mean a well-funded product with money allocated to DevEx and ease-of-use.
        latchkey 3y ago
        > Less people available to own self-hosted solutions. There are other ways that are not proprietary and don't require self-hosting. > Proprietary also can mean a well-funded product with money allocated to DevEx and ease-of-use. How did that work out for Heroku?
        glenngillen 3y ago
        I wouldn’t lay blame for Heroku’s financial problems at the feet of architecture choices it made customers adopt. Also, Heroku’s buildpacks were anything but proprietary. They’ve been adopted by a bunch of other products.
paholg 3y ago
At a prior company, we had a terraform slack channel, where people would post lock and unlock emojis and apply terraform manually. Obviously, it's not a perfect system, and it doesn't indefinitely scale, but it worked well enough.
- jlund-molfese 3y ago
  We use Atlantis [0] for CI/CD automation of Terraform pull requests to a centralized repository. It's pretty good too, especially for a self-hosted solution. I can't see how Terraform Cloud's costs would be justifiable for us without a custom contract. [0] https://www.runatlantis.io/ https://www.runatlantis.io/
  - rjst01 3y ago
    At my last gig we built a bespoke version of this on top of our CI provider. Now i'm a technical co-founder in a startup and this is something I haven't quite solved yet, and this looks like exactly what we're after. Thanks!
- hypeatei 3y ago
  You can setup remote backends which support locking (e.g. Azure storage, Amazon S3, etc..) that way it's automatic. You can see a list on the left-hand side here: https://developer.hashicorp.com/terraform/language/settings/backends/remote https://developer.hashicorp.com/terraform/language/settings/...
- rollcat 3y ago
  StackOverflow famously manages database migrations this way; you post a message in a shared channel and get the next revision number.
- flumpcakes 3y ago
  How does this work? I thought the terraform state file was the single source of truth - if people are applying terraform 'manually' I assume that means on their local device? Are people sharing around the state file but don't have a central location for a lock file? Apologises if this seems obvious...
ransom1538 3y ago
Honest question. If you need to pump data into datadog from your cloud, why not just stop doing that, and use the cloud tools? Datadog isnt' creating metrics. The metrics are already on your cloud provider (gcp,aws,etc) -- AND all these providers have ways to view metrics. Why do people pay 20k per month for data dog? They dont' want to use GCP metrics explorer? Same with TF cloud. You can just setup a jenkins job and do all the features TF cloud has in a few hours. Why pay TF cloud? Are people confused? I must be missing something. I worked at startups that thought all metrics need to be in datadog, but they didnt' understand they can just use metrics explorer. You can't even use sql to create graphs in datadog, it's awful. IMHO If you are serious, dump the metrics into bigquery/redshift and start doing sql like an adult.
- claytonjy 3y ago
  I would not consider metrics explorer to be a particularly good product. For queries where the query builder isn't good enough and you write PromQL instead, they don't even let you alias the legend; instead you must see the entire query as the label for that line. A pretty minor nitpick, but indicative of the level of attention they give the product.
- devonbleak 3y ago
  If you start pulling all the metrics we currently use in DataDog out of something like BQ/RedShift you're going to spend WAY more in terms of engineering hours and infrastructure than we're currently paying DataDog. I keep hearing something about a company that switched their monitoring from DataDog to DataBricks and I can see that yes you probably could go build a monitoring solution on top of a datastore like that. But I certainly wouldn't want to.
- viraptor 3y ago
  For viewing, cloudwatch is simply nowhere as convenient, writing whole custom queries takes a lot of time, and setting up altering for just the right thing sometimes requires you to copy the complex result as another metric. Basically I can create a time-shifted difference of two metrics reporting at different intervals, smooth it and make it alert the right teams through slack and pagerduty in a minute or so (while getting a visual feedback on the query the whole time) - this would take a significant amount of time to do through the plain AWS process. It's a bit like "you can write everything you want in assembler without the overhead of extra layers of X". Yeah, sure, I can. But I appreciate the extra layers of X. I'll take the right cost/convenience balance, like an adult, thank you.
deleted 3y ago
[deleted]

mathverse 3y ago

I think Terraform Cloud (and most Hashi's enterprise offerings) aim at absolutel behemoths of deployments with many, many infra teams where the complexity comes from scale and the companies are not Google or Facebook and therefore this problem can't be solved through talent. For many such enterprises it's easier to throw money at the problem.

candiddevmike 3y ago
Then they should price them with smaller margins and make it up with volume.
- ghaff 3y ago
  Although it's certainly not always the right strategy, it's certainly a valid strategy to serve a smaller number of relatively price-insensitive customers (and provide them with ancillary services/high-touch sales/etc. that they want). The fact that you may not personally be on the market for that sort of thing doesn't mean there isn't a market.
  - bandrami 3y ago
    In particular think major regional non-technical companies, like grocery store chains or logistics companies or hospital networks. Beyond basic help desk support none of their IT is in-house and they don't want it to be in-house any more than an IT company wants to run an MRI machine for its employees, so they have dedicated a block of money to making that problem go away and don't really care how much it all costs as long as it comes in under that line.
noirbot 3y ago
You would think that, but having just went through trying to implement Terraform Cloud at work, its tools for doing anything that approaches behemoth deployment are abysmal. Permissions management is a nightmare and any sort of orchestration between different layers is barely there.
tedivm 3y ago
Except those same companies then run into hidden limits that exist on Terraform Cloud and hadn't been previously mentioned. Honestly, even bigger companies are better off going with Spacelift as it can scale extremely well, has reasonable support, and is much more feature rich. As a disclaiming I'm currently writing a book on Terraform, and I've been interacting with a lot of the people in the space. Up until a few years ago I was a huge fan of Hashicorp, but their price changes and lack of support were what made it so I couldn't recommend Terraform Cloud anymore. The license changes they made were the icing on that cake.

nprateem 3y ago

It is mission critical, just not the SaaS version.

teaearlgraycold 3y ago

I personally like what render.com is doing with IaC through their blueprints format. It's definitely a kid's toy version of IaC, but it's a really nice step up from Heroku.

latchkey 3y ago
It appears to be proprietary.
- teaearlgraycold 3y ago
  Yes. Still a great tool for small dev shops
  - latchkey 3y ago
    Why would you pick a proprietary solution for a small dev shop?
    - teaearlgraycold 3y ago
      Less people available to own self-hosted solutions. Proprietary also can mean a well-funded product with money allocated to DevEx and ease-of-use.
      - latchkey 3y ago
        > Less people available to own self-hosted solutions. There are other ways that are not proprietary and don't require self-hosting. > Proprietary also can mean a well-funded product with money allocated to DevEx and ease-of-use. How did that work out for Heroku?
        glenngillen 3y ago
        I wouldn’t lay blame for Heroku’s financial problems at the feet of architecture choices it made customers adopt. Also, Heroku’s buildpacks were anything but proprietary. They’ve been adopted by a bunch of other products.

paholg 3y ago

At a prior company, we had a terraform slack channel, where people would post lock and unlock emojis and apply terraform manually. Obviously, it's not a perfect system, and it doesn't indefinitely scale, but it worked well enough.

jlund-molfese 3y ago
We use Atlantis [0] for CI/CD automation of Terraform pull requests to a centralized repository. It's pretty good too, especially for a self-hosted solution. I can't see how Terraform Cloud's costs would be justifiable for us without a custom contract. [0] https://www.runatlantis.io/ https://www.runatlantis.io/
- rjst01 3y ago
  At my last gig we built a bespoke version of this on top of our CI provider. Now i'm a technical co-founder in a startup and this is something I haven't quite solved yet, and this looks like exactly what we're after. Thanks!
hypeatei 3y ago
You can setup remote backends which support locking (e.g. Azure storage, Amazon S3, etc..) that way it's automatic. You can see a list on the left-hand side here: https://developer.hashicorp.com/terraform/language/settings/backends/remote https://developer.hashicorp.com/terraform/language/settings/...
rollcat 3y ago
StackOverflow famously manages database migrations this way; you post a message in a shared channel and get the next revision number.
flumpcakes 3y ago
How does this work? I thought the terraform state file was the single source of truth - if people are applying terraform 'manually' I assume that means on their local device? Are people sharing around the state file but don't have a central location for a lock file? Apologises if this seems obvious...

ransom1538 3y ago

Honest question. If you need to pump data into datadog from your cloud, why not just stop doing that, and use the cloud tools? Datadog isnt' creating metrics. The metrics are already on your cloud provider (gcp,aws,etc) -- AND all these providers have ways to view metrics. Why do people pay 20k per month for data dog? They dont' want to use GCP metrics explorer? Same with TF cloud. You can just setup a jenkins job and do all the features TF cloud has in a few hours. Why pay TF cloud? Are people confused? I must be missing something. I worked at startups that thought all metrics need to be in datadog, but they didnt' understand they can just use metrics explorer. You can't even use sql to create graphs in datadog, it's awful. IMHO If you are serious, dump the metrics into bigquery/redshift and start doing sql like an adult.

claytonjy 3y ago
I would not consider metrics explorer to be a particularly good product. For queries where the query builder isn't good enough and you write PromQL instead, they don't even let you alias the legend; instead you must see the entire query as the label for that line. A pretty minor nitpick, but indicative of the level of attention they give the product.
devonbleak 3y ago
If you start pulling all the metrics we currently use in DataDog out of something like BQ/RedShift you're going to spend WAY more in terms of engineering hours and infrastructure than we're currently paying DataDog. I keep hearing something about a company that switched their monitoring from DataDog to DataBricks and I can see that yes you probably could go build a monitoring solution on top of a datastore like that. But I certainly wouldn't want to.
viraptor 3y ago
For viewing, cloudwatch is simply nowhere as convenient, writing whole custom queries takes a lot of time, and setting up altering for just the right thing sometimes requires you to copy the complex result as another metric. Basically I can create a time-shifted difference of two metrics reporting at different intervals, smooth it and make it alert the right teams through slack and pagerduty in a minute or so (while getting a visual feedback on the query the whole time) - this would take a significant amount of time to do through the plain AWS process. It's a bit like "you can write everything you want in assembler without the overhead of extra layers of X". Yeah, sure, I can. But I appreciate the extra layers of X. I'll take the right cost/convenience balance, like an adult, thank you.

deleted 3y ago

[deleted]