3 ms·

I hadn’t seen these and had been looking at Terraform Cloud for something recently… and as an existing Terraform user who has had to debug and examine things… t

I hadn’t seen these and had been looking at Terraform Cloud for something recently… and as an existing Terraform user who has had to debug and examine things… that pricing is basically a perverse incentive waiting to happen. I’ve seen a number of terraform modules that try to use many small elements for things like AWS IAM permissions or Kubernetes configMaps, it helps document their individual purpose, it makes the output a bit cleaner sometimes, and until these pricing changes was basically just a style choice… now however it’s going to cost people more… I’d estimate at least a hundred (probably more) unnecessary resources are spread throughout the various custom modules and terraform registry modules in the last terraform repo I looked at… which adds up fast. So moving forward I’d expect to see a slow trend towards features/fixes that function with more resources rather than less… Terraform has become somewhat legendary for its glacial pace of progress (go to GitHub, sort all the issues by thumbs up, have a read, see what gets closed what gets re-requested, what gets ignored with multiple pull requests) … this pricing change doesn’t fill me with confidence in the projects long term future… and neither does laying off staff at a company that already appears to be struggling to support its existing products while simultaneously trying to grow several new ones.

glenngillen 3y ago
Of the thousands of providers that exist for Terraform (where the implementation details you discuss would exist) only a half dozen are maintained by HashiCorp. Usually in some form of partnership with the vendor in question (e.g., AWS). There’s no pricing based incentive structure for the type of sprawl you’re worried about. Not to say the trend you observed won’t continue. Just that if it does it’ll likely be due to composability or maintenance needs of the specific providers rather than juicing pricing.
- techdragon 3y ago
  The sprawl of providers contains a lot of stale forks and copies, and on top of that its not easy to say “let’s just use John Smiths provider to setup the root credentials of our new K8S cluster because Hashicorp aren’t fixing issues we care about. I use a few third party providers and the quality difference is wildly variable, the update cadence is sporadic and unreliable on average, and not wanting to build scripts and tools is why maintaining my own terraform providers forked fro hashicorp isn’t really viable time wise either.
  - glenngillen 3y ago
    Sprawl of providers due to forks for whatever reason is a completely different issue to the resource sprawl you originally called out. The point remains though: there’s no incentive system for the people who make the implementations decisions re resource granularity within providers to increase that because of this pricing change. For the most part they don’t work at HashiCorp. And even when they do, those abstractions are more often due to the underlying API they’re communicating with.
    - techdragon 3y ago
      I think I may have been too subtle with the point about security. Hashicorp’s providers are more trusted because they come from the tool vendor, they are using them in a commercial product and running them on their own hardware as part of terraform cloud. They are all but “implicitly” trusted since you trust Hashicorp code with secrets in order to have Terraform do its job. Yes you can architect a lot of safety layers around credentials and treat Terraform as untrusted, but it’s a sliding scale. There is an incentive for project management on the AWS, GCE, Azure, Kubernetes, and the other Hashicorp maintained providers, to not prioritise work that reduces the number of potentially chargeable resources. The first one I thought of was the time provider. It’s a virtual module like the null provider and all it does is put a logical delay into the dependency chain to handle edge cases… it would be all too easy to start assuming that customers use this module more in order to handle functionality that would require more code in other modules. They probably have metrics on resource and module use via terraform cloud (I don’t have the privacy policy and ToS memorised) How strong the incentive is and if it’s ever really more than a subconscious influence on Hashicorp’s code the code that customers are more likely to use than 3rd party providers… is basically impossible to tell, but the inventive is absolutely there because Hashicorp’s pricing changes have made “number of resources in use by a terraform cloud customers” into a metric that the management will be looking at… the business development, the parts of the company that are responsible for making the money happen, will be measuring this number because it’s obviously important to them now… And once you begin to measure something as a metric the incentive to game the metrics begins.
      - 3y ago

glenngillen 3y ago

Of the thousands of providers that exist for Terraform (where the implementation details you discuss would exist) only a half dozen are maintained by HashiCorp. Usually in some form of partnership with the vendor in question (e.g., AWS). There’s no pricing based incentive structure for the type of sprawl you’re worried about. Not to say the trend you observed won’t continue. Just that if it does it’ll likely be due to composability or maintenance needs of the specific providers rather than juicing pricing.

techdragon 3y ago
The sprawl of providers contains a lot of stale forks and copies, and on top of that its not easy to say “let’s just use John Smiths provider to setup the root credentials of our new K8S cluster because Hashicorp aren’t fixing issues we care about. I use a few third party providers and the quality difference is wildly variable, the update cadence is sporadic and unreliable on average, and not wanting to build scripts and tools is why maintaining my own terraform providers forked fro hashicorp isn’t really viable time wise either.
- glenngillen 3y ago
  Sprawl of providers due to forks for whatever reason is a completely different issue to the resource sprawl you originally called out. The point remains though: there’s no incentive system for the people who make the implementations decisions re resource granularity within providers to increase that because of this pricing change. For the most part they don’t work at HashiCorp. And even when they do, those abstractions are more often due to the underlying API they’re communicating with.
  - techdragon 3y ago
    I think I may have been too subtle with the point about security. Hashicorp’s providers are more trusted because they come from the tool vendor, they are using them in a commercial product and running them on their own hardware as part of terraform cloud. They are all but “implicitly” trusted since you trust Hashicorp code with secrets in order to have Terraform do its job. Yes you can architect a lot of safety layers around credentials and treat Terraform as untrusted, but it’s a sliding scale. There is an incentive for project management on the AWS, GCE, Azure, Kubernetes, and the other Hashicorp maintained providers, to not prioritise work that reduces the number of potentially chargeable resources. The first one I thought of was the time provider. It’s a virtual module like the null provider and all it does is put a logical delay into the dependency chain to handle edge cases… it would be all too easy to start assuming that customers use this module more in order to handle functionality that would require more code in other modules. They probably have metrics on resource and module use via terraform cloud (I don’t have the privacy policy and ToS memorised) How strong the incentive is and if it’s ever really more than a subconscious influence on Hashicorp’s code the code that customers are more likely to use than 3rd party providers… is basically impossible to tell, but the inventive is absolutely there because Hashicorp’s pricing changes have made “number of resources in use by a terraform cloud customers” into a metric that the management will be looking at… the business development, the parts of the company that are responsible for making the money happen, will be measuring this number because it’s obviously important to them now… And once you begin to measure something as a metric the incentive to game the metrics begins.
    - 3y ago