5 ms·

Infrastructure is quantum state. AWS APIs lie to you. Status pages lie to you. If I had a nickel every time the solution to a failing HTTP call was "just send i

Infrastructure is quantum state. AWS APIs lie to you. Status pages lie to you. If I had a nickel every time the solution to a failing HTTP call was "just send it again"... Use whatever tool you like: SQL, Rust, carrier-pigeon - but it wont be a panacea to solving cloud infrastructure.

rubiquity 5y ago
Everything in the real world is quantum state but that doesn’t stop every SaaS application out there from using an RDBMS as their system of record. This is why we have reconciliation processes. Terraform state files are just an ad-hoc storage format without any of the features that SQL and RDBMS have had for decades (I don’t mean to pick on terraform so much, it’s just the one I’m most familiar with).
- stingraycharles 5y ago
  Like someone else says, SELECTs make sense, INSERT/UPDATE/DELETE to manage infrastructure state, rather than using “proper” infrastructure as code, sounds like a path to hell to me.
  - eloff 5y ago
    Why specifically is that worse than config files?
    - stingraycharles 5y ago
      config files / templates are typically declarative, you tel some orchestrator “this is how I want my infrastructure to look like, please detect changes and make it happen”. it makes it much more easy to reason about things, as state is abstracted away.
      - yawaramin 5y ago
        What's the difference in declarativeness between: - Opening a YAML file in a text editor, deleting some lines, then doing a Kubernetes or Terraform or whatever apply operation - Creating a SQL migration to delete some rows from a table, checking the migration into the repo, and then running a database migration ?
  - dspillett 5y ago
    If properly modelled so the modifying operations are consistent and logical, SQL for modification has the advantage over "proper code to update" has the advantage that in many cases the same query to find something is used with only minor alteration to modify it. Now the real magic, would be to make changes atomic!
    - verdverm 5y ago
      TF has locks so that two people cannot modify infra at the same time. There are also dependencies within the resources to consider, such as network creation before firewall rule before VM
      - yawaramin 5y ago
        Yeah, and Terraform providers can be buggy and actually mess up those dependencies. I ran into this very recently where Terraform planned my changes successfully but failed to delete something because it had a dependency on another thing which it was planning to delete later. It would be very difficult to mess this up with SQL foreign keys.
- oneplane 5y ago
  Unless it can also do state verification, planning and the three-way: desired state, expected state and actual state in one diff, it doesn't hold a candle to Terraform. At the same time this SQL would be useful when used in combination with a classic database migration framework which essentially comes down to a bad version of terraform. Inversely, you could simply use de Terraform CDK and an SQL adapter to write SQL into Terraform definitions. People who want to use SQL can use that, and in the background it's still terraform with all its benefits.
dnautics 5y ago
Everything is "quantum state". You do a database fetch and in the time that the result travels over the wire to the FE another node could have made a mutation. The nice thing is that adapters to sql engines often have exactly the sort of protections you want baked in, e.g. "check to make sure the updated at timestamps match before committing an update"
void_mint 5y ago
I'll bite. What infrastructure management tool do _you_ prefer then?
- tmp_anon_22 5y ago
  Every tool is flawed and picking the right one for your problem is about weighting tradeoffs, that said tools with large well-run communities, the right corporate sponsors and contributors, and enough tenure to round out the sharpest edge cases, are generally "better". So Terraform and Ansible come to mind. Both deeply flawed, but they become the least-worst current option. People should still strive for better, test new ideas with new tools, but choices for production code should play it safe almost every time.
  - void_mint 5y ago
    I feel like the purpose of your posts is more about bemoaning infrastructure tools and less about discussing the original post's product. I'm not really sure it's productive. I definitely do not find it productive.

rubiquity 5y ago

Everything in the real world is quantum state but that doesn’t stop every SaaS application out there from using an RDBMS as their system of record. This is why we have reconciliation processes. Terraform state files are just an ad-hoc storage format without any of the features that SQL and RDBMS have had for decades (I don’t mean to pick on terraform so much, it’s just the one I’m most familiar with).

stingraycharles 5y ago
Like someone else says, SELECTs make sense, INSERT/UPDATE/DELETE to manage infrastructure state, rather than using “proper” infrastructure as code, sounds like a path to hell to me.
- eloff 5y ago
  Why specifically is that worse than config files?
  - stingraycharles 5y ago
    config files / templates are typically declarative, you tel some orchestrator “this is how I want my infrastructure to look like, please detect changes and make it happen”. it makes it much more easy to reason about things, as state is abstracted away.
    - yawaramin 5y ago
      What's the difference in declarativeness between: - Opening a YAML file in a text editor, deleting some lines, then doing a Kubernetes or Terraform or whatever apply operation - Creating a SQL migration to delete some rows from a table, checking the migration into the repo, and then running a database migration ?
- dspillett 5y ago
  If properly modelled so the modifying operations are consistent and logical, SQL for modification has the advantage over "proper code to update" has the advantage that in many cases the same query to find something is used with only minor alteration to modify it. Now the real magic, would be to make changes atomic!
  - verdverm 5y ago
    TF has locks so that two people cannot modify infra at the same time. There are also dependencies within the resources to consider, such as network creation before firewall rule before VM
    - yawaramin 5y ago
      Yeah, and Terraform providers can be buggy and actually mess up those dependencies. I ran into this very recently where Terraform planned my changes successfully but failed to delete something because it had a dependency on another thing which it was planning to delete later. It would be very difficult to mess this up with SQL foreign keys.
oneplane 5y ago
Unless it can also do state verification, planning and the three-way: desired state, expected state and actual state in one diff, it doesn't hold a candle to Terraform. At the same time this SQL would be useful when used in combination with a classic database migration framework which essentially comes down to a bad version of terraform. Inversely, you could simply use de Terraform CDK and an SQL adapter to write SQL into Terraform definitions. People who want to use SQL can use that, and in the background it's still terraform with all its benefits.

dnautics 5y ago

Everything is "quantum state". You do a database fetch and in the time that the result travels over the wire to the FE another node could have made a mutation. The nice thing is that adapters to sql engines often have exactly the sort of protections you want baked in, e.g. "check to make sure the updated at timestamps match before committing an update"

void_mint 5y ago

I'll bite. What infrastructure management tool do _you_ prefer then?

tmp_anon_22 5y ago
Every tool is flawed and picking the right one for your problem is about weighting tradeoffs, that said tools with large well-run communities, the right corporate sponsors and contributors, and enough tenure to round out the sharpest edge cases, are generally "better". So Terraform and Ansible come to mind. Both deeply flawed, but they become the least-worst current option. People should still strive for better, test new ideas with new tools, but choices for production code should play it safe almost every time.
- void_mint 5y ago
  I feel like the purpose of your posts is more about bemoaning infrastructure tools and less about discussing the original post's product. I'm not really sure it's productive. I definitely do not find it productive.