8 ms·
Infrastructure as Code Should Feel
- sidcool 5y agoOkayish article. It's more a marketing one than technical.
- drewcoo 5y agoI am an IaC advocate. I see articles like this as a barrier to meaningful IaC adoption. > In the tech industry, we can be guilty of the same crime sometimes. Certain practices get ingrained into the profession to the point where we forget exactly why we did it in the first place. But in school he hadn't known why he did it in the first plave. And more often in tech I see cargo cult practices that people don't understand but follow some steps that may or may not "work." Often these are misunderstood "best practices." He goes on to say: > Infrastructure as Code (IaC) is a practice I really feel should be implemented everywhere it is relevant. It is good common practice, the popularity of which continues to grow. Which is exactly how cargo cult practices spread. What do they do? How do the do it? I dunno but everyone seems to be doing something and I think it must be like FOO. It's also incredibly dangerous to say we should follow a practice because of how it "feels." Does it "feel stable" or "feel understandable" or "feel fast?" Who cares. Show me the value, don't try to give me the tingles!
- cube2222 5y agoTo be fair, the author does provide arguments for why Infrastructure as Code should be used, in the next few paragraphs.
- iamtew 5y agoThe real world don't care about feelings.Feelings are extremely subjective making these kind of statements quite vague. As a person with alexithymia, this is so confusing, I have no idea what "feels good" is supposed to mean. I know it's not bad, but that's a big spectrum of meaning. Good doesn't mean great or brilliant either. Just confusing. Emotional blindness: https://en.wikipedia.org/wiki/Alexithymia https://en.wikipedia.org/wiki/Alexithymia
- falcolas 5y agoFWIW, "feel" and "good" as used in this article is not about emotions; it's more of a shortcut that means "Do I trust that this is doing what I expect it to regarding (safety | stability | understandability)?" "Feel": "I trust." "Good": "Does what I expect it to."
- NateEag 5y ago"Trust" is (or at least can be) an emotion.
- politician 5y agoTrust is the faith that the subject won’t betray you or your interests despite being in a position to do so. It’s a belief, not an emotion.
- NateEag 5y agoHmm, yeah, good point. I think you're right. I guess the emotions associated with trust might be "peace" or "confidence". Thanks for correcting my error!
- NateEag 5y agoPhysical reality is barely changed by peoples' emotions (small shifts in their biochemistry). Physical reality is very often changed significantly by the choices people make, which are rooted in their emotions. So, while the universe as such may not care about emotions, they still matter as long as you're interacting with humans (including oneself). FWIW, I'd not heard of alexithymia before, and I now am thinking I may have had it most of my life. I've gotten much better at recognizing emotions in myself and others over the past two years through a combination of prayer, therapy, introspection (a skeptic might argue my prayer was just an obscured form of introspection), and reading (Non-violent Communication by Marshall Rosenberg helped me immensely). I've seen the world become much more comprehensible as this has changed - other humans are a big part of the world, given how dependent we all are on civilization for survival and thriving, and as hinted above, I now understand my own actions and choices much better too. Not being able to recognize my emotions didn't mean I had none.
- rocqua 5y ago> It's also incredibly dangerous to say we should follow a practice because of how it "feels." Does it "feel stable" or "feel understandable" or "feel fast?" Who cares. Show me the value, don't try to give me the tingles! I disagree. First of all, how something feels says something about people's intuition which is a decent indicator of reality. More importantly though, how something feels affects how (or even if) people will use it. A system which 'feels' wrong will be under-used and generally worked around. This makes it a bad system in the real world, regardless of 'how great it would be if everyone used it'. It's similar with vaccines. The fact that to many people a vaccine 'felt wrong' made the vaccine actually less effective at blocking the spread of corona. The value of something depends very much on 'how it feels'.
- sausagefeet 5y agoThe "safe" section is directed primarily at the development environment but once you start collaborating with something like Terraform, there are a whole bunch of ways you can forget to plan or apply a change (for example applying in a PR but forgetting to merge or closing it because you don't want to go forward with the change but forgot you applied it or a part of it). Shameless plug but my co-founder and I started Terrateam to try to make it really easy to get going with Terraform on GitHub and stay safe doing it. https://www.terrateam.io/blog/posts/safety-first/ https://www.terrateam.io/blog/posts/safety-first/
- rpadovani 5y agoYou shouldn't be able to apply in an MR tho, unless is some playground account. Developers shouldn't have enough permissions to modify production infra from a MR.
- sausagefeet 5y agoOrganizations have different rules around when to apply a change. Some prefer to do it pre-merge, that way if something goes wrong they can modify it and apply again, and others prefer to do it post-merge. We are not opinionated own when a customer does it, we just have rules to make sure a change is both merged and applied. Additionally, who can perform an apply is a different, but related, question to when. The bet behind Terrateam is that a lot of developers and SREs don't want to leave their MR page when applying and planning so we are supporting as much functionality as possible there.
- pid-1 5y agoI've been doing IaC in AWS since terraform 0.11 (now added Azure and AWS CDK to the mix). Most cloud providers (and don't get me started with software platforms) support IaC as a complete afterthought. The day you decide to use IaC, you will have chosen to spend your days fighting against their APIs to make things nice and immutable. Moreover, most languages and tooling are rather immature. Things like secrets, testing, CI/CD, multi repo infra, collaboration, describing stuff that changes itself (e.g. databases that update automatically) are not well understood, solved problems. Consequently I'm quite diligent when writing my infra as code nowadays. I reserve it for things that really play nice with immutability and are not likely to change all the time. Also, as the author says, not coupling infra code with apllication code is gold advice.
- chrisshroba 5y agoAny pointers to what you use instead for managing those kinds of corner cases like things that modify themselves?
- vageli 5y agoOne solution is adding the attributes you expect to be modified to ignored_changes in a lifecycle block[0], which instructs terraform to disregard changes to those attributes when calculating diffs for plan/apply. That allows you to specify a value for that attribute at creation time and allow for its subsequent manual/non-terraform-managed modification without causing "drift". [0]: https://www.terraform.io/language/meta-arguments/lifecycle https://www.terraform.io/language/meta-arguments/lifecycle
- nickjj 5y agoI like Terraform but I've grown to not fully trust its plan output. There are lots of cases where you can plan something successfully without errors but then when you go and apply it you'll run into errors and now your infrastructure is in a half working state where some resources applied successfully and others failed.
- jen20 5y agoYou’re not wrong on this. Being able to give 100% guaranteed valid plans would require cloud providers to publish full specifications of their APIs, including (most importantly) the domain of each property relative to possible values of all the other properties of a resource. Given that most cloud providers struggle to publish an API where even the data types for properties are correct, this does not seem realistic today. Another way to achieve a good result would be an accurate “dry run” API from the provider, but these are also often inconsistent when they exist, and make planning painfully slow.
- nickjj 5y agoYeah, it's not an easy problem to solve. But it is worth pointing out because you can't trust plans for figuring out if something will work or not. However they are great for letting you know what's about to get CRUD'd, which in itself is very valuable to help prevent a set of issues but it doesn't guarantee victory in the end.
- pech0rin 5y agoWhy is the “how” cut off at the beginning of the article title. Makes it really confusing without it.
- mitchbob 5y agoRemoving "How" is one of those automatic edits to titles made by HN, and I think it's a bad idea, as it often changes the meaning. Fortunately, it's possible to for the submitter to click "edit" and change it back to the original.
- tobr 5y agoHN does this automatically when you submit a title starting with “How”. I guess it reduces the clickbaityness of certain types of titles, but fails in cases like these. If it happens and you don’t like it you can edit the submission to add the word back.
- SkipperCat 5y agoSome folks seem to be taken back by the word "feel" in the title, but it rings true to me. I work at a place with lots of super smart people. Almost everything is automated, but a lot of that automation breaks and you have to constantly get the creator involved to turn that "one little knob" to get it working for your job. That's fine for the creator because he/she knows exactly what to do when the problem occurs. However, if you're an internal customer trying to use that system, its infuriating. If you're going to create IaC tooling for a team of 50 people, it needs to work without contacting the author. If not, you've just taken a 5 minute job and turned it into a 2 hour search for the bug event.
- DoneWithAllThat 5y agoI had this exact issue at a last company. We had a system that tore down and rebuilt our entire clusters to updates QPS estimates. It was the right idea and the right approach, but it was a big gnarly ball of code that broke frequently, requiring the steps it got stuck on to be done by hand and the code fixed for next time. So anyone working on it still had to know and be co dude that’s executing any arbitrary manual steps as well as understanding the code enough to fix it (or get the expert involved). I’m doubtful it was a net positive in that state.
- SirensOfTitan 5y agoI don’t really think IaC is there yet, as someone who has used terraform (and now pulumi) in production for some time. My biggest gripe is that the feedback loop is fairly slow: planning often predicts valid config that fails in deployment. It doesn’t help that the process of planning is really slow (see https://github.com/pulumi/pulumi/issues/8872 https://github.com/pulumi/pulumi/issues/8872 it can take minutes for pulumi to plan changes if using azblob backend) with little incentive for teams like pulumi to investigate or fix unless you use their cloud. I get that pulumi is in the business of promoting their own cloud, but there are few (if any im aware of) IaC solutions not intrinsically tied to for-profit SaaS because it’s incredibly labor intensive to build consistent interfaces to several constantly changing cloud providers.
- hobo_mark 5y agoAs someone starting with terraform literally this weekend, why did you move away?
- SirensOfTitan 5y agoAsking my engineers to learn a completely new DSL (HCL) with its own quirks to change infrastructure felt like it ran against the spirit of IaC. If they were tweaking infrastructure every day, that’d be one thing, but it’s fairly hard to commit a language to your long term memory when you’re interacting with it only on occasion.
- soco 5y agoYou can do Terraform in Java. Or Python. Or... https://www.terraform.io/cdktf https://www.terraform.io/cdktf
- p_l 5y agoCDK is reasonably new thing for terraform. A lot of us have cut our teeth on previous major version of terraform DSL, when it was even less capable than it is now (no maps, no for each, etc.) and it wasn't that uncommon to be told to generate terraform JSON directly to work around DSL shortcomings, generating essentially all of the provider objects yourself.
- sebastianconcpt 5y agoOn the other hand I can't think in automating something like this: https://news.ycombinator.com/item?id=30371604 https://news.ycombinator.com/item?id=30371604
- mountainriver 5y agoIaC is unfortunately the wrong abstraction for most use cases. People have become religious about it and it’s slowing down development. I’ve been doing IaC since terraform and k8s came around. We initially all thought it was great, now I think we’re covering up these wonderful UIs for a much worse interface that takes significantly longer, and in many cases gives us very little. What we will hopefully converge on is UIs that basically do the same things as IaC.
- jugg1es 5y agoI was about to disagree with you until I read the part where you are using terraform with k8s. IaC starts breaking down once you start using k8s - especially if you are using a flavor like EKS that is both kubernetes but is also its own IaC platform.
- nijave 5y agok8s has its own built in IaC with object definitions (usually yaml or some sort of yaml templates) and controllers to apply the changes. I don't think most people are managing k8s objects directly with edit/patch--they're just applying complete yaml/object definitions to override current state. If you want drift/auto apply you can use a CD solution like Argo
- jugg1es 5y agoYes, that is what I was referring to.
- throwaway787544 5y agoAgreed. IaC has become another gated bottleneck; rather than waiting on an admin to change a server, we wait on an admin to write and test the IaC to change a server. One reason it's this way is poor design of cloud tech. Cloud tech isn't immutable, idempotent, and versioned, most of the time. Often there's arbitrary limitations and conventions that are not obvious which cause changes to break unexpectedly ("we don't know the result until apply" seems to happen every single apply, and success in dev is no guarantee of success in prod). And there are very few turn-key solutions, so we're left to keep rebuilding the same things and running into the same problems. We need people creating SaaS/PaaS/IaaS to create systems that give us what we really want, without having to hire somebody to write really crap fake code to work around what those systems lack. We need GUIs that allow anyone to safely make changes without being an expert. We need turn-key solutions. We need advanced best practice deployment methods to be the first option, not implemented 5 years down the road. We need security to be easy. We need multitenancy to be the default. We need simple billing limits. ..... Ok a lot of that has nothing to do with IaC. But it's clear to me that a lot of what we do do today is an unfortunate side effect of how these systems were designed, and what they still lack. Somebody built a railroad, and we're still trying to force the railroads work like real roads.
- cosmiccatnap 5y agoI feel like infrastructure as code is one of those things that highlights your companies poor practices more than it does highlight the failures in say terraform or cloudformation. A company that is willing to work at the code level to automate their systems will find tools like terraform and cloud formation useful, companies that have a LAMP stack from 2010 and no way to test their infrastructure changes in a safe way will not...
- timcavel 5y ago
- ozim 5y agoI think more about culture that you should have "automate first" mindset. Yes, changing 500 lines in a text file if you can write grep replace in 30 seconds is nice. Automating deployment that you are doing once a month and is taking 20 mins to prepare and run, I don't feel it. Because developing such automation takes considerable resources especially if you cannot test it right away on prod and have to make test env and other overhead. Maybe one should automate preparation parts of such deployment first and then see how it goes.
- terom 5y agoAutomation is not only about saving time, it's also about guaranteeing consistency and quality.