8 ms·
Under no circumstance would I allow my projects to be configured by a full-blown programming language. There is no way to ensure that there are no regressions
by jevgeni 7y ago
Under no circumstance would I allow my projects to be configured by a full-blown programming language. There is no way to ensure that there are no regressions under different configurations. The amount of complexity introduced just for configuration is insane.
If you feel the need to put code in your runtime config, that means your design sucks.
- q3k 7y agoBut configuration is a complex task! At least from my experience. Limiting yourself to solving it with mediocre tools (for instance language that disallow writing tests or even comments!) is a footgun, where you disregard good engineering tools for stringly types, duplicated configuration.
- jevgeni 7y agoWe might have different target audiences of our work, but in my line configs should not require tests. Because their task is to adjust the functioning of an application. If they all of a sudden require tests, that means (at least to me) that it's not part of the configuration anymore, but a part of the application proper and should be treated as such.
- q3k 7y agoFor me the target is mostly verbose YAML configuration of existing software (which I can now emit from an in-memory object instead of templating and having to remember all the YAML quirks) and k8s manifests. Especially when I have to stamp the same piece of software multiple times in multiple deployments (environments/regions/etc). For instance: I just brought up a new Factorio game server yesterday, just by adding a single line to my jsonnet configuration machinery: https://gerrit.hackerspace.pl/c/hscloud/+/246 https://gerrit.hackerspace.pl/c/hscloud/+/246 . This in turn brought up a bunch of new k8s resources, including some persistent volumes, a deployment, and a loadbalancer service, after running a single `kubecfg update` command.
- jevgeni 7y agoI agree, that YAML is usually a mistake. In your scenario, it's more like provisioning of infrastructure and in that case - configuration is the code, so I can see where you are coming from. I mostly write financial software so the use case is usually closer to classical single machine applications. And in that case, I prefer simple TOML.
- darkwater 7y agoAnd what about "configs" like Kubernetes deployments?
- shawnz 7y agoYou could use the Maven approach: for whatever the declarative configuration isn't powerful enough to specify, you write a plugin. That plugin can then be developed and managed just like any other software project, rather than having to bring those tools into the configuration realm.
- rfoo 7y agoWait. In this case aren't the plugin itself "code in runtime config"? edit: Now I understand that being a plugin it must have proper abstraction, but ad-hoc logic do exist.
- shawnz 7y agoI guess it depends on how you look at it, but if the plugin is a self-contained module that can be managed independently from the app it's configuring, I wouldn't really say it's code that's "in" the configuration. > ... but ad-hoc logic do exist. True. I don't disagree, no tool is right for every job.
- jevgeni 7y agoPersonally, I group these use cases under "infrastructure provisioning". The use case here is completely different - for example, you won't have a non-developer modifying the configs. So in this case, automation is a good idea.
- shawnz 7y agoConfiguration is complex if you use it for complex things. That's just one way of designing software, it's not necessary to make software so extremely configurable that you need a DSL to configure it. Even if you really feel you need that level of configurability, then the software could be designed as a library instead and users could consume it with whatever language they prefer (where bindings are available). That would basically be the same amount of complexity as a configuration DSL, except you could use all your standard project management tooling to manage it.
- u801e 7y ago> But configuration is a complex task! Should it be? If an application doesn't have configuration, then the logical thing to do would be hard code all the values it needs in the code itself. This could be done with a number of variables/constants defined in the entrypoint method for each value needed, or defining a data structure that contains all the values needed in the same method. Reading a config file and serializing it into those variables or data structure shouldn't be a hard problem. Formats like YAML or json allow for deserailizing the string into a data structure (though you have to add your own validation). ini format files can be done the same way in a loop, but still require adding validation. The validation could be based on data type or allowed values. While that's rather tedious, it's not complex.
- jbreckmckye 7y ago> There is no way to ensure that there are no regressions under different configurations. No, but static config files won't give you that proof either. Unless I'm misunderstanding your point? At least if I write build scripts in e.g. ts-node I can use the type system to provide some modicum of safety, which seems like a plus.
- jevgeni 7y agoProof, no. But ensuring it in terms of manual labour is far easier with a static file, than debugging a convoluted script pretending to be a config file.
- deleted 7y ago[deleted]
- ironmagma 7y agoThat’s why my personal preference is to keep the config file as code but to keep it as close to a static config as possible. The Turing completeness of the environment is there merely as an escape hatch, for the one branch you need to make. If your tooling requires static configs, but you really need dynamism, you’re stuck with dynamically generated static configs, which are even worse than a dynamic config.
- KMag 7y agoAs I've mentioned elsewhere, I agree, but I think in an ideal world, there would be an explicit syntactic construct warning everyone where the program was using the escape hatch to leave the Turing-incomplete subset.
- klibertp 7y agoYou could create and use an embedded DSL tailored specifically for your application. This is what Jenkins does with their "declarative pipeline": it's basically Groovy-based DSL, and if you need anything not covered by it you need to use `script { ... }` element. The problem here is of course how not to make your DSL suck, which Jenkins' did not avoid. But if you're lucky, your language of choice already has a few libraries implementing embedded DSLs for configuration, and it's quite possible that at least one of them is actually usable.
- greyskull 7y agoGenerate static configuration, effect only what's in the static configuration and persist it to version control. You get the ergonomics of a programming language while getting a diff if anything changes and get it code reviewed. Could even write unit tests if you start to get unintended changes.
- jevgeni 7y agoThat seems like an acceptable solution to this. I'd still prefer not to be in a situation, where configuration is so complex and dynamic, that I need config generation, but if one does land in such a situation, I think this is the most sane approach.
- mrkeen 7y ago> Under no circumstance would I allow my projects to be configured by a full-blown programming language. Well, the choice is Java, or Java and yaml. > The amount of complexity introduced just for configuration is insane. Again, it's Java, vs. Java + Spring + env + application.yaml + application-$profile.yaml (in my case usually). Once config constants enter your language, they're type-checked. Tree-structures suddenly need to be well-formed. No more trying to remember if you force https-only with 'enforceHttps', 'enforceHTTPS', 'enforceTLS', 'enforceSSL'. (= True,= true, :true, :1, :on, = "on") > no regressions under different configurations. It's great for that! Instantiate three copies of your service from Main - say, one backed by a hashmap, one connecting to a live db, one connecting to your docker-compose db, etc. Run the the same set of commands (in the same language your tests are already written in) and assert the outputs.
- jevgeni 7y agoThe choice is: no Java at all. :p
- ric2b 7y agoSo to change a configuration option I need to recompile the project?
- mrkeen 7y agoMy flow is: 1) I think about changing it 2) I change it 3) I figure out some way to verify it 4) I deploy it to prod I don't want to do step 2 unless I can do step 3. It just so happens that type-checking happens during recompilation, and testing happens after.
- tomp 7y agoI find general-purposeness a great idea if your specific config depends on something complex, e.g. the username/host/path where the code is running. That's quite hard to in a confgiuration-specific format, without anticipating this specific scenario in advance.
- BiteCode_dev 7y agoThat depends of the scale. See ansible and the YAML config files. In this case, a real language would have been a better choice. Also, sometimes you need to generate config files. In that case, templating is not always a good choice.
- kelnos 7y agoThen again, we have Chef (ruby DSL), and from experience I've found that they tend to turn into a mess that very easily can become hard to reason about. Although, I'm not sure if this is because of the design of Chef itself, or just simply because it uses a real language.
- wenc 7y agoWhat about .bashrc, .vimrc or Makefiles?
- IshKebab 7y agoMakefiles aren't executable - they are fine. .bashrc is badly designed. Bash should have a proper non-executable config system. But... it's Bash. If you're expecting sane robust design from Bash you haven't been paying attention!
- comex 7y agoMakefiles are executable, with their own crappy ad-hoc Turing-complete language. For example, here’s the full list of functions supported by GNU make: https://www.gnu.org/software/make/manual/html_node/Functions.html https://www.gnu.org/software/make/manual/html_node/Functions... Naturally, trying to do anything even slightly complex with it quickly leads to an unreadable cacophony of dollar signs, backslashes, and soft and hard tabs (you need both!). Just like the modern YAML-based systems, Make’s language wasn’t originally intended to be a full scripting language, but had scripting bolted on because people needed it. If only it had chosen to embed a real programming language instead, you could have the functionality without sacrificing readability.
- joejev 7y agoIt's not an ad-hoc turing complete language, it is just a programming language. Makefiles are just programs. There are some strange implicits, but that isn't unique to the Make programming language.
- comex 7y agoIt's Turing complete because all standard programming languages are Turing complete (modulo some technicalities which are not relevant here). It's ad-hoc because it was built gradually on top of the original Unix make, which supported rules and variables but not any of the more advanced features. This history explains many of its quirks and relatively extreme limitations as a programming language, such as: - Expressions must be all on one line, except within `define` blocks or if you use backslashes to join lines together (which tends to result in a lot of backslashes). - Hard tabs are required to introduce commands in rules but are (mostly) banned outside of them, so if you want any kind of indentation for an if block or user-defined function, you need to use spaces instead. - User-defined functions. Calling them is merely more verbose than calling builtin functions (you need to invoke the `call` builtin function). Defining them is worse: function parameters are numbered rather than named, similar to a shell, but in a shell you can reassign them to named variables, whereas Make's expansion rules make that awkward to achieve. - And of course, no types other than strings (a limitation partly shared by shells, but most shells do have arrays, even if they're awkward to use). Admittedly, most of that gradual evolution happened a long time ago, and GNU Make has been relatively stable as a language since then. It's not "ad-hoc" in the sense that it's constantly changing or ill-defined. But that stability also means that it never outgrew the limitations of its original design. Some research I did for fun: The oldest version of GNU Make I can find [1], from 1988, already had a handful of functions, including `foreach`, `filter`, `patsubst`, etc., as well as support for multi-line definitions (`define`/`endef`). `call`, on the other hand, didn't appear until 1999. Amusingly, its initial implementation came with a comment [2] bemoaning the lack of "nested lists and quotes", and even semi-seriously proposing that GNU Guile (Lisp implementation) be integrated into Make, in order to let Makefile authors use a 'real' programming language. No fewer than 13 years later, that proposal actually became a reality; unfortunately, it was an optional feature which distributions tended to leave disabled, ergo Makefile authors could not rely on it being present, so the feature has seen approximately zero use. [1] within https://gcc.gnu.org/pub/binutils/old-releases/binutils-1988-07-11.tar.bz2 https://gcc.gnu.org/pub/binutils/old-releases/binutils-1988-... [2] https://github.com/mirror/make/blob/c4353af3f9b143b213644c09da5cf46ef611b250/function.c#L1725 https://github.com/mirror/make/blob/c4353af3f9b143b213644c09...
- Diederich 7y agoAt least one large and successful tech company uses Python for configuration: https://muratbuffalo.blogspot.com/2016/02/holistic-configuration-management-at.html https://muratbuffalo.blogspot.com/2016/02/holistic-configura...
- jevgeni 7y agoGlad it works for them. For me and my team it doesn't.
- nitely 7y agoAs well as every company using Django (the most popular Python web framework)...
- jevgeni 7y agoJust because Django is popular, doesn't mean that companies using it know what they are doing, or have weighed all pros and cons. I've seen some outright tragic Django setups with major companies.
- nitely 7y ago> I've seen some outright tragic Django setups with major companies. Well, the same thing can be said about pretty much anything. I've seen good and bad Django setups, so what?. The point is configuration as code works well if you know what you are doing. A config minimal language (toml, yaml, etc) may also work well as long as you don't need something that's not supported by it, and as long as you don't need to debug it.
- wolco 7y agoFrameworks that encourage convention over configuration provide a better tradeoff. If you are willing to accept the defaults you aren't required to write any configuration.
- tyre 7y ago
- ebg13 7y ago> If you feel the need to put code in your runtime config, that means your design sucks. Hello from every plugin system ever. True fact: not all configuration is basic.
- jevgeni 7y agoI wouldn’t call it configuration. And usually these things are implemented with way more forethought.
- ebg13 7y ago> I wouldn’t call it configuration. I actually agree with you. Elsewhere in the comments I say that the OP article failed to name this properly: https://news.ycombinator.com/item?id=22788696 https://news.ycombinator.com/item?id=22788696 But it is what the article is really discussing.
- rainforest 7y ago> If you feel the need to put code in your runtime config, that means your design sucks. I would look at Terraform and beg to differ - they started with a simple stringly typed language and had to build a typed language. Still no support for conditionals, so everyone has to abuse the only bit of control flow on offer - ternary variables and counts. CircleCI is just as bad - it's a nice yaml format until you want a conditional step in workflow that's conditional on something other than the branch name. It's very risky to settle on a static config at design time since the users will find a need. How would you cater to that user without offering a programming interface to configure it?