3 ms·

Is configuration really so complex nowadays that we need entire languages (DSLs, really) for managing them? Isn't there a better way? I feel like its so 'popul

Is configuration really so complex nowadays that we need entire languages (DSLs, really) for managing them? Isn't there a better way? I feel like its so 'popular' to provide config hooks into libraries and systems, rather than exposing a Programmatic API that's at least unit testable to understand the results you get back, which makes it infinitely more reliable, IMO. APIs are harder, I realize, because it forces you to really think about what you're going to expose and how before hand, rather than perhaps after the fact. I also realize the tradeoff is that configs are suppose to be more limited by design (you can only express so much logic even when using some of these complex templating systems, they all reach some finite point of complexity). They're also suppose to be more portable (configs don't require runtimes per say, though that looks like its about to change!) Arguably however, its gotten so complex to manage configurations that I would rather just have exposed APIs for systems so I can have better, robust testing in a language. I realize this means some systems will no longer be language independent, but when I can use an API instead, I almost always do. Configuration management has just gotten out of control. To think, people say webpack is 'config hell'. Ever try dealing with a complex templated Kubernetes config? Saltstack YAML Jinja2 madness? I'll take webpack any day. At least its typed! /rant With that said, this looks neat. I can also stand by the idea that if there is a sane way to create a DSL like this that exports to any reasonable format (json, ini, properties file, YAML) then that actually, I will completely concede, may be worth it.

rtpg 6y ago
You can either have a configuration bit which is like "add_two_to_every_result: true", or you can have "result_post_processing = function(x) {return x + 2;}". When working on complex systems, I know which thing is going to be easier to use for my "bespoke" systems, and which one is going to devolve into spaghetti messes. Unfortunately I think loads of system have gone for the former. I think that stuff like grunt ended up being a mess because it tried to straddle the two, and webpack has made stuff cleaner in that respect, by accepting computation. Almost every build system in the world, however, really doesn't like this model. The first CI system whose configuration file is, like, Lua, will become a billion dollar business overnight. Special frustration goes to Bazel, which gives itself a cliff-shaped learning curve out of its "we really don't want to run arbitrary code" feeling, while actually having all the moving parts. You just need to dig through a bunch of "really really please don't do this" stuff to find the real docs.
- randallsquared 6y ago
  > You can either have a configuration bit which is like "add_two_to_every_result: true", or you can have "result_post_processing = function(x) {return x + 2;}". In this case there's an intermediate "increment_result_by: 2" that would be better than either. Once in a while there are use cases that are so undefined that "do whatever you want" makes sense, but usually this is an excuse to not think too much about what the user wants to do, and why. The benefit of both of the limited solutions over the other is that it's possible to fully test the entire space of configurations. it definitely is possible to let this style of config devolve into spaghetti, but at least there's some hope of untangling it in the future, which is much less possible if you accepted essentially any result transformation some enterprising user had spent a week programming. If the right solution for configuring your system is a full programming language, consider distributing your system as a library or framework, rather than a system that needs "configuration". If your library is fully tested and someone screws up in their own code, that's a different kind of problem than if someone provides a syntactically correct config that causes your system to fail. Blurring the lines between these kinds of problems is not going to make supporting your (non-programmer) users easier, in most cases.
- mdaniel 6y ago
  > The first CI system whose configuration file is, like, Lua, will become a billion dollar business overnight. Jenkinsfile is written in Groovy and is not a billion dollar business that I'm aware of. Were you trying to emphasize the lua part? Space from JetBrains has its build config written in Kotlin, for a more strongly typed version: https://www.jetbrains.com/help/space/automation-getting-started.html https://www.jetbrains.com/help/space/automation-getting-star... I believe there are others but those are the two I have handy
  - no_wizard 6y ago
    Jenkins has persistent problems in my experience with reliability. We only ever really got shell scripts to work well with Jenkins at a previous place of employment years ago. The API layer was hard to leverage properly (part of what I was getting at in my rant. Which is why I get why configuration has taken over a lot of these kinds of complex systems) I think a lot of this got fixed with Jenkins X because its underlying tech is just docker images. There again though you're mostly managing complex configuration rather than dealing with an exposed API layer

rtpg 6y ago

You can either have a configuration bit which is like "add_two_to_every_result: true", or you can have "result_post_processing = function(x) {return x + 2;}". When working on complex systems, I know which thing is going to be easier to use for my "bespoke" systems, and which one is going to devolve into spaghetti messes. Unfortunately I think loads of system have gone for the former. I think that stuff like grunt ended up being a mess because it tried to straddle the two, and webpack has made stuff cleaner in that respect, by accepting computation. Almost every build system in the world, however, really doesn't like this model. The first CI system whose configuration file is, like, Lua, will become a billion dollar business overnight. Special frustration goes to Bazel, which gives itself a cliff-shaped learning curve out of its "we really don't want to run arbitrary code" feeling, while actually having all the moving parts. You just need to dig through a bunch of "really really please don't do this" stuff to find the real docs.

randallsquared 6y ago
> You can either have a configuration bit which is like "add_two_to_every_result: true", or you can have "result_post_processing = function(x) {return x + 2;}". In this case there's an intermediate "increment_result_by: 2" that would be better than either. Once in a while there are use cases that are so undefined that "do whatever you want" makes sense, but usually this is an excuse to not think too much about what the user wants to do, and why. The benefit of both of the limited solutions over the other is that it's possible to fully test the entire space of configurations. it definitely is possible to let this style of config devolve into spaghetti, but at least there's some hope of untangling it in the future, which is much less possible if you accepted essentially any result transformation some enterprising user had spent a week programming. If the right solution for configuring your system is a full programming language, consider distributing your system as a library or framework, rather than a system that needs "configuration". If your library is fully tested and someone screws up in their own code, that's a different kind of problem than if someone provides a syntactically correct config that causes your system to fail. Blurring the lines between these kinds of problems is not going to make supporting your (non-programmer) users easier, in most cases.
mdaniel 6y ago
> The first CI system whose configuration file is, like, Lua, will become a billion dollar business overnight. Jenkinsfile is written in Groovy and is not a billion dollar business that I'm aware of. Were you trying to emphasize the lua part? Space from JetBrains has its build config written in Kotlin, for a more strongly typed version: https://www.jetbrains.com/help/space/automation-getting-started.html https://www.jetbrains.com/help/space/automation-getting-star... I believe there are others but those are the two I have handy
- no_wizard 6y ago
  Jenkins has persistent problems in my experience with reliability. We only ever really got shell scripts to work well with Jenkins at a previous place of employment years ago. The API layer was hard to leverage properly (part of what I was getting at in my rant. Which is why I get why configuration has taken over a lot of these kinds of complex systems) I think a lot of this got fixed with Jenkins X because its underlying tech is just docker images. There again though you're mostly managing complex configuration rather than dealing with an exposed API layer