5 ms·

Sure -- if your users can't write code, then you obviously can't tell them to use code to generate a config. But take a step back and look at a lot of the exam

Sure -- if your users can't write code, then you obviously can't tell them to use code to generate a config. But take a step back and look at a lot of the examples that people are bringing up here -- Docker, Babel, Webpack, Github Actions, code linters, testing libraries, Ansible. These are all environments where it's pretty reasonable to expect users to know how to write code. If you're mostly getting hung up about me calling this a config, let me see if I can rephrase the original article's point in a way that is more palatable to you: A lot of people are building programs explicitly for programmers that use config files, when those programs would be much better suited by instead dropping the configs entirely and using something more like scripted options. These program authors struggle building a config schema that is powerful enough for programmers yet simple enough that programmers won't get irritated learning yet another multi-page API. What the authors have missed is that they shouldn't be trying to control their program with a config at all. Their entire product is geared at programmers, so they should just let programmers generate the options that they pass in. There's very little reason for Babel to be setting options with a JSON config file instead of a JS script, because nobody uses Babel other than Javascript programmers. For these specific programs that are struggling to express themselves with YAML/JSON, a non-config scripted setup would be less error-prone and easier for their users to learn.

pdonis 7y ago
> These are all environments where it's pretty reasonable to expect users to know how to write code. For those kinds of environments, yes, I agree that it makes a lot more sense to just have users write code in the same language that the main program is in, than to try to hand-roll a config file format and structure. I didn't get the sense from the article that its scope was limited to these kinds of environments, but I may have missed something.
ZenPsycho 7y ago
move over templated yaml files, get ready for templated python files. I heard you like config files, so I set up a config file for your config file so you can config while you config.
- danShumway 7y ago
  Real talk though, at the point where people are building templating engines for YAML files, doesn't it become kind of obvious that YAML isn't powerful enough or expressive enough for at least some of the projects that are using it? We've got a project in Python, but Python is too hard to read, so instead we have YAML. But YAML is too hard to write, so now we have YTT. We've now taken a program that was written in one language, and magically transformed it into 3 languages instead :)
  - pdonis 7y ago
    > We've got a project in Python, but Python is too hard to read You said the kind of scripting system you were describing was for cases where your users can write code. If your users can write code, I don't see why Python would be too hard to read. If your users can't write code, then, as you agreed, you shouldn't expect them to, which means the kind of system you were describing won't work. And that means you should rewrite your code so it doesn't need such a complicated configuration that it seems like writing code is necessary to express it.
    - danShumway 7y ago
      Yeah, I was trying to joke -- if your users are OK with Python, then just let them use Python. What I was getting at was that this is a scenario where concerns over Python users needing to actually use Python has morphed into those same users now needing to use 3 languages.
  - ZenPsycho 7y ago
    If you're templating YAML, it just means that you don't know that you can generate YAML. Dev ops is a very different world. Yes they can script, but that doesn't mean they will modify any part of your program, even a python config file directly. They will template it. If you find yourself thinking that your config language isn't powerful enough, the solution is to either generate it, or to fix the program to not require a complex config file. There is a long term, 10 year cost to making config turing complete that I don't believe is worth paying. It'll save you time up front, maybe, but I certainly am not going to be impressed with having to modify your program just to "configure" it. Finally, given that's the environment we're dealing with, from now on the rational thing to do would be to design config file formats expecting them to eventually get templated. Because that's how they get automated.
ZenPsycho 7y ago
Keep in mind that config files are not just about writing them. They are also about reading, and in my personal experience, when I encounter a heavily scripted config file in some javascript project, my inclination is to flatten it out to a plain JSON file just so I can see what it's actually doing. This is the beginning of the logic that lead to things like Automake, and completely impenetrable makefiles. The problem with programmable scripted config isn't the writing of it. it's the reading and maintenance. Programmed config is a layer of obfuscation, that I, as a maintainer, need to spend time and mental effort decoding to get work done.
- danShumway 7y ago
  I'm sympathetic to this, and I suspect the original author is as well (see their comments on Emacs). I think you're making a very good point that people need to keep in mind when they write configs. However, very specifically where Makefiles are concerned: > This is the beginning of the logic that lead to things like Automake, and completely impenetrable makefiles. The problem with Make in specific isn't scripting the build, it's that Make is a completely separate domain-specific language that isn't particularly well structured or particularly elegant in the first place. The comparison I would draw is with Grunt. Grunt is also a programmable config that allows you to do powerful stuff. It's also IMO a giant disaster and most projects shouldn't use it. Grunt is using programming to control how a project builds, but it's also tying tons of layers of abstraction into this secondary API that's unreasonably difficult to debug, and that lends itself to really bad, unnecessarily verbose control flows. I was teaching interns how our build process worked at a company, and I had an "aha" moment when I realized that in the time I could teach them Grunt's API, I could also teach them how to write a maybe 100 line pure-JS script that did literally everything Grunt was doing for us. When I look at Makefiles and Gruntfiles and cringe, very often the thing that makes me cringe are these giant pillars of abstractions and control flows of "program X calls program Y and pipes its output to program Z, and good luck intercepting that specific output to see what's going wrong". There's something to be said for getting rid of these giant build systems that are spanning multiple processes and languages in general, and instead doing something very direct in one language in one file. And I kind of think that's a concern that cuts across different config types. At my current job, we're in the process of dropping Team City, because we realized that for our needs, we can get by with a single 30-ish line bash script that gets copied to each build machine and auto-run every 5 minutes. And Team City is using a lot of declarative options in a very structured web UI, but the Bash script is still a lot easier to debug and reason about than Team City's whole... thing. Which is not to say you're wrong, I do think you're making a completely valid point. But I don't think the obfuscation you're talking about is inevitable, I think that's just a side effect of programmers thinking they're clever and trying to make generalized build systems for specific tasks.

pdonis 7y ago

> These are all environments where it's pretty reasonable to expect users to know how to write code. For those kinds of environments, yes, I agree that it makes a lot more sense to just have users write code in the same language that the main program is in, than to try to hand-roll a config file format and structure. I didn't get the sense from the article that its scope was limited to these kinds of environments, but I may have missed something.

ZenPsycho 7y ago

move over templated yaml files, get ready for templated python files. I heard you like config files, so I set up a config file for your config file so you can config while you config.

danShumway 7y ago
Real talk though, at the point where people are building templating engines for YAML files, doesn't it become kind of obvious that YAML isn't powerful enough or expressive enough for at least some of the projects that are using it? We've got a project in Python, but Python is too hard to read, so instead we have YAML. But YAML is too hard to write, so now we have YTT. We've now taken a program that was written in one language, and magically transformed it into 3 languages instead :)
- pdonis 7y ago
  > We've got a project in Python, but Python is too hard to read You said the kind of scripting system you were describing was for cases where your users can write code. If your users can write code, I don't see why Python would be too hard to read. If your users can't write code, then, as you agreed, you shouldn't expect them to, which means the kind of system you were describing won't work. And that means you should rewrite your code so it doesn't need such a complicated configuration that it seems like writing code is necessary to express it.
  - danShumway 7y ago
    Yeah, I was trying to joke -- if your users are OK with Python, then just let them use Python. What I was getting at was that this is a scenario where concerns over Python users needing to actually use Python has morphed into those same users now needing to use 3 languages.
- ZenPsycho 7y ago
  If you're templating YAML, it just means that you don't know that you can generate YAML. Dev ops is a very different world. Yes they can script, but that doesn't mean they will modify any part of your program, even a python config file directly. They will template it. If you find yourself thinking that your config language isn't powerful enough, the solution is to either generate it, or to fix the program to not require a complex config file. There is a long term, 10 year cost to making config turing complete that I don't believe is worth paying. It'll save you time up front, maybe, but I certainly am not going to be impressed with having to modify your program just to "configure" it. Finally, given that's the environment we're dealing with, from now on the rational thing to do would be to design config file formats expecting them to eventually get templated. Because that's how they get automated.

ZenPsycho 7y ago

Keep in mind that config files are not just about writing them. They are also about reading, and in my personal experience, when I encounter a heavily scripted config file in some javascript project, my inclination is to flatten it out to a plain JSON file just so I can see what it's actually doing. This is the beginning of the logic that lead to things like Automake, and completely impenetrable makefiles. The problem with programmable scripted config isn't the writing of it. it's the reading and maintenance. Programmed config is a layer of obfuscation, that I, as a maintainer, need to spend time and mental effort decoding to get work done.

danShumway 7y ago
I'm sympathetic to this, and I suspect the original author is as well (see their comments on Emacs). I think you're making a very good point that people need to keep in mind when they write configs. However, very specifically where Makefiles are concerned: > This is the beginning of the logic that lead to things like Automake, and completely impenetrable makefiles. The problem with Make in specific isn't scripting the build, it's that Make is a completely separate domain-specific language that isn't particularly well structured or particularly elegant in the first place. The comparison I would draw is with Grunt. Grunt is also a programmable config that allows you to do powerful stuff. It's also IMO a giant disaster and most projects shouldn't use it. Grunt is using programming to control how a project builds, but it's also tying tons of layers of abstraction into this secondary API that's unreasonably difficult to debug, and that lends itself to really bad, unnecessarily verbose control flows. I was teaching interns how our build process worked at a company, and I had an "aha" moment when I realized that in the time I could teach them Grunt's API, I could also teach them how to write a maybe 100 line pure-JS script that did literally everything Grunt was doing for us. When I look at Makefiles and Gruntfiles and cringe, very often the thing that makes me cringe are these giant pillars of abstractions and control flows of "program X calls program Y and pipes its output to program Z, and good luck intercepting that specific output to see what's going wrong". There's something to be said for getting rid of these giant build systems that are spanning multiple processes and languages in general, and instead doing something very direct in one language in one file. And I kind of think that's a concern that cuts across different config types. At my current job, we're in the process of dropping Team City, because we realized that for our needs, we can get by with a single 30-ish line bash script that gets copied to each build machine and auto-run every 5 minutes. And Team City is using a lot of declarative options in a very structured web UI, but the Bash script is still a lot easier to debug and reason about than Team City's whole... thing. Which is not to say you're wrong, I do think you're making a completely valid point. But I don't think the obfuscation you're talking about is inevitable, I think that's just a side effect of programmers thinking they're clever and trying to make generalized build systems for specific tasks.