4 ms·

great now integrate that in all the automation tools that devops use, and train them how to use the integration instead of just templating the config file. or

great now integrate that in all the automation tools that devops use, and train them how to use the integration instead of just templating the config file. or put another way, stop trying to reinvent the wheel and expect that OTHER people will script the generation of the config file, in their own way using their own tools and languages, and just make that easier for them. when devops are involved, the question isn't whether they're comfortable with whatever language, the question is how easy it will be for them to template the file, because they simply will not learn your special configuration language, whether they possibly could or not. The point isn't about whether we should make the automation people learn a new templating language. the point is they simply will not. they will find where the relevant parts of the config file are, and they'll stick a ${my_variable} in there, process it using their own tool, whatever it is, regardless of how cleverly (or not) you designed the scripting language around the config. And now whether your config file is python yaml or json is just irrelevant. They will not use python to loop through a file list, even if they could, or know how. They will template that. all you've done by switching your config to python is create a breeding ground for script injection attacks. So ultimately it's the wrong question. What I'm proposing is, design our config file expecting the ${subsititution_variable} will get edited in at some point, and you guard against script injection assuming that is comimng.

danShumway 7y ago
> great now integrate that in all the automation tools that devops use, and train them how to use the integration instead of just templating the config file. I don't understand what you're getting at. This is just a 3 line executable you call when you're done with whatever templating process you want to use. Ship it with the program. node generate_config.js path/to/json/templated/with/literally/any/tool.json But if you really think that a devops engineer will struggle calling that command, then fine. Put these 7 lines at the top of your program. var config = (()=>{ try { return require('./config.js'); } catch (err) { return JSON.parse(fs.readFileSync('./config.json'); } })(); Now your engineers can stick a `./config.json` in and template it with whatever automation tool they want in whatever integrated pipeline they want to work with. They never need to acknowledge that runtime option generation is possible. They never need to run even one extra shell command. So I just don't see this as a real problem. If your config script is ultimately returning a static object literal then there is no special knowledge with this setup that anyone needs to learn. All we're doing is giving people the option to script a config on a lower level in a real programming language. We're not inventing a new scripting language or toolchain. And it's still trivial for engineers to ignore that if they want to compile a config with a 3rd-party templating tool. > What I'm proposing is, design our config file expecting the ${subsititution_variable} will get edited in at some point, and you guard against script injection assuming that is coming. I would maintain that a config file isn't really where you should be handling security, that's something that should be ingrained into your program at a deeper, more fundamental layer. But granted -- if for some reason your application can't be run in a secure unprivileged environment, and you need to accept untrusted config files within that environment, and you also can't run the config generation code in a secure unprivileged environment, then it would make sense to strictly limit a config file to a static format like JSON, where you know that no one else can inject actual code. But that's a very different concern than usability, and affects an even smaller subset of software than what we were talking about above. This is just not relevant for systems like Github Actions, Babel, Webpack, or Ansible, all of which are built on either the assumption that the config is trustworthy, or that the trustworthiness of the config doesn't matter because of sandboxing. In most (not all, but most) cases, if you don't trust your user config, you're already at the point where you should be running your software in an unprivileged sandbox.
- chrisweekly 7y ago
  +1; for me, this comment sums up my feelings on the entire (worthwhile) thread.
- ZenPsycho 7y ago
  Yes of course you're right about all that. My only point is, it's a fact that devops add templating tags to YAML files. This is not theoretical. The templating tags are likely not escaped properly, so, aside from the worst case scenario of script injection, the more innocent situation of causing parsing problems is possible and likely. It's worth very seriously asking the question of WHY devops are doing this instead of just generating the yaml file with code, which from our perspective as developers would be easier and more sane, and allow us to use the full power of a programming language. I think it's a more important problem to solve than ergonomics of config for the original developers, because end of the day, who will need to interact with this configuration file more? Developers or the people responsible for deploying the application?

danShumway 7y ago

> great now integrate that in all the automation tools that devops use, and train them how to use the integration instead of just templating the config file. I don't understand what you're getting at. This is just a 3 line executable you call when you're done with whatever templating process you want to use. Ship it with the program. node generate_config.js path/to/json/templated/with/literally/any/tool.json But if you really think that a devops engineer will struggle calling that command, then fine. Put these 7 lines at the top of your program. var config = (()=>{ try { return require('./config.js'); } catch (err) { return JSON.parse(fs.readFileSync('./config.json'); } })(); Now your engineers can stick a `./config.json` in and template it with whatever automation tool they want in whatever integrated pipeline they want to work with. They never need to acknowledge that runtime option generation is possible. They never need to run even one extra shell command. So I just don't see this as a real problem. If your config script is ultimately returning a static object literal then there is no special knowledge with this setup that anyone needs to learn. All we're doing is giving people the option to script a config on a lower level in a real programming language. We're not inventing a new scripting language or toolchain. And it's still trivial for engineers to ignore that if they want to compile a config with a 3rd-party templating tool. > What I'm proposing is, design our config file expecting the ${subsititution_variable} will get edited in at some point, and you guard against script injection assuming that is coming. I would maintain that a config file isn't really where you should be handling security, that's something that should be ingrained into your program at a deeper, more fundamental layer. But granted -- if for some reason your application can't be run in a secure unprivileged environment, and you need to accept untrusted config files within that environment, and you also can't run the config generation code in a secure unprivileged environment, then it would make sense to strictly limit a config file to a static format like JSON, where you know that no one else can inject actual code. But that's a very different concern than usability, and affects an even smaller subset of software than what we were talking about above. This is just not relevant for systems like Github Actions, Babel, Webpack, or Ansible, all of which are built on either the assumption that the config is trustworthy, or that the trustworthiness of the config doesn't matter because of sandboxing. In most (not all, but most) cases, if you don't trust your user config, you're already at the point where you should be running your software in an unprivileged sandbox.

chrisweekly 7y ago
+1; for me, this comment sums up my feelings on the entire (worthwhile) thread.
ZenPsycho 7y ago
Yes of course you're right about all that. My only point is, it's a fact that devops add templating tags to YAML files. This is not theoretical. The templating tags are likely not escaped properly, so, aside from the worst case scenario of script injection, the more innocent situation of causing parsing problems is possible and likely. It's worth very seriously asking the question of WHY devops are doing this instead of just generating the yaml file with code, which from our perspective as developers would be easier and more sane, and allow us to use the full power of a programming language. I think it's a more important problem to solve than ergonomics of config for the original developers, because end of the day, who will need to interact with this configuration file more? Developers or the people responsible for deploying the application?