6 ms·

You know what else is human readable, easy to parse if you're using PHP, and supports comments? PHP. I understand why some languages rely on common configurat

You know what else is human readable, easy to parse if you're using PHP, and supports comments? PHP. I understand why some languages rely on common configuration file formats. I don't understand why the popular dynamic script-y languages don't more commonly use the natively-expressable associative/list data structures that they're famous for making convenient.

krapp 8y ago
>I don't understand why the popular dynamic script-y languages don't more commonly use the natively-expressable associative/list data structures that they're famous for making convenient. You picked the wrong language... PHP comes with its own JSON parser. And INI and XML and even CSV. But, the reason is that, generally, you want config files to describe data or state only. Yes, you could just make your config native code, but then the temptation to add functions and methods and logic to that becomes irresistible and soon your config is an application that needs its own config. Config formats need to be simple, and preferably not Turing complete.
- user5994461 8y ago
  Any configuration will eventually become a programming language. See The Configuration Complexity Clock. https://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html https://mikehadlow.blogspot.com/2012/05/configuration-comple...
  - krapp 8y ago
    Just because it's common doesn't mean it should be encouraged by writing your config in native code to begin with. INI is still simple, and JSON doesn't support logic, so the madness can be held at bay at least for a time. XML and s-expressions are lost causes, though.
akavel 8y ago
Interestingly, the Lua programming language actually evolved from configuration files: https://www.lua.org/history.html https://www.lua.org/history.html (and is still officially deemed useful for writing them)
- DonHopkins 8y ago
  That was also one of the rationales behind TCL's design. John Ousterhout explained in one of his early TCL papers that, as a "Tool Command Language" like the shell but unlike Lisp, arguments were treated as quoted literals by default (presuming that to be the common case), so you don't have to put quotes around most strings, and you have to use punctuation like ${}[] to evaluate expressions. TCL's syntax is optimized for calling functions with literal parameters to create and configure objects, like a declarative configuration file. And it's often used that way with Tk to create and configure a bunch of user interface widgets. Oliver Steel has written some interesting stuff about "Instance-First Development" and how it applies to the XML/JavaScript based OpenLaszlo programming language, and other prototype based languages. Instance-First Development: https://blog.osteele.com/2004/03/classes-and-prototypes/ https://blog.osteele.com/2004/03/classes-and-prototypes/ >The equivalence between the two programs above supports a development strategy I call instance-first development. In instance-first development, one implements functionality for a single instance, and then refactors the instance into a class that supports multiple instances. >[...] In defining the semantics of LZX class definitions, I found the following principle useful: >Instance substitution principal: An instance of a class can be replaced by the definition of the instance, without changing the program semantics. In OpenLaszlo, you can create trees of nested instances with XML tags, and when you define a class, its name becomes an XML tag you can use to create instances of that class. That lets you create your own domain specific declarative XML languages for creating and configuring objects (using constraint expressions and XML data binding, which makes it very powerful). The syntax for creating a bunch of objects is parallel to the syntax of declaring a class that creates the same objects. So you can start by just creating a bunch of stuff in "instance space", then later on as you see the need, easily and incrementally convert only the parts of it you want to reuse and abstract into classes. What is OpenLaszlo, and what's it good for? http://www.donhopkins.com/drupal/node/124 http://www.donhopkins.com/drupal/node/124 Constraints and Prototypes in Garnet and Laszlo: http://www.donhopkins.com/drupal/node/69 http://www.donhopkins.com/drupal/node/69
  - pjmlp 8y ago
    In our Tcl based application server (many eons ago), we followed exactly that approach. All configuration files were Tcl data structures that were sourced on server start.
irascible 8y ago
Because code injection.
derefr 8y ago
You can use arbitrary tools to programmatically generate YAML (or JSON, or XML, any of the other "data only" formats.) This allows for tools to drive other tools by generating a spec file and feeding it in. See e.g. Kubernetes for a good example of that. There's no language that I'm aware of that can natively generate PHP syntax, and there's no common multi-language-platform library for generating PHP syntax. I think that's most of the reason. To contradict myself, though: Ruby encodes Gemfiles and Rakefiles as Ruby syntax. And Elixir encodes Mixfiles, Mix.Config files, Distillery release-config files, and a bunch of other common data formats as Elixir syntax. And, of course, pretty much every Lisp just serializes the sexpr representation of the live config for its config format (which means that, frequently, a lot of Lisp runs code at VM-bootstrap time, because people write Turing-complete config files.)
- wwweston 8y ago
  > There's no language that I'm aware of that can natively generate PHP syntax This is a solid argument against using PHP (or any such language) as a cross-language data interchange format. There are others :) And I totally agree you want a language independent format for anything you might have to feed across an ecosystem of tools. For a PHP-system generating/altering its own config files... PHP's `var_export` generates a PHP-parseable string representation of a variable (though it sadly doesn't use the short array syntax). Turing-complete config files probably have some hazards, like Lisp itself does. YMMV regarding whether those hazards can be avoided by circumspect developers or need to be fenced off.
  - scrollaway 8y ago
    You don't know when you'll need to generate or parse your config files with something that either can't read, write or execute your language. Django's settings.py sucks. I've used Django since the 0.9 days. It's extremely impractical and needs to be worked around constantly.
    - piva00 8y ago
      Yup, totally agree with you, settings.py has always been a pain in the ass. Not really an acute one but the kind that is uncomfortable but not enough to make you do something about it.
ramses0 8y ago
I think some of it is PLOP (Principle of Least Power). $CFG = random() > 0.5 ? "yes" : "no"; ...is likely "too powerful". It'd be nice if there were ways in certain programming languages to do something like "drop privileges" to avoid loops, function calls, external access, etc.
Jach 8y ago
People get really upset when they have to type "array(" instead of "[" or "{" (pre-PHP 5.something) and quotes instead of no quotes (and punting the character escape problem to something else) I guess. Using code-as-data works really well in Lisp-like languages. Reading a Clojure project's project.clj file or a Lisp project's project.asdf file is pretty pleasant. A programming language's choice in how it decides to handle library config info for building and specifying dependencies (XML, makefiles, JSON, YAML, INI, nothing, etc...) will be a good indicator for the culture of the language around config files in general. Composer for PHP only came out in 2012.
njbooher 8y ago
The makers of Drush, the cli for Drupal, subscribed to your line of thinking in the early versions and inventory items were defined in PHP files. Migrating from that will be interesting.
zmmmmm 8y ago
Because it's just in general incredibly short sighted to think that your config file is never going to be read by code written in another language. There's also an argument about whether making configuration files able to execute arbitrary code is a good idea. You get straight into the JavaScript 'eval' problems which we've spent a decade escaping.
- dcbadacd 8y ago
  Arbitrary code execution in configuration files has caused a few vulnerabilities in Wordpress extensions already, so yes, it's a terrible idea.
chx 8y ago
I have linked this https://groups.drupal.org/node/159044 https://groups.drupal.org/node/159044 elsewhere. Please note PHP was considered.
kakwa_ 8y ago
Using includes/imports is not the greatest idea ever. Your configuration file is one of your program interface. It's something that must be well define. If your configuration file is a programing language this interface is not that well defined. Also you expose yourself to all kind of weird bugs because some (too smart for their own good) people will monkey patch your software using it. It adds a lot of unnecessary stuff in the configuration file, things like ';' or '$' are not really useful. Lastly, common configuration file format are good because there are... common. You can have 2 pieces of software in 2 different languages accessing the same configuration file. A common example of that is configuration management, There are a lot of modules/formula in salt/ansible/puppet/chef doing fine parsing of the configuration files and permits fine grain settings, and I'm not mentioning augeas. If your configuration is a php/python/perl/ruby file good luck with that. I know it's really common for php applications to do configuration files in php, but frankly, it's a bit annoying.
- Sean1708 8y ago
  > If your configuration file is a programing language this interface is not that well defined. While I do agree with the rest of your comment I don't think they were advocating using the full language for configuration, just the maps/arrays/etc. (e.g. Python's `literal_eval`).
  - kakwa_ 8y ago
    true, but some user will use the full language. Something like: config = {'key1': 'value1', 'key2': 'value2'} could be written as: config = {} config['key1'] = 'value1' config['key2'] = 'value2' With large chunk possible between the 3. It basically transforms the configuration file into an API like any library, which is not really what you want for an end user program.
    - wwweston 8y ago
      If a key objection/perceived threat is that this might give someone an insertion point they're not meant to use for code ... well, let's consider that we're talking about applications distributed as interpreted language source here. Disallowing code-as-config isn't even closing the door of this particular barn after the horse has left, it's putting two strands of police line tape across the bottom half of the gap where the door was never installed and hoping any equines thinking of passage politely consider the message in case it hadn't already occurred to them which side of the entrance they preferred to be on. Consider this: Design and optimize for the common case. Why do we have config files? Because developers actually want a place dedicated to simple or structured application configuration data, for which PHP assignments with arrays + primitives can function at least as effectively as JSON. Most developers would prefer that config data get loaded quickly so the application can get on to doing actual app-y things. Using the language for this means you're parsing at least as fast as you can interpret and you can also take advantage of any code caching that's part of your deployment (especially nice in the PHP-likely event that config settings would be reloaded with every request). Abuse isn't likely to be the common case. The end users you invoked certainly aren't going to be the ones looking for opportunities to insert code over data. Developers have other places to put code and, as mentioned, probably actually want a place dedicated to data. You're still right that of course someone will do it, just like someone will inevitably create astronaut architecture hierarchy monstrosities in any language with classical inheritance or make potentially hidden/scary changes to language function using metaprogramming facilities. But potential for abuse doesn't automatically mean a feature should be disallowed. A lot of the time it's better to let people who can be circumspect have the benefits of a potential approach, and if somebody thinks they need to solve a problem by using a technique that's arguably abuse, well, let them either find out why it's a bad idea or enjoy having solved their problem in an unusual way. Not the end of the world. Possibly even legit.
Annatar 8y ago
Because that forces the end user, who might not know anything about the programming language one’s application is written in to wrestle with the low level implementation details. In the words of Keith Wesolowski, the programmer assumes that the end user is a “Linux superengineer”, which is almost always a wrong assumption to make.
pjmlp 8y ago
Fully agree with you, I have done so multiple times.

krapp 8y ago

>I don't understand why the popular dynamic script-y languages don't more commonly use the natively-expressable associative/list data structures that they're famous for making convenient. You picked the wrong language... PHP comes with its own JSON parser. And INI and XML and even CSV. But, the reason is that, generally, you want config files to describe data or state only. Yes, you could just make your config native code, but then the temptation to add functions and methods and logic to that becomes irresistible and soon your config is an application that needs its own config. Config formats need to be simple, and preferably not Turing complete.

user5994461 8y ago
Any configuration will eventually become a programming language. See The Configuration Complexity Clock. https://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html https://mikehadlow.blogspot.com/2012/05/configuration-comple...
- krapp 8y ago
  Just because it's common doesn't mean it should be encouraged by writing your config in native code to begin with. INI is still simple, and JSON doesn't support logic, so the madness can be held at bay at least for a time. XML and s-expressions are lost causes, though.

akavel 8y ago

Interestingly, the Lua programming language actually evolved from configuration files: https://www.lua.org/history.html https://www.lua.org/history.html (and is still officially deemed useful for writing them)

DonHopkins 8y ago
That was also one of the rationales behind TCL's design. John Ousterhout explained in one of his early TCL papers that, as a "Tool Command Language" like the shell but unlike Lisp, arguments were treated as quoted literals by default (presuming that to be the common case), so you don't have to put quotes around most strings, and you have to use punctuation like ${}[] to evaluate expressions. TCL's syntax is optimized for calling functions with literal parameters to create and configure objects, like a declarative configuration file. And it's often used that way with Tk to create and configure a bunch of user interface widgets. Oliver Steel has written some interesting stuff about "Instance-First Development" and how it applies to the XML/JavaScript based OpenLaszlo programming language, and other prototype based languages. Instance-First Development: https://blog.osteele.com/2004/03/classes-and-prototypes/ https://blog.osteele.com/2004/03/classes-and-prototypes/ >The equivalence between the two programs above supports a development strategy I call instance-first development. In instance-first development, one implements functionality for a single instance, and then refactors the instance into a class that supports multiple instances. >[...] In defining the semantics of LZX class definitions, I found the following principle useful: >Instance substitution principal: An instance of a class can be replaced by the definition of the instance, without changing the program semantics. In OpenLaszlo, you can create trees of nested instances with XML tags, and when you define a class, its name becomes an XML tag you can use to create instances of that class. That lets you create your own domain specific declarative XML languages for creating and configuring objects (using constraint expressions and XML data binding, which makes it very powerful). The syntax for creating a bunch of objects is parallel to the syntax of declaring a class that creates the same objects. So you can start by just creating a bunch of stuff in "instance space", then later on as you see the need, easily and incrementally convert only the parts of it you want to reuse and abstract into classes. What is OpenLaszlo, and what's it good for? http://www.donhopkins.com/drupal/node/124 http://www.donhopkins.com/drupal/node/124 Constraints and Prototypes in Garnet and Laszlo: http://www.donhopkins.com/drupal/node/69 http://www.donhopkins.com/drupal/node/69
- pjmlp 8y ago
  In our Tcl based application server (many eons ago), we followed exactly that approach. All configuration files were Tcl data structures that were sourced on server start.

irascible 8y ago

Because code injection.

derefr 8y ago

You can use arbitrary tools to programmatically generate YAML (or JSON, or XML, any of the other "data only" formats.) This allows for tools to drive other tools by generating a spec file and feeding it in. See e.g. Kubernetes for a good example of that. There's no language that I'm aware of that can natively generate PHP syntax, and there's no common multi-language-platform library for generating PHP syntax. I think that's most of the reason. To contradict myself, though: Ruby encodes Gemfiles and Rakefiles as Ruby syntax. And Elixir encodes Mixfiles, Mix.Config files, Distillery release-config files, and a bunch of other common data formats as Elixir syntax. And, of course, pretty much every Lisp just serializes the sexpr representation of the live config for its config format (which means that, frequently, a lot of Lisp runs code at VM-bootstrap time, because people write Turing-complete config files.)

wwweston 8y ago
> There's no language that I'm aware of that can natively generate PHP syntax This is a solid argument against using PHP (or any such language) as a cross-language data interchange format. There are others :) And I totally agree you want a language independent format for anything you might have to feed across an ecosystem of tools. For a PHP-system generating/altering its own config files... PHP's `var_export` generates a PHP-parseable string representation of a variable (though it sadly doesn't use the short array syntax). Turing-complete config files probably have some hazards, like Lisp itself does. YMMV regarding whether those hazards can be avoided by circumspect developers or need to be fenced off.
- scrollaway 8y ago
  You don't know when you'll need to generate or parse your config files with something that either can't read, write or execute your language. Django's settings.py sucks. I've used Django since the 0.9 days. It's extremely impractical and needs to be worked around constantly.
  - piva00 8y ago
    Yup, totally agree with you, settings.py has always been a pain in the ass. Not really an acute one but the kind that is uncomfortable but not enough to make you do something about it.

ramses0 8y ago

I think some of it is PLOP (Principle of Least Power). $CFG = random() > 0.5 ? "yes" : "no"; ...is likely "too powerful". It'd be nice if there were ways in certain programming languages to do something like "drop privileges" to avoid loops, function calls, external access, etc.

Jach 8y ago

People get really upset when they have to type "array(" instead of "[" or "{" (pre-PHP 5.something) and quotes instead of no quotes (and punting the character escape problem to something else) I guess. Using code-as-data works really well in Lisp-like languages. Reading a Clojure project's project.clj file or a Lisp project's project.asdf file is pretty pleasant. A programming language's choice in how it decides to handle library config info for building and specifying dependencies (XML, makefiles, JSON, YAML, INI, nothing, etc...) will be a good indicator for the culture of the language around config files in general. Composer for PHP only came out in 2012.

njbooher 8y ago

The makers of Drush, the cli for Drupal, subscribed to your line of thinking in the early versions and inventory items were defined in PHP files. Migrating from that will be interesting.

zmmmmm 8y ago

Because it's just in general incredibly short sighted to think that your config file is never going to be read by code written in another language. There's also an argument about whether making configuration files able to execute arbitrary code is a good idea. You get straight into the JavaScript 'eval' problems which we've spent a decade escaping.

dcbadacd 8y ago
Arbitrary code execution in configuration files has caused a few vulnerabilities in Wordpress extensions already, so yes, it's a terrible idea.

chx 8y ago

I have linked this https://groups.drupal.org/node/159044 https://groups.drupal.org/node/159044 elsewhere. Please note PHP was considered.

kakwa_ 8y ago

Using includes/imports is not the greatest idea ever. Your configuration file is one of your program interface. It's something that must be well define. If your configuration file is a programing language this interface is not that well defined. Also you expose yourself to all kind of weird bugs because some (too smart for their own good) people will monkey patch your software using it. It adds a lot of unnecessary stuff in the configuration file, things like ';' or '$' are not really useful. Lastly, common configuration file format are good because there are... common. You can have 2 pieces of software in 2 different languages accessing the same configuration file. A common example of that is configuration management, There are a lot of modules/formula in salt/ansible/puppet/chef doing fine parsing of the configuration files and permits fine grain settings, and I'm not mentioning augeas. If your configuration is a php/python/perl/ruby file good luck with that. I know it's really common for php applications to do configuration files in php, but frankly, it's a bit annoying.

Sean1708 8y ago
> If your configuration file is a programing language this interface is not that well defined. While I do agree with the rest of your comment I don't think they were advocating using the full language for configuration, just the maps/arrays/etc. (e.g. Python's `literal_eval`).
- kakwa_ 8y ago
  true, but some user will use the full language. Something like: config = {'key1': 'value1', 'key2': 'value2'} could be written as: config = {} config['key1'] = 'value1' config['key2'] = 'value2' With large chunk possible between the 3. It basically transforms the configuration file into an API like any library, which is not really what you want for an end user program.
  - wwweston 8y ago
    If a key objection/perceived threat is that this might give someone an insertion point they're not meant to use for code ... well, let's consider that we're talking about applications distributed as interpreted language source here. Disallowing code-as-config isn't even closing the door of this particular barn after the horse has left, it's putting two strands of police line tape across the bottom half of the gap where the door was never installed and hoping any equines thinking of passage politely consider the message in case it hadn't already occurred to them which side of the entrance they preferred to be on. Consider this: Design and optimize for the common case. Why do we have config files? Because developers actually want a place dedicated to simple or structured application configuration data, for which PHP assignments with arrays + primitives can function at least as effectively as JSON. Most developers would prefer that config data get loaded quickly so the application can get on to doing actual app-y things. Using the language for this means you're parsing at least as fast as you can interpret and you can also take advantage of any code caching that's part of your deployment (especially nice in the PHP-likely event that config settings would be reloaded with every request). Abuse isn't likely to be the common case. The end users you invoked certainly aren't going to be the ones looking for opportunities to insert code over data. Developers have other places to put code and, as mentioned, probably actually want a place dedicated to data. You're still right that of course someone will do it, just like someone will inevitably create astronaut architecture hierarchy monstrosities in any language with classical inheritance or make potentially hidden/scary changes to language function using metaprogramming facilities. But potential for abuse doesn't automatically mean a feature should be disallowed. A lot of the time it's better to let people who can be circumspect have the benefits of a potential approach, and if somebody thinks they need to solve a problem by using a technique that's arguably abuse, well, let them either find out why it's a bad idea or enjoy having solved their problem in an unusual way. Not the end of the world. Possibly even legit.

Annatar 8y ago

Because that forces the end user, who might not know anything about the programming language one’s application is written in to wrestle with the low level implementation details. In the words of Keith Wesolowski, the programmer assumes that the end user is a “Linux superengineer”, which is almost always a wrong assumption to make.

pjmlp 8y ago

Fully agree with you, I have done so multiple times.