4 ms·

JSON is not a human friendly config format; it has harsh rules for quoting, brackets and commas and most importantly has no intuitive facility for comments. Y

by zzzeek 9y ago

JSON is not a human friendly config format; it has harsh rules for quoting, brackets and commas and most importantly has no intuitive facility for comments. YAML should be preferred to JSON for config. However, ConfigParser is even better and a list of IPs need only be space separated (eg hosts = config.get('hosts').split() , quite simple ) or you could even make the value of the parameter be a JSON list if you really like to type brackets and quotes. The "simplest" approach is in fact to use the most idiomatic approach that everyone recognizes, and if you're doing config files in python that means ConfigParser. Edit: it seems they went with a newly invented language called json-config that looks like a hybrid of JSON and C-style comments. That's fine, but using an invented config format that nobody knows isnt the "simplest" approach at all. It's the most complicated solution. When people use invented idioms in their projects for things like config and logging that have widely accepted idiomatic solutions included in Python std lib that's an immediate red flag.

woolvalley 9y ago
I find it very frustrating to hand write yaml personally. Always get some indentation or substructure rule wrong. JSON isn't as human friendly to write, but it's easier to avoid mistakes with it. Personally the ini format is probably the most ideal config format. Simple to write, 1 dimensional, has comments.
- zzzeek 9y ago
  Ansible is a whole programming platform that is based on YAML. We write thousands of lines in it at Red Hat. Ansible tells you immediately where you have syntax errors and it is much easier to read and write than JSON. But for config, it is usually overkill. If ConfigParser isn't working for your config job you should try to see what it is you're doing that requires such complex config (which again is all about doing the simplest approach possible).
  - philipov 9y ago
    How does Ansible handle defining a subdirectory of a previously-defined directory? Is it more complicated than "${BASEDIR}/subdir"?
e12e 9y ago
True about the comments for json. But you could use: "_comment": "Some comment" or a similar convention. But you'd have to handle it somehow when instantiating objects. I do like python style docstring comments that are available in the object graph. Which would allow for checking (live) config comments at run time: config.server => "10.x.x" config.server.comment => "This is on a private ip network because..."
philipov 9y ago
I like YAML, but it needs shell-style $ substitution tokens to allow constructing values from previously-defined values (such as when defining a PATH variable, or subdirectory). To that end, I'm going down the path of writing my own extension to YAML. Can you recommend one that already exists that accomplishes this goal? I don't see an alternative, because I can't find something standard that actually meets the requirements. Shell scripting languages are platform-specific, and not hierarchically structured, so that's what I'm trying to replace to begin with. Ansible requires a central server and full commitment to immutable deployments, so it's too heavy for a package manager that can support piecemeal adoption or that could be used by researchers doing ad-hoc development. It doesn't support windows anyway (linux for windows subsystem doesn't count). Conda is nice, but even though it uses YAML for defining dependencies, it nonetheless falls back to shell scripts for environment variables, and doesn't offer features for environment modularization. EDIT: Hierarchical structure is needed to be able to both express and manipulate configs for software that expects input as XML, so ConfigParser isn't good enough. Setting a value to a blob of JSON or XML denies the ability to operate on that blob to do things like merge subtrees. Instead, I'm using ruamel.yaml as my starting point, and adding post-processing after it's converted to nested dict/lists.
- zzzeek 9y ago
  > Ansible requires a central server and full commitment to immutable deployments i dont know what that means. Ansible is like a much more structured form of shell scripting, you can run ansible playbooks from anywhere to do anything and make them do anything. i don't know what an "immutable deployment" is. also ansible has nothing to do with "config", YAML is used as a declarative scripting language rather than a config file format.
  - philipov 9y ago
    Really, I just want to know how ansible provides for token substitution, so I could reuse a value defined elsewhere, so I could define all the subdirectories in a playbook relative to a single base directory. YAML doesn't offer a syntax for writing "${BASEDIR}/subdir". There's gotta be something in ansbile that makes this possible, but I can't find it.

by zzzeek 9y ago

woolvalley 9y ago

I find it very frustrating to hand write yaml personally. Always get some indentation or substructure rule wrong. JSON isn't as human friendly to write, but it's easier to avoid mistakes with it. Personally the ini format is probably the most ideal config format. Simple to write, 1 dimensional, has comments.

zzzeek 9y ago
Ansible is a whole programming platform that is based on YAML. We write thousands of lines in it at Red Hat. Ansible tells you immediately where you have syntax errors and it is much easier to read and write than JSON. But for config, it is usually overkill. If ConfigParser isn't working for your config job you should try to see what it is you're doing that requires such complex config (which again is all about doing the simplest approach possible).
- philipov 9y ago
  How does Ansible handle defining a subdirectory of a previously-defined directory? Is it more complicated than "${BASEDIR}/subdir"?

e12e 9y ago

True about the comments for json. But you could use: "_comment": "Some comment" or a similar convention. But you'd have to handle it somehow when instantiating objects. I do like python style docstring comments that are available in the object graph. Which would allow for checking (live) config comments at run time: config.server => "10.x.x" config.server.comment => "This is on a private ip network because..."

philipov 9y ago

I like YAML, but it needs shell-style $ substitution tokens to allow constructing values from previously-defined values (such as when defining a PATH variable, or subdirectory). To that end, I'm going down the path of writing my own extension to YAML. Can you recommend one that already exists that accomplishes this goal? I don't see an alternative, because I can't find something standard that actually meets the requirements. Shell scripting languages are platform-specific, and not hierarchically structured, so that's what I'm trying to replace to begin with. Ansible requires a central server and full commitment to immutable deployments, so it's too heavy for a package manager that can support piecemeal adoption or that could be used by researchers doing ad-hoc development. It doesn't support windows anyway (linux for windows subsystem doesn't count). Conda is nice, but even though it uses YAML for defining dependencies, it nonetheless falls back to shell scripts for environment variables, and doesn't offer features for environment modularization. EDIT: Hierarchical structure is needed to be able to both express and manipulate configs for software that expects input as XML, so ConfigParser isn't good enough. Setting a value to a blob of JSON or XML denies the ability to operate on that blob to do things like merge subtrees. Instead, I'm using ruamel.yaml as my starting point, and adding post-processing after it's converted to nested dict/lists.

zzzeek 9y ago
> Ansible requires a central server and full commitment to immutable deployments i dont know what that means. Ansible is like a much more structured form of shell scripting, you can run ansible playbooks from anywhere to do anything and make them do anything. i don't know what an "immutable deployment" is. also ansible has nothing to do with "config", YAML is used as a declarative scripting language rather than a config file format.
- philipov 9y ago
  Really, I just want to know how ansible provides for token substitution, so I could reuse a value defined elsewhere, so I could define all the subdirectories in a playbook relative to a single base directory. YAML doesn't offer a syntax for writing "${BASEDIR}/subdir". There's gotta be something in ansbile that makes this possible, but I can't find it.