10 ms·

Why JSON Isn’t A Good Configuration Language (2018)

63 points by optimalsolver 4y ago

wgj 4y ago
> Language JSON is only a data _format_, not a language. I agree it's awful for maintaining configs. YAML has issues with the more complex parts of its spec, but it's great for the stuff JSON gets used for.
- scubbo 4y ago
  What is the distinction between a data format and a data/configuration language? (It's clearly not a programming language, but that wasn't the claim)
- kelnos 4y ago
  I've started liking TOML quite a bit. I am so tired of getting bitten by YAML's weird parsing behaviors. I still sometimes forget to quote strings, and then end up having the string "no" get interpreted as a boolean "false", and then spend an embarrassing amount of time confused as to what's going on.
arielcostas 4y ago
Why not use XML? Fairly similar to HTML, which almost everyone has probably seen/used at least once, and it's kinda easy to understand. It supports comments, attributes and elements and schemas (which help with validation, editor autocompletion and documentation), and is also widely supported in many languages.
- billconan 4y ago
  too verbose. not ergonomic.
- wronglyprepaid 4y ago
  XML is incredibly complicated to process correctly, the support for various features, like schema validation, is also rather limited.
  - arielcostas 4y ago
    Most languages already include an XML in their standard library, you don't need to implement it yourself
- 8lall0 4y ago
  Hell no, horrible language, too verbose.
  - arielcostas 4y ago
    I prefer XML's verbosity to a full disfunctional language like YAML where adding an extra space might break your config
linkdd 4y ago
> Write your own No. No. No no no. Definitely no. I wish for a standardized configuration format so that I don't have to learn a new format for each software. At least JSON, TOML (and INI) and YAML are widely used enough so there is little chances that you don't know them.
- ComradePhil 4y ago
  They suggest "Writing your own" after they suggest the formats you mention and they clearly do not recommend it if anything else works for you: > If for some reason a key-value configuration format doesn’t meet your needs, and you can’t use a scripting language due to performance or size constraints, then it might be appropriate to write your own configuration format. But if you find yourself in this scenario, think long and hard before making a choice that will not only require you to write and maintain a parser but also require your users to become familiar with yet another configuration format.
  - linkdd 4y ago
    The question I must ask then is: When does a key-value configuration format not meet your needs? Using a scripting language is still about setting key/values. No, I claim that writing your own is *never* an appropriate choice. So "think long and hard and then say no".
    - giaour 4y ago
      Sometimes you need to support something more flexible than JSON but just can't hand your (non-dev) users all the footguns that come with a Turing complete configuration language. "Key/Value" makes the problem sound simple, but maybe you're writing a rules engine, and the values are expressions that will be evaluated at runtime. It's uncommon, but the need isn't impossible to imagine. I'm personally of the opinion that using a scripting language is the worst option. Going that route is giving up on most forms of static analysis, since you can't inspect configuration values without executing untrusted code.
      - 4y ago
120bits 4y ago
I went from simple flat file with tab separated configs to xml to Protobuf and to JSON/TOML. Every time we had issues, it was because we were trying to spin our OWN version of configuration file. It breaks backward compatibility and versioning. Code collaboration was horrible and devs were getting frustrated.
mdmglr 4y ago
Write your own is a bad option. I have tried it and found that you end up spending lots of time maintaining a parser. I have found some success in my org using JSON with JSON Schema [1]. Combine with a json schema aware IDE like VS Code it solves the documentation problem. [1] https://json-schema.org/ https://json-schema.org/
- gnulinux 4y ago
  But article literally makes the same point you're making. They only put "make your own" as the last resort option. Here: > If for some reason a key-value configuration format doesn’t meet your needs, and you can’t use a scripting language due to performance or size constraints, then it might be appropriate to write your own configuration format. But if you find yourself in this scenario, think long and hard before making a choice that will not only require you to write and maintain a parser but also require your users to become familiar with yet another configuration format.
Gravyness 4y ago
> With so many better options for configuration languages, there’s no good reason to use JSON I have been surprised by how silly that conclusion is, even though it's from 2018. JSON is so bad it is on the standard library of all major languages, natively supported by php, python, nodejs, javascript, ruby and many others, unlike YAML, TOML, FADFH or whatever solution someone comes up to a problem that is essentially solved. If you want perfection then by all means reinvent the wheel. But otherwise, don't be an edgy teen, just go with what the entire industry tells you that works.
- linkdd 4y ago
  In C++ you get the great https://github.com/nlohmann/json https://github.com/nlohmann/json In Rust you have the amazing serde with serde_json but at this point, you can use toml which is also based on serde. I consider serde as being standard. In C you have the lib jason which is very good. In Elixir, I use the compile-time configuration (config/config.exs) with environment variables for production, but that's because in the end, my Elixir system is in a Docker container, running on Kubernetes, with a ConfigMap defining the environment variables, so in the end, it's YAML (or JSON).
- valbaca 4y ago
  > unlike YAML, TOML, FADFH or whatever solution TOML and YAML are each supported by every language you listed. https://github.com/toml-lang/toml/wiki https://github.com/toml-lang/toml/wiki https://yaml.org/ https://yaml.org/
  - hillcrestenigma 4y ago
    They aren't supported by the standard libraries of those languages though. It is still a major advantage of JSON.
    - lmm 4y ago
      Does that matter nowadays? Unless you're writing a small script in a language with terrible dependency management like Python, you're going to have a bunch of non-standard-library dependencies, one more doesn't make much difference.
disintegrator 4y ago
CUE has been an absolute joy to work with on configuration problems. It’s a typed language which is something that’s been a massive gap in the config language space when looking at yaml, toml, json, etc... https://cuelang.org/ https://cuelang.org/
- betwixthewires 4y ago
  Why would you need typing in a configuration file? I would think a configuration file would be specific to your program and any interpretation of data would be handled by your program.
  - planckscnst 4y ago
    CUE's type system is also its validation system. It's extremely flexible. You can define custom types and your own constraints on those types. It allows for disjunctive constraints (e.g. this value must be an integer > 0 or the string value "none"). The types compose well together. I'd highly recommend reading through the docs - they're great.
  - arccy 4y ago
    if your config is longer and semi repetitive, you can use it to template things out while still having a strong schema at every step
  - disintegrator 4y ago
    My initial comment was very vague and I'm glad to see others have replied to fill in the gaps. If your service/system has a sufficiently large configuration space, like say, Kubernetes, then a typed configuration language can greatly improve the development experience by spotting errors early on. You get a very fast feedback loop that you set the wrong value for some config option before attempting to deploy the change. Different services will give you appropriate feedback about wrong config options but that is sometimes a few steps removed from your development environment e.g. You might find out after you push a PR and CI/CD fails or after you merge the PR even. The type system also has great second order benefits like allowing us to build an language server protocol implementation for CUE that has rich diagnostics, auto-complete, jump to definition, rename symbol features. Something that cannot be done _as well_ effectively in untyped languages. I'm still scratching the surface. CUE does more than add types to config and I would encourage you to dig into it if you have spare time. By virtue of providing a great type system, it also manages to reduce boilerplate as yet another second order benefit. Boilerplate reduction is something where tools like Jsonnet attack as a primary goal but over enough time you're back at having seas of untyped config and indirection that are hard to navigate or contribute to and you're back at square one.
hsbauauvhabzb 4y ago
I learned about json5 on news.ycombinator, which turned json config from an absolute pain to work with, to something more tolerable. Thank you kind stranger who recommended it. Edit: one thing I think all json python libs suck at is error messages. A syntax error in a json tree shouldn’t require me to rip nodes out to a/b test a parse error. Sure it can’t correctly break down AST but it could at least try to provide something more effective than Exception(f”{line} {col} glhf!”)
princevegeta89 4y ago
JSON is not a language it's just a format.
- rfiat 4y ago
  I disagree. It has a formal grammar so at least in a CS sense [0] it's a language. Why is JSON not a language but e.g. TOML or YAML are? [0] https://en.wikipedia.org/wiki/Formal_language https://en.wikipedia.org/wiki/Formal_language
betwixthewires 4y ago
Seems I'm in the minority here, but I agree with the author on his main point. JSON is fantastic for organizing and passing data around, storing it, even for configuration files where the user doesn't need to edit the raw configuration. If you are like me and write a lot of tools that run in the background, lack a UI and where the only thing a user will be doing is editing a configuration file and running it, you're asking for trouble with JSON. Curly brackets and nesting and lack of basic formatting are bad for UX, and while it's a fantastic compromise for machine and human readability, it focuses more on machine readability and the human readability is just enough for you to look at it. It is not easy for a human to parse with their eyes. I go with TOML for my configuration files most of the time. I am not a fan of YAML or XML. I don't think those offer any real benefit over JSON, and both have more downsides than JSON. TOML is a breeze to read and understand just looking at it, and is analogous to JSON as far as machine readability to the point that it's trivial to convert to it when you need to. I do use JSON a lot, even for configuration files, but only when I'm storing a configuration in a flat file that the user should never have to see.
- kcartlidge 4y ago
  I'd agree. JSON is user-readable, but not really user-friendly and user-editable. I'm okay with YAML but TBH, and I know this makes me totally uncool, for config that only nests to a single level I quite like INI files. I sometimes cheat slightly and take account of the ordering of entries in a section but generally speaking it's easy to read, easy to write, and very easy to parse.
- dan_hawkins 4y ago
  Also: no comments in JSON. This makes self-documented config files difficult to create and maintain.
molyss 4y ago
I’m totally happy using the properties format from java. There’s comments, it’s easy to grep. Main missing part are value types and multiline (or maybe it’s there, I’m not sure). The parameter reference substitution would be nice to have too.
angarg12 4y ago
We use tons of configuration at work and this has come up time and time again. The biggest pain is when inevitably you need to "parameterize" your JSON files. The most common route I've seen is to turn your configs into templates e.g. Jinja and go from there. Welcome to hell. My solution? Write a DSL with Typescript, and "compile" your configs down to JSON. Define the structure of your configs as TS types. Write functions with free variables to work as templates, or define snippets as composable fragments. Writing the end result to a JSON file results in transparency (what you see is what you get) and compatibility.
- unrealhoang 4y ago
  jsonnet[1] might suit your use-case better, it was created to do exactly that. [1]https://jsonnet.org https://jsonnet.org
  - Matheus28 4y ago
    Not OP. Looks interesting, but I'd bet that typescript's type system would enforce constraints a lot better
    - whacked_new 4y ago
      in my experience, for things like static configs, jsonnet has been superior to TS for constraints, if you pair the configs with a json schema (also generated from jsonnet, of course). TS's types are easier to use at write-time, but json schema includes a lot of batteries that save a lot of time when you need them, but become annoying in TS, like patternProperties in dictionaries, length constraints in keys. Of course, there are situations like key1 XOR key2 where you will need custom logic one way or another, but given json schema's evolution over the years, I think it's pretty solid. An added benefit is that if you stay in json schemas, you're almost guaranteed a validator exists in $other_language, it _almost_ feels like a first-class construct everywhere.
      - Matheus28 4y ago
rektide 4y ago
{"this_article": "// would be better", "this_article": "// if it werent", "this_article": "// immediately", "this_article": "// right off the bat", "this_article": "// wrong", "this_article": "meh"} //=> {"this_article": "meh"} Hacker spirit represent. Where there's a will, there's a way. (Death to all oppressors!) I actually agree, JSON is kind of a pain in the ass to work with. It's shocking to think of a pre-'jq' world, given how saturated we all are with JSON, but we lived like that for a long time. (We also evidently didn't even have Firebug until 2006?[1] Holy shit!) But is it a problem? I dunno. Is it worth doing something else? Ooof, scary proposition. I think one of the missing questions we don't ask, don't know is: what tools help us fix our json fast. What tools can see, oh there's a comma missing between these two lines, and just fix it for us. [1] https://en.wikipedia.org/wiki/Firebug_(software) https://en.wikipedia.org/wiki/Firebug_(software)
- tikhonj 4y ago
  As far as I can tell, you're relying on non-standard behavior. This is what [the standard][1] has to say about keys being unique: > * The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.* This means that different tools can—and, if experience is any guide, will—handle your example inconsistently. This can be especially troubling when JSON processing tools are used internally by some other system like a database, queue or server; most of the time these tools parse and serialize JSON without changing its meaning, but they can change or lose information when people rely on patterns like the one that you used. I would honestly be surprised if there isn't at least one realistically used tool that interprets your example differently, like producing {"this_article": "// would be better"} instead. [1]: https://www.ecma-international.org/publications-and-standards/standards/ecma-404/ https://www.ecma-international.org/publications-and-standard...
  - rektide 4y ago
    I'd love to see the counter-evidence! Works on every browser, every server-side JS engine of note. Edit: I believe a number of these non-determinism did get formally resolved in EcmaScript itself, which is yes, not the same, but is an active, alive, & notable specification with some bearing.
dub 4y ago
My wish for configuration languages is that as an industry we continue to adopt scriptable build systems like Bazel that make it easy to transform human-written configuration into machine-readable configuration during compile time. Want comments in JSON? Spend 20 minutes refactoring a single BUILD rule and now humans can write JSON5 that's transformed into JSON Want more flexibility? Have humans write cue or dhall or jsonnet Wish you also had a copy of a subset of the same config data in YAML? Easy Want to write a compile-time check in a programming language of your choice that a certain setting is never missing? Easy Want all of this to work with reproducible builds across a variety of computers and distributed build farms? Easy We don't have to let legacy build systems limit our imagination.
fb03 4y ago
I actually really like RON[1] [1] https://github.com/ron-rs/ron https://github.com/ron-rs/ron
throw0101a 4y ago
How about ISC-style like for BIND? * https://bind9.readthedocs.io/en/latest/reference.html#configuration-file-grammar https://bind9.readthedocs.io/en/latest/reference.html#config... * https://wiki.debian.org/Bind9#File_.2Fetc.2Fbind.2Fnamed.conf https://wiki.debian.org/Bind9#File_.2Fetc.2Fbind.2Fnamed.con... * https://www.zytrax.com/books/dns/ch7/#overview https://www.zytrax.com/books/dns/ch7/#overview I've been experimenting with NSD (and Unbound) recently, and their configuration is JSON-like. There seem to be some 'arbitrary' attributes that are treated as "top-level": > At the top level only server:, key:, pattern:, zone:, tls-auth:, and remote-control: are allowed. These are followed by their attributes or a new top-level keyword. The zone: attribute is followed by zone options. The server: attribute is followed by global options for the NSD server. […] * https://nsd.docs.nlnetlabs.nl/en/latest/manpages/nsd.conf.html#file-format https://nsd.docs.nlnetlabs.nl/en/latest/manpages/nsd.conf.ht... It seems to be good/best practice to indent the sub-attributes of the 'top-level' attributes to know where the 'stanzas' for each top-level start and end. The white space has no significance. While I like the functionality of NSD/Unbound, I lean toward liking the use of braces (curly brackets; {}) à la BIND to explicitly denote stanzas and statements, even if one also uses indentation for human consumption.
ryanthedev 4y ago
Idk. I feel like most things are YAML based or now some custom flavor (bicep, hcl, Nginx) But that’s more from the IaaC world. I guess .net application configuration might still be heavy on the JSON side. I just feel like this isn’t as much of a problem anymore. There are so many ways to utilize alternatives.
krapp 4y ago
(clears throat, taps microphone.) LuaJIT. Or at least Lua. Thank you for coming to my TED talk.
phendrenad2 4y ago
This is why I hate the Javascript ecosystem with a burning passion, and consider every single person who has contributed to it to be the very portrait of supremely incompetence. Everyone in web dev spent 2010-2020 schlepping these intolerable JSON configuration files around, unable to add comments to document what various lines did, simply because the JS ecosystem developers couldn't be bothered to implement a YAML parser. Le sigh.
- shepherdjerred 4y ago
  Yikes.
ukoki 4y ago
JSON is a great configuration language: * It's in the standard library for most languages so you can write tools that read and write it that have zero dependencies. This is great for portability and for use in offline environments. * You can use the same JSON files as input for tools that expect JSON (eg Terraform .tfvars.json) and tools that that expect YAML (anything kubernetes related) * You can easily see at a glance exactly what whitespace characters are in multiline strings like PEM certificate strings. This is useful to avoid LF vs CRLF line ending issues, and ensuring start and end whitespace is exactly what you want to make sure input strings are compatible with whatever is consuming the configuration. The downside is that it's a little bit ugly compared to YAML, but editor auto-formatting and key-sorting plugins make that a minor issue.
- luismedel 4y ago
  IMHO points 1 and 2 are more community efforts rather than JSON's design strengths. Regarding point 3, are you talking about using \n in single-line strings (the ones you can use in JSON) I think that the main reason this debate exists is we are using a good data-format (which JSON is) to configure things, which it's not a use case JSON is designed for.
aiddun 4y ago
I really like the solution Tailwind and some other JavaScript tools have taken to this where instead of a tailwind.json, there’s a tailwind.config.js, which is a plain ol JS file that exports a JS object. Allows for importing constants from other modules, scripting, comments, conditionals, etc
woojoo666 4y ago
My favorite so far is StrictYaml [1]. It's a subset of yaml that directly addresses a lot of the article's concerns: * supports comments * flexible with good signal to noise (no need for so many brackets and commas) * gets rid of a lot of YAML's complexity [2] * explicitly typed for less ambiguities [3] The main page also gives comparisons against HJSON, HOCON and TOML [4] [1]: https://hitchdev.com/strictyaml/ https://hitchdev.com/strictyaml/ [2]: https://hitchdev.com/strictyaml/features-removed/ https://hitchdev.com/strictyaml/features-removed/ [3]: https://hitchdev.com/strictyaml/why/implicit-typing-removed/ https://hitchdev.com/strictyaml/why/implicit-typing-removed/ [4]: https://hitchdev.com/strictyaml/#why-strictyaml https://hitchdev.com/strictyaml/#why-strictyaml
- Spivak 4y ago
  StrictYAML throws out too much for my taste — tags, flow state, and anchors are all useful non-footgun features.
  - woojoo666 4y ago
    To me those are in the category of "nice to have", and the problem is that every developer has different preferences for these [1] [2]. But the main features of StrictYaml, like supporting comments and less syntactic noise, I think are pretty uncontroversial, and perhaps it's worth it to get people to switch over for those alone. It doesn't need to be perfect, it just needs to be a significant enough improvement over JSON, and I'd say those two features are more than enough [1]: https://github.com/crdoconnor/strictyaml/issues/37 https://github.com/crdoconnor/strictyaml/issues/37 [2]: https://github.com/crdoconnor/strictyaml/issues/38 https://github.com/crdoconnor/strictyaml/issues/38
benibela 4y ago
All the text formats are bad and verbose Everything is much more efficient with binary format like protobufs or flatbuffers. And if the format has a schema, the schema is the documentation, an d you do not need to have comment in the config file itself.
- terrabitz 4y ago
  As a transport I agree. But as a config file format? I'm a little skeptical. The main benefit of a flat file format for configs is that it can be added to source control and edited by any plaintext editor. If I kept config as protobuf, I would need special tools to handle it, which could be annoying (e.g. if I'm remoted into a remote system). It would also make it more difficult to see diffs between different config versions stored in git.

63 points by optimalsolver 4y ago

wgj 4y ago

> Language JSON is only a data _format_, not a language. I agree it's awful for maintaining configs. YAML has issues with the more complex parts of its spec, but it's great for the stuff JSON gets used for.

scubbo 4y ago
What is the distinction between a data format and a data/configuration language? (It's clearly not a programming language, but that wasn't the claim)
kelnos 4y ago
I've started liking TOML quite a bit. I am so tired of getting bitten by YAML's weird parsing behaviors. I still sometimes forget to quote strings, and then end up having the string "no" get interpreted as a boolean "false", and then spend an embarrassing amount of time confused as to what's going on.

arielcostas 4y ago

Why not use XML? Fairly similar to HTML, which almost everyone has probably seen/used at least once, and it's kinda easy to understand. It supports comments, attributes and elements and schemas (which help with validation, editor autocompletion and documentation), and is also widely supported in many languages.

billconan 4y ago
too verbose. not ergonomic.
wronglyprepaid 4y ago
XML is incredibly complicated to process correctly, the support for various features, like schema validation, is also rather limited.
- arielcostas 4y ago
  Most languages already include an XML in their standard library, you don't need to implement it yourself
8lall0 4y ago
Hell no, horrible language, too verbose.
- arielcostas 4y ago
  I prefer XML's verbosity to a full disfunctional language like YAML where adding an extra space might break your config

linkdd 4y ago

> Write your own No. No. No no no. Definitely no. I wish for a standardized configuration format so that I don't have to learn a new format for each software. At least JSON, TOML (and INI) and YAML are widely used enough so there is little chances that you don't know them.

ComradePhil 4y ago
They suggest "Writing your own" after they suggest the formats you mention and they clearly do not recommend it if anything else works for you: > If for some reason a key-value configuration format doesn’t meet your needs, and you can’t use a scripting language due to performance or size constraints, then it might be appropriate to write your own configuration format. But if you find yourself in this scenario, think long and hard before making a choice that will not only require you to write and maintain a parser but also require your users to become familiar with yet another configuration format.
- linkdd 4y ago
  The question I must ask then is: When does a key-value configuration format not meet your needs? Using a scripting language is still about setting key/values. No, I claim that writing your own is *never* an appropriate choice. So "think long and hard and then say no".
  - giaour 4y ago
    Sometimes you need to support something more flexible than JSON but just can't hand your (non-dev) users all the footguns that come with a Turing complete configuration language. "Key/Value" makes the problem sound simple, but maybe you're writing a rules engine, and the values are expressions that will be evaluated at runtime. It's uncommon, but the need isn't impossible to imagine. I'm personally of the opinion that using a scripting language is the worst option. Going that route is giving up on most forms of static analysis, since you can't inspect configuration values without executing untrusted code.
    - 4y ago

120bits 4y ago

I went from simple flat file with tab separated configs to xml to Protobuf and to JSON/TOML. Every time we had issues, it was because we were trying to spin our OWN version of configuration file. It breaks backward compatibility and versioning. Code collaboration was horrible and devs were getting frustrated.

mdmglr 4y ago

Write your own is a bad option. I have tried it and found that you end up spending lots of time maintaining a parser. I have found some success in my org using JSON with JSON Schema [1]. Combine with a json schema aware IDE like VS Code it solves the documentation problem. [1] https://json-schema.org/ https://json-schema.org/

gnulinux 4y ago
But article literally makes the same point you're making. They only put "make your own" as the last resort option. Here: > If for some reason a key-value configuration format doesn’t meet your needs, and you can’t use a scripting language due to performance or size constraints, then it might be appropriate to write your own configuration format. But if you find yourself in this scenario, think long and hard before making a choice that will not only require you to write and maintain a parser but also require your users to become familiar with yet another configuration format.

Gravyness 4y ago

> With so many better options for configuration languages, there’s no good reason to use JSON I have been surprised by how silly that conclusion is, even though it's from 2018. JSON is so bad it is on the standard library of all major languages, natively supported by php, python, nodejs, javascript, ruby and many others, unlike YAML, TOML, FADFH or whatever solution someone comes up to a problem that is essentially solved. If you want perfection then by all means reinvent the wheel. But otherwise, don't be an edgy teen, just go with what the entire industry tells you that works.

linkdd 4y ago
In C++ you get the great https://github.com/nlohmann/json https://github.com/nlohmann/json In Rust you have the amazing serde with serde_json but at this point, you can use toml which is also based on serde. I consider serde as being standard. In C you have the lib jason which is very good. In Elixir, I use the compile-time configuration (config/config.exs) with environment variables for production, but that's because in the end, my Elixir system is in a Docker container, running on Kubernetes, with a ConfigMap defining the environment variables, so in the end, it's YAML (or JSON).
valbaca 4y ago
> unlike YAML, TOML, FADFH or whatever solution TOML and YAML are each supported by every language you listed. https://github.com/toml-lang/toml/wiki https://github.com/toml-lang/toml/wiki https://yaml.org/ https://yaml.org/
- hillcrestenigma 4y ago
  They aren't supported by the standard libraries of those languages though. It is still a major advantage of JSON.
  - lmm 4y ago
    Does that matter nowadays? Unless you're writing a small script in a language with terrible dependency management like Python, you're going to have a bunch of non-standard-library dependencies, one more doesn't make much difference.

disintegrator 4y ago

CUE has been an absolute joy to work with on configuration problems. It’s a typed language which is something that’s been a massive gap in the config language space when looking at yaml, toml, json, etc... https://cuelang.org/ https://cuelang.org/

betwixthewires 4y ago
Why would you need typing in a configuration file? I would think a configuration file would be specific to your program and any interpretation of data would be handled by your program.
- planckscnst 4y ago
  CUE's type system is also its validation system. It's extremely flexible. You can define custom types and your own constraints on those types. It allows for disjunctive constraints (e.g. this value must be an integer > 0 or the string value "none"). The types compose well together. I'd highly recommend reading through the docs - they're great.
- arccy 4y ago
  if your config is longer and semi repetitive, you can use it to template things out while still having a strong schema at every step
- disintegrator 4y ago
  My initial comment was very vague and I'm glad to see others have replied to fill in the gaps. If your service/system has a sufficiently large configuration space, like say, Kubernetes, then a typed configuration language can greatly improve the development experience by spotting errors early on. You get a very fast feedback loop that you set the wrong value for some config option before attempting to deploy the change. Different services will give you appropriate feedback about wrong config options but that is sometimes a few steps removed from your development environment e.g. You might find out after you push a PR and CI/CD fails or after you merge the PR even. The type system also has great second order benefits like allowing us to build an language server protocol implementation for CUE that has rich diagnostics, auto-complete, jump to definition, rename symbol features. Something that cannot be done _as well_ effectively in untyped languages. I'm still scratching the surface. CUE does more than add types to config and I would encourage you to dig into it if you have spare time. By virtue of providing a great type system, it also manages to reduce boilerplate as yet another second order benefit. Boilerplate reduction is something where tools like Jsonnet attack as a primary goal but over enough time you're back at having seas of untyped config and indirection that are hard to navigate or contribute to and you're back at square one.

hsbauauvhabzb 4y ago

I learned about json5 on news.ycombinator, which turned json config from an absolute pain to work with, to something more tolerable. Thank you kind stranger who recommended it. Edit: one thing I think all json python libs suck at is error messages. A syntax error in a json tree shouldn’t require me to rip nodes out to a/b test a parse error. Sure it can’t correctly break down AST but it could at least try to provide something more effective than Exception(f”{line} {col} glhf!”)

princevegeta89 4y ago

JSON is not a language it's just a format.

rfiat 4y ago
I disagree. It has a formal grammar so at least in a CS sense [0] it's a language. Why is JSON not a language but e.g. TOML or YAML are? [0] https://en.wikipedia.org/wiki/Formal_language https://en.wikipedia.org/wiki/Formal_language

betwixthewires 4y ago

Seems I'm in the minority here, but I agree with the author on his main point. JSON is fantastic for organizing and passing data around, storing it, even for configuration files where the user doesn't need to edit the raw configuration. If you are like me and write a lot of tools that run in the background, lack a UI and where the only thing a user will be doing is editing a configuration file and running it, you're asking for trouble with JSON. Curly brackets and nesting and lack of basic formatting are bad for UX, and while it's a fantastic compromise for machine and human readability, it focuses more on machine readability and the human readability is just enough for you to look at it. It is not easy for a human to parse with their eyes. I go with TOML for my configuration files most of the time. I am not a fan of YAML or XML. I don't think those offer any real benefit over JSON, and both have more downsides than JSON. TOML is a breeze to read and understand just looking at it, and is analogous to JSON as far as machine readability to the point that it's trivial to convert to it when you need to. I do use JSON a lot, even for configuration files, but only when I'm storing a configuration in a flat file that the user should never have to see.

kcartlidge 4y ago
I'd agree. JSON is user-readable, but not really user-friendly and user-editable. I'm okay with YAML but TBH, and I know this makes me totally uncool, for config that only nests to a single level I quite like INI files. I sometimes cheat slightly and take account of the ordering of entries in a section but generally speaking it's easy to read, easy to write, and very easy to parse.
dan_hawkins 4y ago
Also: no comments in JSON. This makes self-documented config files difficult to create and maintain.

molyss 4y ago

I’m totally happy using the properties format from java. There’s comments, it’s easy to grep. Main missing part are value types and multiline (or maybe it’s there, I’m not sure). The parameter reference substitution would be nice to have too.

angarg12 4y ago

We use tons of configuration at work and this has come up time and time again. The biggest pain is when inevitably you need to "parameterize" your JSON files. The most common route I've seen is to turn your configs into templates e.g. Jinja and go from there. Welcome to hell. My solution? Write a DSL with Typescript, and "compile" your configs down to JSON. Define the structure of your configs as TS types. Write functions with free variables to work as templates, or define snippets as composable fragments. Writing the end result to a JSON file results in transparency (what you see is what you get) and compatibility.

unrealhoang 4y ago
jsonnet[1] might suit your use-case better, it was created to do exactly that. [1]https://jsonnet.org https://jsonnet.org
- Matheus28 4y ago
  Not OP. Looks interesting, but I'd bet that typescript's type system would enforce constraints a lot better
  - whacked_new 4y ago
    in my experience, for things like static configs, jsonnet has been superior to TS for constraints, if you pair the configs with a json schema (also generated from jsonnet, of course). TS's types are easier to use at write-time, but json schema includes a lot of batteries that save a lot of time when you need them, but become annoying in TS, like patternProperties in dictionaries, length constraints in keys. Of course, there are situations like key1 XOR key2 where you will need custom logic one way or another, but given json schema's evolution over the years, I think it's pretty solid. An added benefit is that if you stay in json schemas, you're almost guaranteed a validator exists in $other_language, it _almost_ feels like a first-class construct everywhere.
    - Matheus28 4y ago

rektide 4y ago

{"this_article": "// would be better", "this_article": "// if it werent", "this_article": "// immediately", "this_article": "// right off the bat", "this_article": "// wrong", "this_article": "meh"} //=> {"this_article": "meh"} Hacker spirit represent. Where there's a will, there's a way. (Death to all oppressors!) I actually agree, JSON is kind of a pain in the ass to work with. It's shocking to think of a pre-'jq' world, given how saturated we all are with JSON, but we lived like that for a long time. (We also evidently didn't even have Firebug until 2006?[1] Holy shit!) But is it a problem? I dunno. Is it worth doing something else? Ooof, scary proposition. I think one of the missing questions we don't ask, don't know is: what tools help us fix our json fast. What tools can see, oh there's a comma missing between these two lines, and just fix it for us. [1] https://en.wikipedia.org/wiki/Firebug_(software) https://en.wikipedia.org/wiki/Firebug_(software)

tikhonj 4y ago
As far as I can tell, you're relying on non-standard behavior. This is what [the standard][1] has to say about keys being unique: > * The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs. These are all semantic considerations that may be defined by JSON processors or in specifications defining specific uses of JSON for data interchange.* This means that different tools can—and, if experience is any guide, will—handle your example inconsistently. This can be especially troubling when JSON processing tools are used internally by some other system like a database, queue or server; most of the time these tools parse and serialize JSON without changing its meaning, but they can change or lose information when people rely on patterns like the one that you used. I would honestly be surprised if there isn't at least one realistically used tool that interprets your example differently, like producing {"this_article": "// would be better"} instead. [1]: https://www.ecma-international.org/publications-and-standards/standards/ecma-404/ https://www.ecma-international.org/publications-and-standard...
- rektide 4y ago
  I'd love to see the counter-evidence! Works on every browser, every server-side JS engine of note. Edit: I believe a number of these non-determinism did get formally resolved in EcmaScript itself, which is yes, not the same, but is an active, alive, & notable specification with some bearing.

dub 4y ago

My wish for configuration languages is that as an industry we continue to adopt scriptable build systems like Bazel that make it easy to transform human-written configuration into machine-readable configuration during compile time. Want comments in JSON? Spend 20 minutes refactoring a single BUILD rule and now humans can write JSON5 that's transformed into JSON Want more flexibility? Have humans write cue or dhall or jsonnet Wish you also had a copy of a subset of the same config data in YAML? Easy Want to write a compile-time check in a programming language of your choice that a certain setting is never missing? Easy Want all of this to work with reproducible builds across a variety of computers and distributed build farms? Easy We don't have to let legacy build systems limit our imagination.

fb03 4y ago

I actually really like RON[1] [1] https://github.com/ron-rs/ron https://github.com/ron-rs/ron

throw0101a 4y ago

How about ISC-style like for BIND? * https://bind9.readthedocs.io/en/latest/reference.html#configuration-file-grammar https://bind9.readthedocs.io/en/latest/reference.html#config... * https://wiki.debian.org/Bind9#File_.2Fetc.2Fbind.2Fnamed.conf https://wiki.debian.org/Bind9#File_.2Fetc.2Fbind.2Fnamed.con... * https://www.zytrax.com/books/dns/ch7/#overview https://www.zytrax.com/books/dns/ch7/#overview I've been experimenting with NSD (and Unbound) recently, and their configuration is JSON-like. There seem to be some 'arbitrary' attributes that are treated as "top-level": > At the top level only server:, key:, pattern:, zone:, tls-auth:, and remote-control: are allowed. These are followed by their attributes or a new top-level keyword. The zone: attribute is followed by zone options. The server: attribute is followed by global options for the NSD server. […] * https://nsd.docs.nlnetlabs.nl/en/latest/manpages/nsd.conf.html#file-format https://nsd.docs.nlnetlabs.nl/en/latest/manpages/nsd.conf.ht... It seems to be good/best practice to indent the sub-attributes of the 'top-level' attributes to know where the 'stanzas' for each top-level start and end. The white space has no significance. While I like the functionality of NSD/Unbound, I lean toward liking the use of braces (curly brackets; {}) à la BIND to explicitly denote stanzas and statements, even if one also uses indentation for human consumption.

ryanthedev 4y ago

Idk. I feel like most things are YAML based or now some custom flavor (bicep, hcl, Nginx) But that’s more from the IaaC world. I guess .net application configuration might still be heavy on the JSON side. I just feel like this isn’t as much of a problem anymore. There are so many ways to utilize alternatives.

krapp 4y ago

(clears throat, taps microphone.) LuaJIT. Or at least Lua. Thank you for coming to my TED talk.

phendrenad2 4y ago

This is why I hate the Javascript ecosystem with a burning passion, and consider every single person who has contributed to it to be the very portrait of supremely incompetence. Everyone in web dev spent 2010-2020 schlepping these intolerable JSON configuration files around, unable to add comments to document what various lines did, simply because the JS ecosystem developers couldn't be bothered to implement a YAML parser. Le sigh.

shepherdjerred 4y ago
Yikes.

ukoki 4y ago

JSON is a great configuration language: * It's in the standard library for most languages so you can write tools that read and write it that have zero dependencies. This is great for portability and for use in offline environments. * You can use the same JSON files as input for tools that expect JSON (eg Terraform .tfvars.json) and tools that that expect YAML (anything kubernetes related) * You can easily see at a glance exactly what whitespace characters are in multiline strings like PEM certificate strings. This is useful to avoid LF vs CRLF line ending issues, and ensuring start and end whitespace is exactly what you want to make sure input strings are compatible with whatever is consuming the configuration. The downside is that it's a little bit ugly compared to YAML, but editor auto-formatting and key-sorting plugins make that a minor issue.

luismedel 4y ago
IMHO points 1 and 2 are more community efforts rather than JSON's design strengths. Regarding point 3, are you talking about using \n in single-line strings (the ones you can use in JSON) I think that the main reason this debate exists is we are using a good data-format (which JSON is) to configure things, which it's not a use case JSON is designed for.

aiddun 4y ago

I really like the solution Tailwind and some other JavaScript tools have taken to this where instead of a tailwind.json, there’s a tailwind.config.js, which is a plain ol JS file that exports a JS object. Allows for importing constants from other modules, scripting, comments, conditionals, etc

woojoo666 4y ago

My favorite so far is StrictYaml [1]. It's a subset of yaml that directly addresses a lot of the article's concerns: * supports comments * flexible with good signal to noise (no need for so many brackets and commas) * gets rid of a lot of YAML's complexity [2] * explicitly typed for less ambiguities [3] The main page also gives comparisons against HJSON, HOCON and TOML [4] [1]: https://hitchdev.com/strictyaml/ https://hitchdev.com/strictyaml/ [2]: https://hitchdev.com/strictyaml/features-removed/ https://hitchdev.com/strictyaml/features-removed/ [3]: https://hitchdev.com/strictyaml/why/implicit-typing-removed/ https://hitchdev.com/strictyaml/why/implicit-typing-removed/ [4]: https://hitchdev.com/strictyaml/#why-strictyaml https://hitchdev.com/strictyaml/#why-strictyaml

Spivak 4y ago
StrictYAML throws out too much for my taste — tags, flow state, and anchors are all useful non-footgun features.
- woojoo666 4y ago
  To me those are in the category of "nice to have", and the problem is that every developer has different preferences for these [1] [2]. But the main features of StrictYaml, like supporting comments and less syntactic noise, I think are pretty uncontroversial, and perhaps it's worth it to get people to switch over for those alone. It doesn't need to be perfect, it just needs to be a significant enough improvement over JSON, and I'd say those two features are more than enough [1]: https://github.com/crdoconnor/strictyaml/issues/37 https://github.com/crdoconnor/strictyaml/issues/37 [2]: https://github.com/crdoconnor/strictyaml/issues/38 https://github.com/crdoconnor/strictyaml/issues/38

benibela 4y ago

All the text formats are bad and verbose Everything is much more efficient with binary format like protobufs or flatbuffers. And if the format has a schema, the schema is the documentation, an d you do not need to have comment in the config file itself.

terrabitz 4y ago
As a transport I agree. But as a config file format? I'm a little skeptical. The main benefit of a flat file format for configs is that it can be added to source control and edited by any plaintext editor. If I kept config as protobuf, I would need special tools to handle it, which could be annoying (e.g. if I'm remoted into a remote system). It would also make it more difficult to see diffs between different config versions stored in git.