7 ms·

Configuration files are user interfaces

165 points by todsacerdoti 1y ago

Spivak 1y ago
Or you just use YAML. It's a configuration language for your software, you control which parser you use which can be YAML 1.2, you can use the safe loader which can't run untrusted code, and you're parsing the values into your language's types so any type confusion will be instantly caught. I agree that it's not perfect but worse is better and familiar is a massive win over making your users look up a new file format or set their editor up for it. If you truly hate YAML that's fine, there's plenty of other familiar formats: INI, toml, JSON.
- NeutralForest 1y ago
  I'm just so tired of writing bash scripts inside the YAML. I want to be able to lint and test whatever goes into my pipelines and actions. Not fight shitty DX on top of whatever Azure spits out when it fails.
  - ksenzee 1y ago
    This is entirely understandable, and entirely the fault of whoever thought bash scripts belong in configuration files. If you’re trying to stuff a tiger into a desk drawer, the natural consequences are hardly the fault of the desk drawer.
    - NeutralForest 1y ago
      But it's the situation we're in now. It's what you see in the docs and what my colleagues write as well. I entirely agree that scripting into a markup language doesn't make sense yet the inertia is there and I wish there was some way out.
  - hk1337 1y ago
    > I'm just so tired of writing bash scripts inside the YAML. Why would you be doing that?
    - NeutralForest 1y ago
      Can't tell if asked honestly. Because that's how most platforms handle their pipelines. Terraform or Bicep let you use a declarative language for your platform. Everything else is calling cli commands or scripts from pipelines, written in YAML.
      - hk1337 1y ago
        I suppose I am confused by what you mean with, "writing bash scripts". Like, are you _literally_ scripting in the value of an item or are you just equating that they are similar? Literal being: get_number_of_lines: command: > #!/bin/bash wc -l
        NeutralForest 1y ago
        Something like this: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/azure-cli-v2?view=azure-pipelines#examples https://learn.microsoft.com/en-us/azure/devops/pipelines/tas... Invariably, ppl will write inline scripts instead of actual scripts in their own files. There are also some SDKs for most of these operations that would let you do it in code but they are not always 100% the same as the CLI, some options are different, etc.
        Spivak 1y ago
        Yes, this is how Gitlab pipelines work. It's actually easier to just inline the script most of the time than have a bucket of misc scripts lying around. Especially since you have hooks like before/after_script which would be really awkward to externalize.
  - kiitos 1y ago
    if you're writing bash inside of yaml then something has gone wrong well before yaml entered the picture, this is a problem with e.g. azure not with the yaml format
- chromalchemy 1y ago
  YamlScript (YS) adds more programmatic expressiveness to YAML. It extends an executable Clojure runtime (non-jvm, like Babaskha). Created by YAML inventor/maintainer https://yamlscript.org/ https://yamlscript.org/
ruuda 1y ago
From the application point of view, recently I'm converging on this: define data structures for your config. Ensure it can be deserialized from json and toml. (In Rust this is easy to do with Serde; in Python with Pydantic or dataclasses.) Users can start simple and write toml by hand. If you prefer KSON, sure, write KSON and render to json. If config is UI, I think the structure of the data, and names of fields and values, matter much more than the syntax. (E.g. `timeout = 300` is meaningless regardless of syntax; `timeout_ms = 300` or `timeout = "300 ms"` are self-documenting.) When the configuration grows complex, and you feel the need to abstract and generate things, switch to a configuration language like Cue or RCL, and render to json. The application doesn't need to force a format onto the user!
- wofo 1y ago
  I like this take! Have you used Cue or RCL yourself? How was the experience?
  - diarrhea 1y ago
    Not the OP but I’m a big fan of this pattern as well. At work we generate both k8s manifests as well as application config in YAML from a Cue source. Cue allows both deduplication, being as DRY as one can hope to be, as well as validation (like validating a value is a URL, or greater than 1, whatever). The best part is that we have unit tests that deserialize the application config, so entire classes of problems just disappear. The generated files are committed in VCS, and spell out the entire state verbatim - no hopeless Helm junk full of mystery interpolation whose values are unknown until it’s too late. No. The entire thing becomes part of the PR workflow. A hook in CI validates that the generated files correspond to the Cue source (run make target, check if git repo has changes afterwards). The source of truth are native structs in Go. These native Go types can be imported into Cue and used there. That means config is always up to date with the source of truth. It also means refactoring becomes relatively easy. You rename the thing on the Go side and adjust the Cue side. Very hard to mess up and most of it is automated via tooling. The application takes almost its entire config from the file, and not from CLI arguments or env vars (shudder…). That means most things are covered by this scheme. One downside is that the Cue tooling is rough around the edges and error messages can be useless. Other than that, I fully intend to never build applications differently anymore.
bee_rider 1y ago
KSON looks neat. I think the post is hurt by the desire to sort of… “have a theory” or take a new stance. The configuration file is obviously not a user interface, it is data. It is data that is typically edited with a text editor. The text editor is the UI. The post doesn’t really justify the idea of calling the configuration file, rather than the program used to edit it, the UI. Instead it focuses on a better standard for the data. The advancement of standards that make the data easier to handle inside the text editor is great, though! Maybe the slightly confusing (I dare say confused) blog title will help spread the idea of kson, regardless. Edit: another idea, one that is so obvious that nobody would write a blog post about it, is that configuring your program is part of the UX.
- vrighter 1y ago
  if writing a "data" file is how you configure the app, then that data file is the interface between you and the software. Since you are a user, that makes it a user interface.
  - bee_rider 1y ago
    I’ve never written a file directly. The cells in the SSD are very small, my hand is not steady enough to flip the bits manually. For me the UI for writing files is usually Vim.
    - vrighter 1y ago
      the ui for writing the data file is vim. The ui for the target application is the data file you wrote.
skydhash 1y ago
There’s two configuration that I like: - The key-value pair. Maybe some section marker (INI,..). Easy to sed. - The command kind. Where the file contains the same command that can be used in other place (vim, mg, sway). More suited to TUI and bigger applications. With these two, include statement are nice.
mholt 1y ago
This is why Caddy has config adapters: bring any config file language you like, and Caddy will run it. It's built-into the binary and just takes a command line flag to switch languages: https://caddyserver.com/docs/config-adapters https://caddyserver.com/docs/config-adapters
- deleted 1y ago
  [deleted]
- kevmo314 1y ago
  This makes it difficult to configure Caddy in anything except the native Caddyfile language due to a lack of thorough documentation. It's an interesting idea, but configuring Caddy with a yaml config that someone prior deemed a great idea was quite painful. Curiously, LLMs have made it a lot easier. One step away from an English adapter that routes through an LLM to generate the config.
- kiitos 1y ago
  those formats aren't bijective with each other, right? so there's no way for you to say that foo.cue can be equivalently transformed to any foo.json or any foo.nginx or whatever representation, because those transformations are necessarily lossy, no?
theamk 1y ago
KSON proudly claims "no whitespace sensitivity", which means "misleading indentation" is back. And it's pretty light on syntax, so there are going to be plenty of mistakes made here. Here is an example I made in a few minutes: ports: - 80 - 8000 - 10000 - 12000 - - 14000 Guess how it parses? answer: {"ports":[80,8000,10000,12000,[14000]]}
- cryptonector 1y ago
  Yeah, I'm going to say NO WAY to that.
- SebastianKra 1y ago
  Assuming this takes off, there would be a prettier-plugin that corrects any weird formatting. When I think about it, any language should come with a strict, non-configurable built-in formatter anyways.
  - stronglikedan 1y ago
    > any language should come with a strict, non-configurable built-in formatter Would that be on the language, or the IDEs that support it? Seems out of scope to the language itself, but maybe I'm misunderstanding.
    - Kinrany 1y ago
      The languave server, which in this age of LSP should come from the same project as the language itself.
    - hiccuphippo 1y ago
      I'd say a program or subcommand tha comes with the compiler/interpreter. Go has gofmt, zig has zig fmt. The compiler already parses the language so a formatter is only a few steps from that. And being part of the compiler means it's always up to date, there's less bikeshedding, all IDEs and text editors get formatter support easily.
  - 1y ago
VectorLock 1y ago
I like where their head is at here, especially the "superset of JSON" part. Some of the things I'm not _in love_ with like the %% ending blocks or how maybe a bit of significant whitespace might make things a bit less misleading with indentation as others have said, but overall I think I like this better than YAML.
brap 1y ago
While I agree with the general idea, 2 minutes of looking at KSON was enough for me. If this is UI, it’s an ugly one. I’ll stick to JSON. When JSON isn’t enough it usually means the schema needs an update.
kccqzy 1y ago
Configuration files need to be powerful programming languages (in terms of expressiveness) while being restricted (in terms of network and I/O and non-determinism). We need to aim very high for configuration languages especially when we treat them like user interfaces. Look at Cue (https://cuelang.org/ https://cuelang.org/), Starlark or Dhall (https://dhall-lang.org/ https://dhall-lang.org/) for inspiration, not JSON, unless your configuration file is almost always written programmatically.
- candiddevmike 1y ago
  Or Jsonnet (https://jsonnet.org https://jsonnet.org), if you do like JSON but want less quoting.
- deleted 1y ago
  [deleted]
- madeofpalk 1y ago
  Any configuration language that doesn't support strict/user/explicit types is worthless (ahem jsonnet). The idea of configuring something but not actually having any sort of assurances that what you're configuring is correct is maddening. Building software with nothing but hopes and dreams.
- arccy 1y ago
  expressiveness unfortunately usually means that while you can read the output value, you lose the ability to modify it programmatically...
  - foota 1y ago
    I don't know that I would particularly _want_ to modify starlark programmatically, but it's certainly not impossible. Build files in bazel for instance are nicely modifiable: https://bazel.build/concepts/build-files https://bazel.build/concepts/build-files.
    - arccy 1y ago
gricardo99 1y ago
With all the code syntax highlighting support as a feature, I feel it will become tempting to put code in configuration files (which some of their examples show). That just feels wrong. Code should go in code files/modules/libraries, not mixed with configuration files. If your configuration starts to become code, maybe you need to rethink your software architecture. Or perhaps KSON proves that principle to be too rigid and inferior, and leads to more intelligible, manageable software. I guess we'll have to see.
- bdubbeldam 1y ago
  We agree that there are lots of situations where code ideally does not belong in configuration files, and perhaps if it gets out of hand you need to rethink your software architecture. However, reality shows that embedded code in configs is already widespread - think SQL queries in dbt, bash scripts in CI/CD configs, or regex patterns in routing rules. These aren't going away. KSON's embed blocks acknowledge this reality while making it safer and more maintainable through proper syntax highlighting, validation, and clear boundaries. We're not encouraging bad practices, we're providing better tooling for existing patterns.
dustingetz 1y ago
yes but to validate it you need dynamic runtime logic and therefore a live server with all the I/O glue code that entails. i.e., static types alone cannot render your tax forms
Xss3 1y ago
JSON5 is good enough that it works for frontend devs, backend, qa, firmware, data science, chemists, optical engineers, and the hardware team, in my org at least. Interns pick up on it quickly. The comment option gives enough space for devs to explain new options flags and objects included to those familiar enough to be using it. For customer facing configurations we build a UI.
- stronglikedan 1y ago
  In the kitchen sink example (https://json5.org/ https://json5.org/), they say: > "backwardsCompatible": "with JSON", But in that same example, they have a comment like this: > // comments Wouldn't that make it not compatible with JSON?
  - coldcache 1y ago
    It’s a superset of JSON. I guess they mean it’s backwards compatible in terms of reading existing JSONs?
  - rapfaria 1y ago
    Your existing JSON < 5 will work with json5, not the other way around
  - crazygringo 1y ago
    It's confusing. From what I understand, it's "backwards-compatible" with JSON because valid JSON is also valid JSON5. But it's not "forwards-compatible" precisely because of comments etc.
    - metadat 1y ago
      Stating `backwards compatible` at a feature is at best confusing for readers who haven't already bought into JSON5 as "the way". If people tended to interpret English correctly and not be susceptible to misinterpreting written statements, that would be nice. Reality is a bugger!
hk1337 1y ago
Anecdote: I am still of the opinion that in most (~99%)[1] of the situations people are in "YAML Hell" because they put themselves in "YAML Hell". 1: I pulled that out of my butt, there's no factual data to it.
i_s 1y ago
KSON looks interesting. Where I work we did a metadata type project in Pkl recently, which is somewhat similar. Unfortunately, developments on the tooling front for Pkl have taken an extremely very long time. Not sure the the tooling/LSPs are anywhere close to what the language offers yet. I like the language embedding feature in KSON - we would use that. Have you thought about having functions and variables? That is something you get in Pkl and Dhall which are useful.
- wofo 1y ago
  Thanks for the kind words :) This sounds like the kind of question for Daniel himself to chime in, since he has the best overview of the language's design and vision. He's not super active on HN, but I'll give him a heads up! Otherwise feel free to join our Zulip (https://kson-org.zulipchat.com https://kson-org.zulipchat.com) and we can chat over there.
- islon 1y ago
  What kind of tooling are you missing in Pkl?
  - i_s 1y ago
    If I type the name of an enum, then `.`, (and for my settings explicitly request autocomplete suggestions from VsCode), I either don't get anything, or I get a list of 50+ identifiers that have nothing to do with the enum. There seem to be a lot of cases like that where there is a very obvious context, but the suggestions are completely detached from that context. Stuff that works for any mainstream language like Java or TypeScript in just about every IDE or editor with a plugin.
- dmarcotte 1y ago
  Thanks, i_s. Daniel chiming in here... for functions and variables, we intentionally stay focused on JSON/YAML-equivalent data because the places where JSON or YAML are used as human-edited interfaces (because of their limitations rather than in spite of them) is the slice of the programming stack that KSON is particularly interested in. If the next Kubernetes/GitHub Actions/dbt/etc wants to reach for a language to be their interface, KSON is interested in providing everything YAML offers and more (AND less: less of the problems associated with YAML) We have a design for KSON constants that we're looking forward to sharing soon, but they will be constant and KSON will still be JSON/YAML-equivalent. For the places where someone wants more programming power in their config, I'm glad people like the folks behind Pkl and Dhall are serving that use case.
righthand 1y ago
Yes all user interfaces are a key/value list. That’s why 90% of each iOS update is just another menu or a reorganization of menus and why there are 3 different ways to access the same info/menus on iOS.
JohnMakin 1y ago
Having now worked with terraform for 8 years, I could not agree more. Now, also because of having worked with terraform for 8 years and seeing how that's played out, I've heard and become tired of the whole "superset of json, transcribable to YAML, whitespace is not significant (which has never been a gripe of mine ever, not sure why every product cares so much about that)" promise of a silver bullet, and you very much face the same exact problems, just in different form. Terraform (HCL, to be specific) in particular can become fantastically ugly and verbose and "difficult to modify." Configuration is difficult, the tooling is rarely the problem (at least in my experience).
giveita 1y ago
I think it is telling no programming language has settled on YAML or JSON as a syntax. Because that would drive you nuts. But we allow it for files that tend to make production changes usually without any unit tests! I'd prefer something syntaxed like a programming language but without turing completeness.
- knome 1y ago
  Azure and GitHub build pipelines are written in yaml and have conditionals, variables, template expansions, etc
  - ruuda 1y ago
    GitHub Actions also have function calls, it’s just that they can only occur in very specific places in the program, and to define a function you have to create a Git repository. And don’t forget Ansible playbooks!
    - vrighter 1y ago
      and they all suck horribly to use
- danans 1y ago
  > I think it is telling no programming language has settled on YAML or JSON as a syntax. It's not telling, it's impossible. Neither JSON or YAML are Turing complete. That said, the JS in JSON is for Javascript, whose syntax it is derived from, so at least one major language uses it in its syntax.
comex 1y ago
Sounds interesting as a format, but the implementation is a big supply-chain attack risk if you're not already in the JVM ecosystem. This is because the only implementation is written in Kotlin. There are Python and Rust packages, but they both just link against the Kotlin version. How do you build the Kotlin version? Well, let's look at the Rust package's build.rs: https://github.com/kson-org/kson/blob/main/lib-rust/kson-sys/build.rs https://github.com/kson-org/kson/blob/main/lib-rust/kson-sys... It defaults to simply downloading a precompiled library from GitHub, without any hash verification. You can instead pass an environment variable to build libkson from source. However, this will run the ./gradlew script in the repo root, which… downloads an giant OpenJDK binary from GitHub and executes it. Later in the build process it does the same for pixi and GraalVM. The build scripts also only support a small list of platforms (Windows/Linux/macOS on x86_64/arm64), and don't seem to handle cross-compilation. The compiled library is 2MB for me, which is actually a lot less than I was expecting, so props for that. But that's fairly heavy by Rust standards.
- wofo 1y ago
  Glad you liked the format. I hope we can close the implementation gaps as development advances, and I'd love to see native libraries sprout for all conceivable programming languages! Edit: point taken about verifying checksums, just created an issue for it (https://github.com/kson-org/kson/issues/222 https://github.com/kson-org/kson/issues/222)
- Terr_ 1y ago
  Past-me had hoped that by the Future Year 2025, it'd be typical to publish a parser grammar file for this kind of thing. Both to bootstrap making a parser in a new language, and also as a kind of living spec document.
  - wofo 1y ago
    I think the current grammar should be precise enough for that, though it's embedded in the source code as a comment and not in its own file (see https://github.com/kson-org/kson/blob/857d585ef26d9f73e080b5f269e7ed7652495fc3/src/commonMain/kotlin/org/kson/parser/Parser.kt#L8-L39 https://github.com/kson-org/kson/blob/857d585ef26d9f73e080b5...). It probably can't be fed verbatim into a parser generator, but anyone who reads the parser's source code should have an easy time writing a parser by hand for their programming language of choice (heck, they might even have an LLM translate the original parser into whatever language they want, once there is a comprehensive conformance test suite to validate the resulting code). All in all, I'm confident that KSON can become ubiquitous despite the limitations of the current implementation (provided it catches on, of course).
atoav 1y ago
My personal opinion is: If your config needs to be so complex you can't make do with TOML you should just use a interpreted programming language instead. It is totally acceptable to use a config.py for example. You get lists, dicts, classes, and all more or less known and well behaved and people can automate away the boring stuff. But I'd strongly encourage everybody to think about whether that deep configurability is really needed.
rmah 1y ago
Has anyone used both kson and hjson? We use hjson as a more forgiving json-like config format and really like it. kson seems more feature rich, but I'm a bit concerned about things like embedding code in configs. So any thoughts vs hjson?
- dmarcotte 1y ago
  hjson is new on my radar, but looks like it has some great philosophical overlap with KSON! hjson's "interface for JSON" tagline really resonates. If I understand correctly, hjson helps by not requiring commas, allowing unquoted strings, and supporting multi-line strings. Our support is similar, though we only use # for comments instead of // and support multiline strings both in embed blocks and regular quoted strings. Where KSON diverges from hjson is enabling a YAML-like presentation while improving on the YAML experience. And by making the content type of embed blocks a first-class citizen of the syntax, with clear boundaries for the embedded content, editors and IDEs can provide full tooling support (syntax highlighting, validation, etc.) for the embedded code. Importantly though, KSON remains 'plain data'---it's not a runtime that executes code in your configs.
bedatadriven 1y ago
Came across hocon recently and prefer over both yaml and kson. https://learnxinyminutes.com/hocon/ https://learnxinyminutes.com/hocon/
stevekrouse 1y ago
The people behind KSON are geniuses. No joke. I worked with Daniel. And he's an amazing human too. Congrats on launching!!!
- dmarcotte 1y ago
  Thanks so much, Steve!!!
epolanski 1y ago
Imho the best configuration file is code written in the language itself. Configuring TypeScript applications with the `defineConfig` pattern that takes asynchronous callbacks allowing you to code over settings is very useful. And it's fully typed and programmable. It's particularly useful because it also allows you to trivially use one single configuration file where you only set what's different between environments with some env === "local" then use this db dependency or turn on/off sentry, etc. Zig is another language that shows that configuration and building should just be code in the language itself.
- eternityforest 1y ago
  Programmatically manipulating it and validating it gets harder though.
  - themafia 1y ago
    Unless it's a lisp.
    - maxbond 1y ago
      Specifically what you want is to provably halt and to be free of side effects. There's some configuration languages with these properties for providing expressions in JSON configs (I'm blanking on their names for the moment) which generally don't advertise themselves as Lisp dialects but boil down to a Lisp AST written in JSON. Another example is Starlark, which provides Python-like syntax. Note that this implies a Turing incomplete language. Which makes sense - our goal is to make dangerous programs unrepresentable, so naturally we can't implement every algorithm in our restricted language.
  - epolanski 1y ago
    How so?
- Terr_ 1y ago
  > Imho the best configuration file is code written in the language itself. This depends on the trust-model for who is doing the configuration, especially if you're trying to host for multiple tenants/customers. Truly sandboxing arbitrary code is much harder than reading XML/JSON/etc.
taeric 1y ago
Meanwhile, I peek at my emacs config and continue to wonder why people don't just embrace a programming language. Yes, there are bad consequences that can happen. No, you don't dodge having problems by picking a different data format. You just pick different problems. And take away tools from the users to deal with them.
- wsc981 1y ago
  Just something like an INI file might be fine for most use-cases as well and is easier to reason about.
- slowmovintarget 1y ago
  Elisp for Emacs, Lua for those on Neovim... Definitely more control than guessing the right JSON, or breaking that YAML file. Plus, you get completion, introspection, and help while editing the config because you're in a code-writing environment. Bonus for having search and text manipulation tools under your fingertips instead of clicking checkboxes or tabbing through forms.
  - taeric 1y ago
    Reading this does make me think about how exploring settings in emacs `M-x config` settings is actually better than I would expect. So it isn't like you can't also have the checkboxes and forms.
matsemann 1y ago
Question: if whitespace isn't significant, how does it determine ambiguity? Like, if I have a Person that has a Dog, and both have a Name attribute. If I then add a Name after my dog definition, how does it know if it's the name of the dog or the person?
- wofo 1y ago
  That took some design work! In the end, KSON settled on something called the "end dot", which denotes the end of an object (more details in the docs, at https://kson.org/docs/syntax/#the-end-dot https://kson.org/docs/syntax/#the-end-dot).
kiitos 1y ago
two huge thumbs down for KSON
your_fin 1y ago
I came across https://kdl.dev https://kdl.dev recently, and it has become my favored yaml-replacement when normal code doesn't work. The data model is closer to XML than JSON, though, so unfortunately it's not a drop-in replacement for yaml. Small sample: package { name my-pkg version "1.2.3" dependencies { // Nodes can have standalone values as well as // key/value pairs. lodash "^3.2.1" optional=#true alias=underscore } }
- olejorgenb 1y ago
  I wish they had some sort of include/composable file mechanism though
skobes 1y ago
Why is TOML allegedly "too minimal" for a "small configuration file"?
- PeterWhittaker 1y ago
  I don't understand that either. Considering that it is both well defined and hierarchical, with none of the problems of YAML or JSON (hey, NO is a country, not a truth value), TOML seems like the perfect compromise to me.
- wofo 1y ago
  I've often reached out to TOML for truly minimal configuration files, so I'd say TOML makes sense in a lot of scenarios. However, my point in the article is that simple stuff eventually stops being simple, so you need to be careful. As an example, a friend of mine introduced TOML to a reasonably big open source project a while ago. Recently, he mentioned there were some unexpected downsides to TOML. I've asked him to chime in here, because I think he's more qualified to reply (instead of a best-effort attempt from my side to faithfully reproduce what he told me).
- droelf 1y ago
  I think for a small configuration TOML might be fine. Where it breaks down in my opinion is in larger configuration files. It just becomes pretty unreadable. Think about a Github Action being written in TOML ... would probably not look great!
maxbond 1y ago
Is the K in KSON for Kotlin? Does the S stand for anything? I skimmed the docs but didn't find anything addressing the name.
- bdubbeldam 1y ago
  K stands for KSON. Soft spot for recursive acronyms!
phkahler 1y ago
Programmers don't mind "programming", but most people don't want that UI.
cryptonector 1y ago
I kinda want jq as a config file language. It would need some sort of nice multi-line string support, but yeah.
0xbadcafebee 1y ago
I keep trying to explain the difference between a configuration format and a data format. A configuration format is for humans. A data format is for computers. Try to make one format good for both, you end up with one format that sucks at both. Another fun thing about configuration: it's a great indicator of poor software design. If the configuration can be very complicated, in one single format, in one big file, look at the source code and you'll find a big ball of mud. Or even worse, it's lots of very different software that all shares one big config file. Both are bad for humans and bad for computers.
somat 1y ago
Trying to jam your configuration into a generic object notation is always going to be awkward and ugly, the only real saving grace might be that it is a standard syntax. I use a lot of openbsd and one of the things I really like about it is that they care about the user interface(note 1) and take the time to build custom parsers(note 2) Compare pf to iptables. I think, strictly speaking, ip tables is the more feature-full filter language. but the devs said "we will just use getopt to parse it" and called it a day. leading to what I find one of the ugliest, more painful languages to use. pf is an absolute joy in comparison. I would pick pf for that reason alone. note 1. Not in the sense of chasing graphical configurators. An activity I find wastes vast amounts of developer time leading to neutered interfaces. But in the sense of taking the time to lay out a syntax that is pleasant to read and edit and then write good documentation for it. note 2. If I am honest, semi custom parsers, the standard operating procedure appears to start with a parse.y from a project that is close and mold it into the form needed. which lends itself to a nice sort of consistency in the base system.
thatxliner 1y ago
How can TOML be too minimal? Also, what about Apple Pickle?
- wofo 1y ago
  A friend of mine introduced TOML to a reasonably big open source project and he mentioned there were some unexpected downsides to it. I've asked him to chime in here, because I think he's more qualified to reply (note that I pointed him to a sibling comment that's also asking about TOML, here https://news.ycombinator.com/item?id=45295715 https://news.ycombinator.com/item?id=45295715). As for Apple Pkl, I think we share the goal of robustness and "toolability", but pkl seems way more complex. I personally think it's more useful to keep things simple, to avoid advancing in the configuration complexity clock (for context, see https://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html https://mikehadlow.blogspot.com/2012/05/configuration-comple...).
eviks 1y ago
Naming is too strict, so valid friendly yaml names don't work price_in_€: 42
- wofo 1y ago
  Just tested this on the playground and observed that the following does work: "price_in_€": 42 Or were you already aware of that and lamenting that KSON requires quoting in this case, compared to YAML which doesn't?
  - eviks 1y ago
    Was lamenting the bare IDs' limitation, quoting is bad here since there is no ambiguity to resolve, no type to signal. All these new improved formats should strive to limit the bad parts of existing languages, not eliminate the goods ones
mpweiher 1y ago
Configurations are (parts of) programs and configuration languages are also programming languages. And code is also user interface. So you probably don't really want a separate language for configuration, but rather a programming language that (a) has a reasonable way of creating complex object literals and (b) can be, but does not have to be, interpreted. And you want to be able to subset and/or sandbox the language so that the power you give to those parts of the program that does the configuring is under control. I saw this from one side when working on the BBC SportStats project, where I eliminated all the configuration files. All of them. The deployment was a single jar. That doesn't mean that there wasn't configuring going, but it was done with Java code inside the jar. One reason for doing it this way was that deployment was just as complex/arduous for a configuration file change as it was for a code change. And the code we could easily test, abstract etc. So there were few if any benefits. At a later stage we did have multiple instances of the app, all still using the same jar, but now delivered with different 1-2 line shell scripts that selected one of the built-in configurations.
qcnguy 1y ago
Apple has a similar project called Pkl which they use for their internal cloud configuration. http://pkl-lang.org/ http://pkl-lang.org/ It can be converted to JSON, yaml, and a bunch of other formats, and whilst it looks a bit like JSON it's actually a full blown functional programming language designed for creating config trees. The nice thing about Pkl is that it's got full IDE support and the standard library is patterned after Kotlin. So all the method names are very obvious and intuitive especially if you know Java or Kotlin. It has lots of features for validation like range types and so on, so you can get a lot of semantic checking before any config is emitted. It's the first config-as-language language I've seen that is actually good. KSON looks more like an alternate syntax than a language, which is fine, but because JSON/YAML are just data structures sometimes you really want to do a for loop.
- wpm 1y ago
  Pkl ended up being a bit much for what I was doing but I spent a few days digging into it and really liked what I saw.
M95D 1y ago
So why not give "the yaml document from hell" in KSON as an example so we can see for ourselves how good it is?
- bdubbeldam 1y ago
  That's a great idea, we've just added it to the playground. Here's the KSON, much of YAML's leanness and simplicity without the footguns: server_config: port_mapping: # Expose only ssh and http to the public internet. - '22:22' - '80:80' - '443:443' serve: - '/robots.txt' - '/favicon.ico' - '*.html' - '*.png' # Do not expose our Git repository to the entire world. - '!.git' geoblock_regions: # The legal team has not approved distribution in the Nordics yet. - dk - fi - is - no - se flush_cache: on: [push, memory_pressure] priority: background . allow_postgres_versions: - '9.5.25' - '9.6.24' - '10.23' - '12.13' Testing this actually helped us catch a bug - of course our transpiler hits the Norway problem when converting to YAML. Thanks for helping us with the beta! Fix coming soon.
  - M95D 1y ago
    Except for the dot after "priority:background", that's a valid YAML. I tried 3 online converters to JSON and all of them worked with no footguns. So maybe "the yaml document from hell" is the actual problem, not YAML?
kiitos 1y ago
fun! name: 'Leonardo Bonacci' favorite_books: - title: Metaphysics author: Aristotle. favorite_numbers: - 1 - 2 delete that `.` at the end of Aristotle and you get a totally structurally different result
hulitu 1y ago
> Now, which language should you pick for your glorious configuration file? SSE2 assembler. Fast and self explanatory.
jkmcf 1y ago
I think human editable config files become unsustainable after they reach a certain size, and while I like Ansible a lot, you probably shouldn't be embedding code in them. For well over a decade, I've felt we missed the boat with XML, XSD, and friends to make a generic, literal user interface for configuration such that you didn't need to rely on referencing documentation for valid options and such and usable interfaces could be presented instead of editing the text directly.
Fire-Dragon-DoL 1y ago
Best example to me remains git rebase -i which shows a text file to edit. Love it

165 points by todsacerdoti 1y ago

Or you just use YAML. It's a configuration language for your software, you control which parser you use which can be YAML 1.2, you can use the safe loader which can't run untrusted code, and you're parsing the values into your language's types so any type confusion will be instantly caught. I agree that it's not perfect but worse is better and familiar is a massive win over making your users look up a new file format or set their editor up for it. If you truly hate YAML that's fine, there's plenty of other familiar formats: INI, toml, JSON.

NeutralForest 1y ago
I'm just so tired of writing bash scripts inside the YAML. I want to be able to lint and test whatever goes into my pipelines and actions. Not fight shitty DX on top of whatever Azure spits out when it fails.
- ksenzee 1y ago
  This is entirely understandable, and entirely the fault of whoever thought bash scripts belong in configuration files. If you’re trying to stuff a tiger into a desk drawer, the natural consequences are hardly the fault of the desk drawer.
  - NeutralForest 1y ago
    But it's the situation we're in now. It's what you see in the docs and what my colleagues write as well. I entirely agree that scripting into a markup language doesn't make sense yet the inertia is there and I wish there was some way out.
- hk1337 1y ago
  > I'm just so tired of writing bash scripts inside the YAML. Why would you be doing that?
  - NeutralForest 1y ago
    Can't tell if asked honestly. Because that's how most platforms handle their pipelines. Terraform or Bicep let you use a declarative language for your platform. Everything else is calling cli commands or scripts from pipelines, written in YAML.
    - hk1337 1y ago
      I suppose I am confused by what you mean with, "writing bash scripts". Like, are you _literally_ scripting in the value of an item or are you just equating that they are similar? Literal being: get_number_of_lines: command: > #!/bin/bash wc -l
      - NeutralForest 1y ago
        Something like this: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/azure-cli-v2?view=azure-pipelines#examples https://learn.microsoft.com/en-us/azure/devops/pipelines/tas... Invariably, ppl will write inline scripts instead of actual scripts in their own files. There are also some SDKs for most of these operations that would let you do it in code but they are not always 100% the same as the CLI, some options are different, etc.
      - Spivak 1y ago
        Yes, this is how Gitlab pipelines work. It's actually easier to just inline the script most of the time than have a bucket of misc scripts lying around. Especially since you have hooks like before/after_script which would be really awkward to externalize.
- kiitos 1y ago
  if you're writing bash inside of yaml then something has gone wrong well before yaml entered the picture, this is a problem with e.g. azure not with the yaml format
chromalchemy 1y ago
YamlScript (YS) adds more programmatic expressiveness to YAML. It extends an executable Clojure runtime (non-jvm, like Babaskha). Created by YAML inventor/maintainer https://yamlscript.org/ https://yamlscript.org/

ruuda 1y ago

From the application point of view, recently I'm converging on this: define data structures for your config. Ensure it can be deserialized from json and toml. (In Rust this is easy to do with Serde; in Python with Pydantic or dataclasses.) Users can start simple and write toml by hand. If you prefer KSON, sure, write KSON and render to json. If config is UI, I think the structure of the data, and names of fields and values, matter much more than the syntax. (E.g. `timeout = 300` is meaningless regardless of syntax; `timeout_ms = 300` or `timeout = "300 ms"` are self-documenting.) When the configuration grows complex, and you feel the need to abstract and generate things, switch to a configuration language like Cue or RCL, and render to json. The application doesn't need to force a format onto the user!

wofo 1y ago
I like this take! Have you used Cue or RCL yourself? How was the experience?
- diarrhea 1y ago
  Not the OP but I’m a big fan of this pattern as well. At work we generate both k8s manifests as well as application config in YAML from a Cue source. Cue allows both deduplication, being as DRY as one can hope to be, as well as validation (like validating a value is a URL, or greater than 1, whatever). The best part is that we have unit tests that deserialize the application config, so entire classes of problems just disappear. The generated files are committed in VCS, and spell out the entire state verbatim - no hopeless Helm junk full of mystery interpolation whose values are unknown until it’s too late. No. The entire thing becomes part of the PR workflow. A hook in CI validates that the generated files correspond to the Cue source (run make target, check if git repo has changes afterwards). The source of truth are native structs in Go. These native Go types can be imported into Cue and used there. That means config is always up to date with the source of truth. It also means refactoring becomes relatively easy. You rename the thing on the Go side and adjust the Cue side. Very hard to mess up and most of it is automated via tooling. The application takes almost its entire config from the file, and not from CLI arguments or env vars (shudder…). That means most things are covered by this scheme. One downside is that the Cue tooling is rough around the edges and error messages can be useless. Other than that, I fully intend to never build applications differently anymore.

bee_rider 1y ago

KSON looks neat. I think the post is hurt by the desire to sort of… “have a theory” or take a new stance. The configuration file is obviously not a user interface, it is data. It is data that is typically edited with a text editor. The text editor is the UI. The post doesn’t really justify the idea of calling the configuration file, rather than the program used to edit it, the UI. Instead it focuses on a better standard for the data. The advancement of standards that make the data easier to handle inside the text editor is great, though! Maybe the slightly confusing (I dare say confused) blog title will help spread the idea of kson, regardless. Edit: another idea, one that is so obvious that nobody would write a blog post about it, is that configuring your program is part of the UX.

vrighter 1y ago
if writing a "data" file is how you configure the app, then that data file is the interface between you and the software. Since you are a user, that makes it a user interface.
- bee_rider 1y ago
  I’ve never written a file directly. The cells in the SSD are very small, my hand is not steady enough to flip the bits manually. For me the UI for writing files is usually Vim.
  - vrighter 1y ago
    the ui for writing the data file is vim. The ui for the target application is the data file you wrote.

skydhash 1y ago

There’s two configuration that I like: - The key-value pair. Maybe some section marker (INI,..). Easy to sed. - The command kind. Where the file contains the same command that can be used in other place (vim, mg, sway). More suited to TUI and bigger applications. With these two, include statement are nice.

mholt 1y ago

This is why Caddy has config adapters: bring any config file language you like, and Caddy will run it. It's built-into the binary and just takes a command line flag to switch languages: https://caddyserver.com/docs/config-adapters https://caddyserver.com/docs/config-adapters

deleted 1y ago
[deleted]
kevmo314 1y ago
This makes it difficult to configure Caddy in anything except the native Caddyfile language due to a lack of thorough documentation. It's an interesting idea, but configuring Caddy with a yaml config that someone prior deemed a great idea was quite painful. Curiously, LLMs have made it a lot easier. One step away from an English adapter that routes through an LLM to generate the config.
kiitos 1y ago
those formats aren't bijective with each other, right? so there's no way for you to say that foo.cue can be equivalently transformed to any foo.json or any foo.nginx or whatever representation, because those transformations are necessarily lossy, no?

theamk 1y ago

KSON proudly claims "no whitespace sensitivity", which means "misleading indentation" is back. And it's pretty light on syntax, so there are going to be plenty of mistakes made here. Here is an example I made in a few minutes: ports: - 80 - 8000 - 10000 - 12000 - - 14000 Guess how it parses? answer: {"ports":[80,8000,10000,12000,[14000]]}

cryptonector 1y ago
Yeah, I'm going to say NO WAY to that.
SebastianKra 1y ago
Assuming this takes off, there would be a prettier-plugin that corrects any weird formatting. When I think about it, any language should come with a strict, non-configurable built-in formatter anyways.
- stronglikedan 1y ago
  > any language should come with a strict, non-configurable built-in formatter Would that be on the language, or the IDEs that support it? Seems out of scope to the language itself, but maybe I'm misunderstanding.
  - Kinrany 1y ago
    The languave server, which in this age of LSP should come from the same project as the language itself.
  - hiccuphippo 1y ago
    I'd say a program or subcommand tha comes with the compiler/interpreter. Go has gofmt, zig has zig fmt. The compiler already parses the language so a formatter is only a few steps from that. And being part of the compiler means it's always up to date, there's less bikeshedding, all IDEs and text editors get formatter support easily.
- 1y ago

VectorLock 1y ago

I like where their head is at here, especially the "superset of JSON" part. Some of the things I'm not _in love_ with like the %% ending blocks or how maybe a bit of significant whitespace might make things a bit less misleading with indentation as others have said, but overall I think I like this better than YAML.

brap 1y ago

While I agree with the general idea, 2 minutes of looking at KSON was enough for me. If this is UI, it’s an ugly one. I’ll stick to JSON. When JSON isn’t enough it usually means the schema needs an update.

kccqzy 1y ago

Configuration files need to be powerful programming languages (in terms of expressiveness) while being restricted (in terms of network and I/O and non-determinism). We need to aim very high for configuration languages especially when we treat them like user interfaces. Look at Cue (https://cuelang.org/ https://cuelang.org/), Starlark or Dhall (https://dhall-lang.org/ https://dhall-lang.org/) for inspiration, not JSON, unless your configuration file is almost always written programmatically.

candiddevmike 1y ago
Or Jsonnet (https://jsonnet.org https://jsonnet.org), if you do like JSON but want less quoting.
deleted 1y ago
[deleted]
madeofpalk 1y ago
Any configuration language that doesn't support strict/user/explicit types is worthless (ahem jsonnet). The idea of configuring something but not actually having any sort of assurances that what you're configuring is correct is maddening. Building software with nothing but hopes and dreams.
arccy 1y ago
expressiveness unfortunately usually means that while you can read the output value, you lose the ability to modify it programmatically...
- foota 1y ago
  I don't know that I would particularly _want_ to modify starlark programmatically, but it's certainly not impossible. Build files in bazel for instance are nicely modifiable: https://bazel.build/concepts/build-files https://bazel.build/concepts/build-files.
  - arccy 1y ago

gricardo99 1y ago

With all the code syntax highlighting support as a feature, I feel it will become tempting to put code in configuration files (which some of their examples show). That just feels wrong. Code should go in code files/modules/libraries, not mixed with configuration files. If your configuration starts to become code, maybe you need to rethink your software architecture. Or perhaps KSON proves that principle to be too rigid and inferior, and leads to more intelligible, manageable software. I guess we'll have to see.

bdubbeldam 1y ago
We agree that there are lots of situations where code ideally does not belong in configuration files, and perhaps if it gets out of hand you need to rethink your software architecture. However, reality shows that embedded code in configs is already widespread - think SQL queries in dbt, bash scripts in CI/CD configs, or regex patterns in routing rules. These aren't going away. KSON's embed blocks acknowledge this reality while making it safer and more maintainable through proper syntax highlighting, validation, and clear boundaries. We're not encouraging bad practices, we're providing better tooling for existing patterns.

dustingetz 1y ago

yes but to validate it you need dynamic runtime logic and therefore a live server with all the I/O glue code that entails. i.e., static types alone cannot render your tax forms

Xss3 1y ago

JSON5 is good enough that it works for frontend devs, backend, qa, firmware, data science, chemists, optical engineers, and the hardware team, in my org at least. Interns pick up on it quickly. The comment option gives enough space for devs to explain new options flags and objects included to those familiar enough to be using it. For customer facing configurations we build a UI.

stronglikedan 1y ago
In the kitchen sink example (https://json5.org/ https://json5.org/), they say: > "backwardsCompatible": "with JSON", But in that same example, they have a comment like this: > // comments Wouldn't that make it not compatible with JSON?
- coldcache 1y ago
  It’s a superset of JSON. I guess they mean it’s backwards compatible in terms of reading existing JSONs?
- rapfaria 1y ago
  Your existing JSON < 5 will work with json5, not the other way around
- crazygringo 1y ago
  It's confusing. From what I understand, it's "backwards-compatible" with JSON because valid JSON is also valid JSON5. But it's not "forwards-compatible" precisely because of comments etc.
  - metadat 1y ago
    Stating `backwards compatible` at a feature is at best confusing for readers who haven't already bought into JSON5 as "the way". If people tended to interpret English correctly and not be susceptible to misinterpreting written statements, that would be nice. Reality is a bugger!

hk1337 1y ago

Anecdote: I am still of the opinion that in most (~99%)[1] of the situations people are in "YAML Hell" because they put themselves in "YAML Hell". 1: I pulled that out of my butt, there's no factual data to it.

i_s 1y ago

KSON looks interesting. Where I work we did a metadata type project in Pkl recently, which is somewhat similar. Unfortunately, developments on the tooling front for Pkl have taken an extremely very long time. Not sure the the tooling/LSPs are anywhere close to what the language offers yet. I like the language embedding feature in KSON - we would use that. Have you thought about having functions and variables? That is something you get in Pkl and Dhall which are useful.

wofo 1y ago
Thanks for the kind words :) This sounds like the kind of question for Daniel himself to chime in, since he has the best overview of the language's design and vision. He's not super active on HN, but I'll give him a heads up! Otherwise feel free to join our Zulip (https://kson-org.zulipchat.com https://kson-org.zulipchat.com) and we can chat over there.
islon 1y ago
What kind of tooling are you missing in Pkl?
- i_s 1y ago
  If I type the name of an enum, then `.`, (and for my settings explicitly request autocomplete suggestions from VsCode), I either don't get anything, or I get a list of 50+ identifiers that have nothing to do with the enum. There seem to be a lot of cases like that where there is a very obvious context, but the suggestions are completely detached from that context. Stuff that works for any mainstream language like Java or TypeScript in just about every IDE or editor with a plugin.
dmarcotte 1y ago
Thanks, i_s. Daniel chiming in here... for functions and variables, we intentionally stay focused on JSON/YAML-equivalent data because the places where JSON or YAML are used as human-edited interfaces (because of their limitations rather than in spite of them) is the slice of the programming stack that KSON is particularly interested in. If the next Kubernetes/GitHub Actions/dbt/etc wants to reach for a language to be their interface, KSON is interested in providing everything YAML offers and more (AND less: less of the problems associated with YAML) We have a design for KSON constants that we're looking forward to sharing soon, but they will be constant and KSON will still be JSON/YAML-equivalent. For the places where someone wants more programming power in their config, I'm glad people like the folks behind Pkl and Dhall are serving that use case.

righthand 1y ago

Yes all user interfaces are a key/value list. That’s why 90% of each iOS update is just another menu or a reorganization of menus and why there are 3 different ways to access the same info/menus on iOS.

JohnMakin 1y ago

Having now worked with terraform for 8 years, I could not agree more. Now, also because of having worked with terraform for 8 years and seeing how that's played out, I've heard and become tired of the whole "superset of json, transcribable to YAML, whitespace is not significant (which has never been a gripe of mine ever, not sure why every product cares so much about that)" promise of a silver bullet, and you very much face the same exact problems, just in different form. Terraform (HCL, to be specific) in particular can become fantastically ugly and verbose and "difficult to modify." Configuration is difficult, the tooling is rarely the problem (at least in my experience).

giveita 1y ago

I think it is telling no programming language has settled on YAML or JSON as a syntax. Because that would drive you nuts. But we allow it for files that tend to make production changes usually without any unit tests! I'd prefer something syntaxed like a programming language but without turing completeness.

knome 1y ago
Azure and GitHub build pipelines are written in yaml and have conditionals, variables, template expansions, etc
- ruuda 1y ago
  GitHub Actions also have function calls, it’s just that they can only occur in very specific places in the program, and to define a function you have to create a Git repository. And don’t forget Ansible playbooks!
  - vrighter 1y ago
    and they all suck horribly to use
danans 1y ago
> I think it is telling no programming language has settled on YAML or JSON as a syntax. It's not telling, it's impossible. Neither JSON or YAML are Turing complete. That said, the JS in JSON is for Javascript, whose syntax it is derived from, so at least one major language uses it in its syntax.

comex 1y ago

Sounds interesting as a format, but the implementation is a big supply-chain attack risk if you're not already in the JVM ecosystem. This is because the only implementation is written in Kotlin. There are Python and Rust packages, but they both just link against the Kotlin version. How do you build the Kotlin version? Well, let's look at the Rust package's build.rs: https://github.com/kson-org/kson/blob/main/lib-rust/kson-sys/build.rs https://github.com/kson-org/kson/blob/main/lib-rust/kson-sys... It defaults to simply downloading a precompiled library from GitHub, without any hash verification. You can instead pass an environment variable to build libkson from source. However, this will run the ./gradlew script in the repo root, which… downloads an giant OpenJDK binary from GitHub and executes it. Later in the build process it does the same for pixi and GraalVM. The build scripts also only support a small list of platforms (Windows/Linux/macOS on x86_64/arm64), and don't seem to handle cross-compilation. The compiled library is 2MB for me, which is actually a lot less than I was expecting, so props for that. But that's fairly heavy by Rust standards.

wofo 1y ago
Glad you liked the format. I hope we can close the implementation gaps as development advances, and I'd love to see native libraries sprout for all conceivable programming languages! Edit: point taken about verifying checksums, just created an issue for it (https://github.com/kson-org/kson/issues/222 https://github.com/kson-org/kson/issues/222)
Terr_ 1y ago
Past-me had hoped that by the Future Year 2025, it'd be typical to publish a parser grammar file for this kind of thing. Both to bootstrap making a parser in a new language, and also as a kind of living spec document.
- wofo 1y ago
  I think the current grammar should be precise enough for that, though it's embedded in the source code as a comment and not in its own file (see https://github.com/kson-org/kson/blob/857d585ef26d9f73e080b5f269e7ed7652495fc3/src/commonMain/kotlin/org/kson/parser/Parser.kt#L8-L39 https://github.com/kson-org/kson/blob/857d585ef26d9f73e080b5...). It probably can't be fed verbatim into a parser generator, but anyone who reads the parser's source code should have an easy time writing a parser by hand for their programming language of choice (heck, they might even have an LLM translate the original parser into whatever language they want, once there is a comprehensive conformance test suite to validate the resulting code). All in all, I'm confident that KSON can become ubiquitous despite the limitations of the current implementation (provided it catches on, of course).

atoav 1y ago

My personal opinion is: If your config needs to be so complex you can't make do with TOML you should just use a interpreted programming language instead. It is totally acceptable to use a config.py for example. You get lists, dicts, classes, and all more or less known and well behaved and people can automate away the boring stuff. But I'd strongly encourage everybody to think about whether that deep configurability is really needed.

rmah 1y ago

Has anyone used both kson and hjson? We use hjson as a more forgiving json-like config format and really like it. kson seems more feature rich, but I'm a bit concerned about things like embedding code in configs. So any thoughts vs hjson?

dmarcotte 1y ago
hjson is new on my radar, but looks like it has some great philosophical overlap with KSON! hjson's "interface for JSON" tagline really resonates. If I understand correctly, hjson helps by not requiring commas, allowing unquoted strings, and supporting multi-line strings. Our support is similar, though we only use # for comments instead of // and support multiline strings both in embed blocks and regular quoted strings. Where KSON diverges from hjson is enabling a YAML-like presentation while improving on the YAML experience. And by making the content type of embed blocks a first-class citizen of the syntax, with clear boundaries for the embedded content, editors and IDEs can provide full tooling support (syntax highlighting, validation, etc.) for the embedded code. Importantly though, KSON remains 'plain data'---it's not a runtime that executes code in your configs.

bedatadriven 1y ago

Came across hocon recently and prefer over both yaml and kson. https://learnxinyminutes.com/hocon/ https://learnxinyminutes.com/hocon/

stevekrouse 1y ago

The people behind KSON are geniuses. No joke. I worked with Daniel. And he's an amazing human too. Congrats on launching!!!

dmarcotte 1y ago
Thanks so much, Steve!!!

epolanski 1y ago

Imho the best configuration file is code written in the language itself. Configuring TypeScript applications with the `defineConfig` pattern that takes asynchronous callbacks allowing you to code over settings is very useful. And it's fully typed and programmable. It's particularly useful because it also allows you to trivially use one single configuration file where you only set what's different between environments with some env === "local" then use this db dependency or turn on/off sentry, etc. Zig is another language that shows that configuration and building should just be code in the language itself.

eternityforest 1y ago
Programmatically manipulating it and validating it gets harder though.
- themafia 1y ago
  Unless it's a lisp.
  - maxbond 1y ago
    Specifically what you want is to provably halt and to be free of side effects. There's some configuration languages with these properties for providing expressions in JSON configs (I'm blanking on their names for the moment) which generally don't advertise themselves as Lisp dialects but boil down to a Lisp AST written in JSON. Another example is Starlark, which provides Python-like syntax. Note that this implies a Turing incomplete language. Which makes sense - our goal is to make dangerous programs unrepresentable, so naturally we can't implement every algorithm in our restricted language.
- epolanski 1y ago
  How so?
Terr_ 1y ago
> Imho the best configuration file is code written in the language itself. This depends on the trust-model for who is doing the configuration, especially if you're trying to host for multiple tenants/customers. Truly sandboxing arbitrary code is much harder than reading XML/JSON/etc.

taeric 1y ago

Meanwhile, I peek at my emacs config and continue to wonder why people don't just embrace a programming language. Yes, there are bad consequences that can happen. No, you don't dodge having problems by picking a different data format. You just pick different problems. And take away tools from the users to deal with them.

wsc981 1y ago
Just something like an INI file might be fine for most use-cases as well and is easier to reason about.
slowmovintarget 1y ago
Elisp for Emacs, Lua for those on Neovim... Definitely more control than guessing the right JSON, or breaking that YAML file. Plus, you get completion, introspection, and help while editing the config because you're in a code-writing environment. Bonus for having search and text manipulation tools under your fingertips instead of clicking checkboxes or tabbing through forms.
- taeric 1y ago
  Reading this does make me think about how exploring settings in emacs `M-x config` settings is actually better than I would expect. So it isn't like you can't also have the checkboxes and forms.

matsemann 1y ago

Question: if whitespace isn't significant, how does it determine ambiguity? Like, if I have a Person that has a Dog, and both have a Name attribute. If I then add a Name after my dog definition, how does it know if it's the name of the dog or the person?

wofo 1y ago
That took some design work! In the end, KSON settled on something called the "end dot", which denotes the end of an object (more details in the docs, at https://kson.org/docs/syntax/#the-end-dot https://kson.org/docs/syntax/#the-end-dot).

kiitos 1y ago

two huge thumbs down for KSON

your_fin 1y ago

I came across https://kdl.dev https://kdl.dev recently, and it has become my favored yaml-replacement when normal code doesn't work. The data model is closer to XML than JSON, though, so unfortunately it's not a drop-in replacement for yaml. Small sample: package { name my-pkg version "1.2.3" dependencies { // Nodes can have standalone values as well as // key/value pairs. lodash "^3.2.1" optional=#true alias=underscore } }

olejorgenb 1y ago
I wish they had some sort of include/composable file mechanism though

skobes 1y ago

Why is TOML allegedly "too minimal" for a "small configuration file"?

PeterWhittaker 1y ago
I don't understand that either. Considering that it is both well defined and hierarchical, with none of the problems of YAML or JSON (hey, NO is a country, not a truth value), TOML seems like the perfect compromise to me.
wofo 1y ago
I've often reached out to TOML for truly minimal configuration files, so I'd say TOML makes sense in a lot of scenarios. However, my point in the article is that simple stuff eventually stops being simple, so you need to be careful. As an example, a friend of mine introduced TOML to a reasonably big open source project a while ago. Recently, he mentioned there were some unexpected downsides to TOML. I've asked him to chime in here, because I think he's more qualified to reply (instead of a best-effort attempt from my side to faithfully reproduce what he told me).
droelf 1y ago
I think for a small configuration TOML might be fine. Where it breaks down in my opinion is in larger configuration files. It just becomes pretty unreadable. Think about a Github Action being written in TOML ... would probably not look great!

maxbond 1y ago

Is the K in KSON for Kotlin? Does the S stand for anything? I skimmed the docs but didn't find anything addressing the name.

bdubbeldam 1y ago
K stands for KSON. Soft spot for recursive acronyms!

phkahler 1y ago

Programmers don't mind "programming", but most people don't want that UI.

cryptonector 1y ago

I kinda want jq as a config file language. It would need some sort of nice multi-line string support, but yeah.

0xbadcafebee 1y ago

I keep trying to explain the difference between a configuration format and a data format. A configuration format is for humans. A data format is for computers. Try to make one format good for both, you end up with one format that sucks at both. Another fun thing about configuration: it's a great indicator of poor software design. If the configuration can be very complicated, in one single format, in one big file, look at the source code and you'll find a big ball of mud. Or even worse, it's lots of very different software that all shares one big config file. Both are bad for humans and bad for computers.

somat 1y ago

Trying to jam your configuration into a generic object notation is always going to be awkward and ugly, the only real saving grace might be that it is a standard syntax. I use a lot of openbsd and one of the things I really like about it is that they care about the user interface(note 1) and take the time to build custom parsers(note 2) Compare pf to iptables. I think, strictly speaking, ip tables is the more feature-full filter language. but the devs said "we will just use getopt to parse it" and called it a day. leading to what I find one of the ugliest, more painful languages to use. pf is an absolute joy in comparison. I would pick pf for that reason alone. note 1. Not in the sense of chasing graphical configurators. An activity I find wastes vast amounts of developer time leading to neutered interfaces. But in the sense of taking the time to lay out a syntax that is pleasant to read and edit and then write good documentation for it. note 2. If I am honest, semi custom parsers, the standard operating procedure appears to start with a parse.y from a project that is close and mold it into the form needed. which lends itself to a nice sort of consistency in the base system.

thatxliner 1y ago

How can TOML be too minimal? Also, what about Apple Pickle?

wofo 1y ago
A friend of mine introduced TOML to a reasonably big open source project and he mentioned there were some unexpected downsides to it. I've asked him to chime in here, because I think he's more qualified to reply (note that I pointed him to a sibling comment that's also asking about TOML, here https://news.ycombinator.com/item?id=45295715 https://news.ycombinator.com/item?id=45295715). As for Apple Pkl, I think we share the goal of robustness and "toolability", but pkl seems way more complex. I personally think it's more useful to keep things simple, to avoid advancing in the configuration complexity clock (for context, see https://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html https://mikehadlow.blogspot.com/2012/05/configuration-comple...).

eviks 1y ago

Naming is too strict, so valid friendly yaml names don't work price_in_€: 42

wofo 1y ago
Just tested this on the playground and observed that the following does work: "price_in_€": 42 Or were you already aware of that and lamenting that KSON requires quoting in this case, compared to YAML which doesn't?
- eviks 1y ago
  Was lamenting the bare IDs' limitation, quoting is bad here since there is no ambiguity to resolve, no type to signal. All these new improved formats should strive to limit the bad parts of existing languages, not eliminate the goods ones

mpweiher 1y ago

Configurations are (parts of) programs and configuration languages are also programming languages. And code is also user interface. So you probably don't really want a separate language for configuration, but rather a programming language that (a) has a reasonable way of creating complex object literals and (b) can be, but does not have to be, interpreted. And you want to be able to subset and/or sandbox the language so that the power you give to those parts of the program that does the configuring is under control. I saw this from one side when working on the BBC SportStats project, where I eliminated all the configuration files. All of them. The deployment was a single jar. That doesn't mean that there wasn't configuring going, but it was done with Java code inside the jar. One reason for doing it this way was that deployment was just as complex/arduous for a configuration file change as it was for a code change. And the code we could easily test, abstract etc. So there were few if any benefits. At a later stage we did have multiple instances of the app, all still using the same jar, but now delivered with different 1-2 line shell scripts that selected one of the built-in configurations.

qcnguy 1y ago

Apple has a similar project called Pkl which they use for their internal cloud configuration. http://pkl-lang.org/ http://pkl-lang.org/ It can be converted to JSON, yaml, and a bunch of other formats, and whilst it looks a bit like JSON it's actually a full blown functional programming language designed for creating config trees. The nice thing about Pkl is that it's got full IDE support and the standard library is patterned after Kotlin. So all the method names are very obvious and intuitive especially if you know Java or Kotlin. It has lots of features for validation like range types and so on, so you can get a lot of semantic checking before any config is emitted. It's the first config-as-language language I've seen that is actually good. KSON looks more like an alternate syntax than a language, which is fine, but because JSON/YAML are just data structures sometimes you really want to do a for loop.

wpm 1y ago
Pkl ended up being a bit much for what I was doing but I spent a few days digging into it and really liked what I saw.

M95D 1y ago

So why not give "the yaml document from hell" in KSON as an example so we can see for ourselves how good it is?

bdubbeldam 1y ago
That's a great idea, we've just added it to the playground. Here's the KSON, much of YAML's leanness and simplicity without the footguns: server_config: port_mapping: # Expose only ssh and http to the public internet. - '22:22' - '80:80' - '443:443' serve: - '/robots.txt' - '/favicon.ico' - '*.html' - '*.png' # Do not expose our Git repository to the entire world. - '!.git' geoblock_regions: # The legal team has not approved distribution in the Nordics yet. - dk - fi - is - no - se flush_cache: on: [push, memory_pressure] priority: background . allow_postgres_versions: - '9.5.25' - '9.6.24' - '10.23' - '12.13' Testing this actually helped us catch a bug - of course our transpiler hits the Norway problem when converting to YAML. Thanks for helping us with the beta! Fix coming soon.
- M95D 1y ago
  Except for the dot after "priority:background", that's a valid YAML. I tried 3 online converters to JSON and all of them worked with no footguns. So maybe "the yaml document from hell" is the actual problem, not YAML?

kiitos 1y ago

fun! name: 'Leonardo Bonacci' favorite_books: - title: Metaphysics author: Aristotle. favorite_numbers: - 1 - 2 delete that `.` at the end of Aristotle and you get a totally structurally different result

hulitu 1y ago

> Now, which language should you pick for your glorious configuration file? SSE2 assembler. Fast and self explanatory.

jkmcf 1y ago

I think human editable config files become unsustainable after they reach a certain size, and while I like Ansible a lot, you probably shouldn't be embedding code in them. For well over a decade, I've felt we missed the boat with XML, XSD, and friends to make a generic, literal user interface for configuration such that you didn't need to rely on referencing documentation for valid options and such and usable interfaces could be presented instead of editing the text directly.

Fire-Dragon-DoL 1y ago

Best example to me remains git rebase -i which shows a text file to edit. Love it