16 ms·

TOML: Tom's Obvious Minimal Language

165 points by jonbaer 3y ago

the_mitsuhiko 3y ago
The one thing I wish TOML had added was null. Today many TOML configs still have “nullable” keys which means you need to comment them out to unset them.
- Guthur 3y ago
  Well that would be null right? an absence of the thing.
  - TylerE 3y ago
    Null is one way to represent such a value, but it’s hardly the only. Option-style algebraic types are another, and more or less strictly superior if working in a language with first class support for such, because they force you to explicitly handle the null/None/Empty/whatever case if you’re doing something with it that could fail (basically anything besides assignment or passing it along).
    - Guthur 3y ago
      Seems beyond the simplicity TOML was aiming for. Absence is absence in the file, in your programming language you can wrap in whatever trickier you like or is relevant.
      - TylerE 3y ago
        The trick is distinguishing empty string vs no value vs key not being present.
        Guthur 3y ago
        Key not being present is no value, key present has value, empty string or otherwise, seems simple enough. You can add advanced semantics on load, the file does nothing by itself after all.
        TylerE 3y ago
        It’s that “or” that bites you if you aren’t real sure of the semantics in play.
        Guthur 3y ago
        There is no 'or' for a given file, it is declaratively there or it is not. The 'or' semantics are part of your language environment not the toml file.
- lexicality 3y ago
  The Billion Dollar Mistake lives on. Personally I'm happy it doesn't have explicit null and somewhat surprised it supports NaN (which I thought it didn't) To my mind, a configuration file using "this key doesn't exist" as one of several valid options seems deeply unsound.
  - the_mitsuhiko 3y ago
    > The Billion Dollar Mistake lives on. I disagree. There is a big difference between configuration language formats and programming languages. > To my mind, a configuration file using "this key doesn't exist" as one of several valid options seems deeply unsound. I would agree, but in that case you now often need a secondary key to "turn something off" which is often not done.
    - lexicality 3y ago
      > There is a big difference between configuration language formats and programming languages. You're right! But in my opinion the difference is that a configuration file should have a default state that could be written to disk and loaded with no change in behaviour. eg you can set `feature = "on"` or `feature = "off"` and if you don't set either it's the same as writing `feature = "off"`. Having a secret third thing of `feature = null` should be outlawed and I'm glad TOML doesn't encourage it
      - the_mitsuhiko 3y ago
        There is a secret third thing called "feature not in file" and most TOML implementation encourage it as "key not in file" is typically returned as `null` or `None`. That's how so many TOML files in practice end up with all these secret values. It's also particularly odd in lists where I have seen `["foo", "bar", {}]` show up in the real world with `{}` being used as a replacement value for null.
        lexicality 3y ago
        In my experience (Python and Rust) missing keys are errors, not silently converted to None: In [1]: example = {"foo": None} In [2]: print(example["foo"]) None In [3]: print(example["bar"]) --------------------------------------------------------------------------- KeyError Traceback (most recent call last) Cell In[3], line 1 ----> 1 print(example["bar"]) KeyError: 'bar' In [4]: print(example.get("bar", "sensible default")) sensible default If configuration values were arbitrarily nullable I would either need to deal with the possibility of nulls ending up anywhere in my program (back to The Billion Dollar Mistake) or have to have checks on parsing to say which fields are nullable or not. I acknowledge people have done silly things regarding missing keys already - but I think they should be discouraged from continuing to do so, not enabled.
- arp242 3y ago
  It's been discussed a few times over the years: https://github.com/toml-lang/toml/issues/921 https://github.com/toml-lang/toml/issues/921 https://github.com/toml-lang/toml/issues/803 https://github.com/toml-lang/toml/issues/803 https://github.com/toml-lang/toml/issues/802 https://github.com/toml-lang/toml/issues/802 https://github.com/toml-lang/toml/issues/146 https://github.com/toml-lang/toml/issues/146 https://github.com/toml-lang/toml/issues/30 https://github.com/toml-lang/toml/issues/30 Obviously there is a bit of a demand, and it certainly has valid use cases, but also has downsides.
  - the_mitsuhiko 3y ago
    I'm aware of the discussions. I also gave up trying to get NULL into TOML and I think I'm reference in one of those issues. Personally I do not create TOML configs where absence of key has a meaning, but I'm also exposed to other people's TOMLs where this is an actual issue with all the consequences this causes. And that ignores the issue of nulls in arrays which are sometimes needed. For instance I have been exposed to parameterized SQL queries in TOML where you need to use the empty object to represent NULL, a particularly absurd incarnation of this.
    - arp242 3y ago
      I'm not saying there aren't reasonable and valid use cases, there obviously are. But there are also downsides, so the question is how it all balances out. Almost every feature that has ever been added to anything has reasonable and valid use cases, but clearly not everything should have all the features.
gscho 3y ago
In case anyone is interested I made an online toml syntax checker in the same vane as jsonlint.com and yamllint.com: https://toml-lint.com https://toml-lint.com
bruce511 3y ago
I'm puzzling over where I would use this sort of thing, over say Json, or simple xml, and I realise that it's mostly useful in cases where you're expected to edit the configuration in a text editor. That's not something I come across a lot - all my software (that I use) has a visual interface, and so raw editing of text/config files is not something we do. So context really comes into play here. I can see how this would be very useful for some programs, and near useless for others. And more specifically useful for those who consume user-edited configuration files.
- zaebal 3y ago
  It also reminds me the syntax of INI files.
- lexicality 3y ago
  Curious at to what technologies you work with? The work that I do and tools that I use have all manner of configuration file syntaxes and languages and a big headache for doing sysadmin work is trying to remember if a particular .rc file uses INI, YAML, JSON, TOML or some home rolled nonsense.
- Sujeto 3y ago
  I moved some config files from json to yaml. Maintaining the documentation became a lot easier, because before I had a document describing every config option, which I had to maintain separately, instead now I have comments in the config file itself.
  - bonzini 3y ago
    Note that now you have "the Norway problem" instead. https://hitchdev.com/strictyaml/why/implicit-typing-removed/ https://hitchdev.com/strictyaml/why/implicit-typing-removed/ There's a whole spectrum of config formats including JSON, JSON5, strict YAML, YAML. It's quite messy.
    - GuB-42 3y ago
      I found yaml to be surprisingly complex and tricky to write for how simple it looks. It is made to be beautiful and easy for a human to read, but the trade-off are a lot of implicit rules you need to be aware of.
SPBS 3y ago
JSON is basically perfect if it allowed trailing commas and comments. TOML is not a replacement for JSON because of how badly it chokes on nested lists of objects (being both hard to read and hard to write), due to a misguided attempt to avoid becoming JSON-like[1]. [1] https://github.com/toml-lang/toml/issues/516 https://github.com/toml-lang/toml/issues/516
- eviks 3y ago
  still won't be prefect, all those extra quotes are bad for a heavily used human readable configs format
- arp242 3y ago
  That's already fixed; in the upcoming TOML 1.1 you can write: tbl = { hello = "world", } All the examples in the issue you linked should work. https://github.com/toml-lang/toml/pull/904 https://github.com/toml-lang/toml/pull/904
  - bdudbdjsh 3y ago
    published data standards having version is so wrong. now if you see "config in yaml" you know nothing, zero, nada, about the format because all versions are so different and everyone implemented the version at the time and didn't bother to mention version. not to mention you can use a dozen syntaxes for yaml/toml and each application may not understand them all. all this is so silly. we will stick with json and informal-ini forever one way or another.
    - eviks 3y ago
      Since it's impossible to design anything great in tech on first attempt, what is the road to improvement without versions? (and mentioning version could be a requirement in a future version)
    - arp242 3y ago
      It's not ideal, I agree, but it solves real problems for people, so there's that. Many commonly-used standards today weren't created by a bunch of wise men in a room thinking how to bestow their wisdom upon the rest of us, they often originated in real-world applications, were refined over a period of time based pn real-world experience, and then became a standard. TOML is the same. I hope it will become an RFC some day. We just need to fix a few outstanding issues first. Most commonly used TOML parsers support 1.0; adding 1.1 support should be pretty easy as the changes aren't that large (I did it in the parser I maintain, and it's 10 lines of code or so that had to be changed, most of them quite trivial).
conaclos 3y ago
I pretty like the idea of a superset of JSON that supports (1) comments, (2) trailing commas, (3) unquoted properties, (4) optional {} for the root object, (5) multi-line strings, (6) number separator. JSON6 proposal [1] supports all of this, except (4). Unfortunately it also supports more and make the spec a bit too complex for my taste (JSON has a compact spec; any extension should honor this). Same issue with JSON5 proposal [2]. EKON [3] is certainly the best candidate. However, it didn't get any traction. [1] https://github.com/d3x0r/JSON6 https://github.com/d3x0r/JSON6 [2] https://json5.org/ https://json5.org/ [3] https://github.com/ekon-org/ekon https://github.com/ekon-org/ekon
- 4ad 3y ago
  Among types and other useful properties, CUE supports everything you asked for. We're even considering adding a "data-only" mode to CUE which would be exactly what you asked for. https://cuelang.org https://cuelang.org
  - conaclos 3y ago
    Is there any concise representation of the language spec? Similar to the spec of JSON [1]. [1] https://www.json.org/json-en.html https://www.json.org/json-en.html
    - 4ad 3y ago
      No, we do have a spec[1], but it cannot be as concise as the JSON link posted because the JSON link is mostly only about syntax, and our spec discusses both syntax and semantics. And of course, we are a much richer language than JSON, with many more features and computational behavior. That being said, it would be nice to use railroad diagrams to describe syntax in our spec. [1] https://cuelang.org/docs/references/spec https://cuelang.org/docs/references/spec
- codeflo 3y ago
  JSON as a computer-generated serialization format doesn't need any of those features. But for human-written configuration files, at the very least trailing commas and comments are basically a necessity. The question then is which features to pick beyond that (in my humble opinion, none), and how such a new standard would gain traction. JSON5, JSON6, JSONC all basically attempt to solve the latter problem by branding, but so far, mostly have failed. It doesn't help that there's an insane amount of bike-shedding going on in this space. The people who have forked JSON5 to create JSON6 for example have created an endless amount of confusion over which standard to use, harming both of them. I think they might want to think really hard whether the inclusion of octal literals was really worth the hostile fork.
deleted 3y ago
[deleted]
dist-epoch 3y ago
Funny thing: TOML author/inventor calls it a mistake: > TOML is a bad file format. It looks good at first glance, and for really really trivial things it is probably good. But once I started using it and the configuration schema became more complex, I found the syntax ugly and hard to read. > I personally abandoned TOML and as such, I'm not planning on adding any new features > If you're still reading, I'd like to put forth CSON as a format of choice. https://github.com/avakar/pytoml/issues/15#issuecomment-217739462 https://github.com/avakar/pytoml/issues/15#issuecomment-2177...
- mhd 3y ago
  > Funny thing: TOML author/inventor calls it a mistake You mean the author of this specific python implementation, or did Tom Preston-Werner ever disavow it?
- detaro 3y ago
  Someone called Martin is the "author/inventor" of "Tom's Obvious Minimal Language"?
  - kps 3y ago
    It might be “Tom OR Martin's Language”.
- lionkor 3y ago
  Thats not the author/inventor of TOML.
- esafak 3y ago
  Tom (aka mojombo) is the founder of GitHub. https://github.com/mojombo https://github.com/mojombo
pachico 3y ago
My main issue with TOML is precisely described in the homepage example: [servers] [servers.alpha] is error prone. An sub heading shouldn't need to know about its parent, much less to have to copy it entirely.
- pphysch 3y ago
  But then you need new syntax to indicate subheadings, which is also (probably less) error prone because you might omit it.
- petre 3y ago
  How do you convert it into a nested object then? You could use deeply nested objects like JSON or YAML does, but then it defeats the purpose of an easy to read configuration format. This is also a feature of Java property files.
  - mdaniel 3y ago
    > This is also a feature of Java property files. To the very best of my knowledge, Properties are merely Map<String,String> (see: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Properties.html#getProperty%28java.lang.String%29 https://docs.oracle.com/en/java/javase/11/docs/api/java.base... ) and any subsequent interpretation happens in the application (e.g. Spring treating `foo[1]=bar` as a list-ish assignment but the actual key returned from `getProperty` is the string literal as written `foo[1]`)
- ryan-duve 3y ago
  I think it is not needed? Or did you mean something other than this? Python 3.11.3 (main, May 4 2023, 05:53:32) [GCC 10.2.1 20210110] Type 'copyright', 'credits' or 'license' for more information IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help. In [1]: !cat /tmp/x.toml [servers.alpha] x = 10 In [2]: import tomllib ...: ...: with open("/tmp/x.toml", "rb") as f: ...: t = tomllib.load(f) ...: ...: t Out[2]: {'servers': {'alpha': {'x': 10}}}
mhd 3y ago
Not to be confused with the Tom Programming Language[0], a kind of early 2000s proto-Swift. [0]: http://www.gerbil.org/tom/ http://www.gerbil.org/tom/
camgunz 3y ago
Rather than a JSON replacement, I like TOML as a YAML replacement. It's a lot simpler, I'm not confused by things not having a preceding dash, indentation isn't significant, etc.
- IshKebab 3y ago
  I disagree. TOML is an INI replacement. They both suck at nested structures but are good if you just have a load of top level sections containing simple key-values. YAML on the other hand is just as good at any nesting depth. I mean it's pretty awful at all depths, but the act of nesting doesn't make it any more awful.
  - matsemann 3y ago
    I spent a whole day debugging an issue because an extra - dash in yaml. I had something like rules: - name: Only allow X - condition: ip=123 The - in front of condition of course makes a new list element. So I now had 2 rules. One with no conditions and a name, and an unnamed one with conditions. Since they are or'ed together everything for some reason I couldn't figure out went through, as I thought I had a single rule.. I hate yaml.
    - IshKebab 3y ago
      Yeah I've had this happen to me too. Truly awful format. I think the only reason it has caught on is because it has multiline strings, which are usually not great in other formats. Of course YAML fucks that up too by having a gazillion different kinds of multiline strings, distinguished only by impossible to remember symbols. But it does at least have them.
  - camgunz 3y ago
    Yeah I guess we're saying the same thing: TOML as an INI replacement is more of a YAML as an INI replacement. Re: nesting, I think TOML's dotting is pretty elegant as it flattens the nesting you so often need in configs. It took me a while to get used to the double brackets (and it is pretty confusing that you can put them anywhere, not just after their parent section--what are the edge cases that benefit from this...) but now I'm bought in, to the point where I don't know if I've just gotten used to the madness or what. I like that those two things essentially flatten the config--if you were using the new multi-line-inline-tables (from the forthcoming 1.1) you could actually nest things but now I don't know how much I even want that.
alpaca128 3y ago
TOML is nice as a kind of extended .ini format but I don't like the approach to avoiding nested elements. I prefer KDL. It's like XML but with almost zero syntax. Probably not always the best idea because it's so niche but it's just really simple while having all the basics like comments and multiline strings.
- grncdr 3y ago
  KDL (assuming you mean https://kdl.dev/ https://kdl.dev/) looks great, thanks for mentioning it.
  - p0nce 3y ago
    See also: https://sdlang.org/ https://sdlang.org/
omoikane 3y ago
Previously (2020-09-10): https://news.ycombinator.com/item?id=24436550 https://news.ycombinator.com/item?id=24436550 There was also an earlier entry where it was "Tom's Own Markup Language" (2013-02-24): https://news.ycombinator.com/item?id=5272634 https://news.ycombinator.com/item?id=5272634
nicoburns 3y ago
If one is looking for an alternative config format, the best designed I've seen is https://github.com/gura-conf/gura https://github.com/gura-conf/gura
- eviks 3y ago
  Thanks for the tip, pity they mandate 4 spaces, that's 2 too many
- rascul 3y ago
  Gura doesn't look too bad to me at a glance. Not sure if I'll attempt to use it in the future but I'll bookmark it for now.
overgard 3y ago
Another alternative is HOCON: https://github.com/lightbend/config/blob/master/HOCON.md https://github.com/lightbend/config/blob/master/HOCON.md I'd describe it as JSON-like with much more flexible syntax and the ability to reference itself.
deleted 3y ago
[deleted]
vander_elst 3y ago
I wonder why text protobuf is not widely used for config. There is a formal description of the structure and the configuration structure can be statically checked.
- skybrian 3y ago
  Maybe because protobufs are specified in a language that’s different from everything else and needs its own parser and codegen? Also, there’s often more of an impedance mismatch with the basic datatypes of scripting languages. For example, people using protobufs will often specify int64 by default, when you don’t actually need 64 bits but need more than 32. JSON support is often in a language’s standard library, or close, and its ambiguity around what numbers you can actually use can be seen as a worse-is-better approach. (There are similar tradeoffs with not specifying a max length of a string. It’s awkward, nit-picky, and error-prone to pick a maximum, but there are practical reasons you don’t want to allow unlimited lengths in a UI.)
IshKebab 3y ago
Naming it "Obvious" doesn't actually make it true unfortunately. JSON is still by far the most obvious format, and JSON5 fixes all the big issues with using it as a configuration format. I wish it had wider support. It's just so obviously better than TOML and YAML.
friend_and_foe 3y ago
I use TOML and JSON for everything. I like them both quite a bit. TOML gets a lot of criticism for not being JSON and JSON gets a lot of criticism for... not being entirely fit for it's stated purpose. However, they're both fantastic if you use them for what they're good at: TOML for human readable configurations that you expect a user to modify, JSON for representing data that a machine will read all of the time but that you want the user to be able to look at and grasp. I don't think either of them really need to grow to be good at both, they're fine as they are, just pick the right tool for the job.
- Simran-B 3y ago
  I don't understand the appeal of TOML. Why not use YAML instead? Seems a lot more "obvious" to read and write to me. And it's the best I know that is strong in both, human and machine readability.
  - rascul 3y ago
    YAML is more complex and difficult to write than TOML.
  - OtomotO 3y ago
    Yaml is incredibly complex. KISS
  - MrJohz 3y ago
    YAML has a lot of extra stuff going on that can cause accidents if you don't take care. The classic example is the "Norway problem" where "no" (the country code for Norway) is parsed as "false" instead. If "no" is used as a key, this can cause the Norwegian data to disappear or to throw strange errors on load. The other big issue is that, by default, it allows relatively unrestricted code execution in many environments. If a user is ever able to submit data that is interpreted by a YAML parser, they can run arbitrary code, and potentially even commands. The fixes for these are both fairly well known (always quote strings and keys, use safe_load or an equivalent API), but it's very easy to make a mistake. TOML, by contrast, is much simpler - strings are strings are strings, they must always be quoted but other than that behave pretty much as expected. Similarly, there is no mechanism by which a TOML file can instruct the parser to execute arbitrary code as part of deserialisation, which makes it (a) a lot simpler, and (b) a lot safer.
wkdneidbwf 3y ago
i’ve been eagerly awaiting multi-line support for inline tables for what feels like forever
kmfrk 3y ago
In terms of growing scope, isn't the general case usually something along the lines of: TOML > YAML > JSON > SQLite I'm lazy, so I wish I could just manage them all in Sublime Text.
- mcdonje 3y ago
  No. Different ideal use cases, though they can overlap. TOML is great for simple config files that don't need anything beyond key/value or arrays. YAML is great for config files with more complicated data structure requirements. JSON is great for self-describing serialized interchange. SQLite is in a whole other class of technology. I suppose you can store config information in a table if you're already using SQLite in your app. If you're exposing it the way you would with a normal config file, the non-text format introduces friction.
viksit 3y ago
we need another version of this for LLM Prompts that is highly space efficient. has anyone come across that?
mdswanson 3y ago
Looks like Microsoft's .INI file format.
Waterluvian 3y ago
Using JSON for configuration was and continues to be a terrible blunder. TOML fills that role wonderfully. And it isn’t full of complexity that turns Yaml into a mess.
- eviks 3y ago
  Bad nesting alone would make it a similar blunder for configuration
- shaunpersad 3y ago
  I've had the complete opposite experience. Neither YAML or TOML are very obvious to me. I always have to double and triple check that my syntax does what I assume it will do. I never have that problem with JSON.
  - Waterluvian 3y ago
    Not adding comments to a config file is recipe for complete disaster. You can kinda fake it but it’s just a silly headache. Which is recognized given there’s now “JSONC” but now you’ve got even more confusion determining what systems can actually accept comments or not. That’s always fun. I’m curious what parts of TOML do people find confusing? It’s such a tiny markup language.
enriquto 3y ago
Graybeard opinion: all of toml, yaml, xml, json, kdl, etc., are ridiculously over-engineered for simple configuration files. You can nearly always fulfill all of your needs with a simple two-column text file of key=value pairs that can be parsed by a trivial call to fscanf(3) or whatever your laguage supports (yes, even correctly discarding comments). The world would be a better place if people just gron'd their jsons, xml2'd their xmls and so on, thus pruning the universal dependency tree by a considerable amount.
- rfoo 3y ago
  --flagfile gets me really far but to be fair it sucks when you need nested kv
  - enriquto 3y ago
    just use longer keys (with an ad-hoc separator, if you want)
    - rfoo 3y ago
      What about lists? Something like: --allowed-downtime-0-start=20230523170000Z --allowed-downtime-0-end=20230523170500Z --allowed-downtime-1-start=... Or --allowed-downtime-start='20230523170000Z,...' I can live with them, but neither looks clean :( Genuine question, as I actually like the plain old flagfiles/Java-properties-ish style.
      - enriquto 3y ago
        The output produced by gron looks quite clean to me. You just write your file as if it was a list of js variable assignments. allowed-downtime[0][0] = 20230523170000Z allowed-downtime[0][1] = 20230523170500Z allowed-downtime[1][0] = ... or if you absolutely want the names, you can make a "list of dictionaries": allowed-downtime[0].start = ...
- Rapzid 3y ago
  How do you handle data structures in your key/value world? Parse it out of some adhoc key string schema?
crabbone 3y ago
It's another junk format in the sea of junk formats. It sucks for describing data, its structure is too primitive, has wrong primitives, doesn't allow extension and has no mitigations for that. Essentially, it's used because it's simple to parse and there are existing parsers. But the problems it creates are very complicated. So, here's one example of a popular use-case: pyproject.toml. I mean, PyPA aren't the brightest minds around, so, they've created this joke. The stated goal of pyproject.toml was unification of dependency / requirements specification. But, what they ended up doing is this: [tool.joke] tool.joke.requires=<requirements> [tool.clown_fiesta] tool.clown_fiesta.requires=<requirements> The problem here is that the goal was to unify, to specify requirements only once... but in practice, you still have to write them twice, and, of course, it's possible that "joke" and "clown_fiesta" don't use the same format to specify requirements. So, you have no way to specify requirements for both at once, and no way to process similar structures, to present them differently to different tools. And so, PyPA has created an piece of junk software that today has to be used by hundreds of thousands. Because we live in the world of worse is better.
- eviks 3y ago
  Could you list a couple of great formats? The various issues are indeed widespread across all the popular ones, and there are too many formats to find the greats (wish there were a comparison site with all the features so you could easily exclude those that fail at a specific task)
- jborean93 3y ago
  Can you share an actual example where you need to specify requirements twice. With pyproject.toml I know of 3 ways requirements are specified and they are all used by different things: 1. Build requirements - requirements needed to build your package from an sdist 2. Runtime requirements - requirements your library needs at runtime 3. Extra requirements - optional runtime requirements For example my library pyspnego https://github.com/jborean93/pyspnego/blob/main/pyproject.toml https://github.com/jborean93/pyspnego/blob/main/pyproject.to... has all 3: 1. Cython for Win32 needed to build the sdist and setuptools as the general build system - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L1-L6 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 2. cryptography as a runtime dependency - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L28-L30 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 3. optional extras kerberos and yaml - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L33-L38 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... Granted the toml format and what is used in pyproject.toml has its warts but I'm curious what your joke and clown_fiesta examples are actually from as from where I am standing each section currently serve different purposes.
throwawaaarrgh 3y ago
TOML is, indeed, a human friendly config format. Whereas INI is a simple config format. YAML/JSON/CSV/XML/etc are not config formats at all. They are data serialization formats. If you don't know the difference, you probably didn't get a CS degree, or read or understand the specs. Config formats should be tailored to the application, because the entire point of a config file is to make it easy for a human to configure one specific application. That's why there are 100 different config file formats for old UNIX programs. For each one, it's vastly easier to use that one config format for that one application, than it would be to force some arbitrary data serialization format to represent what the human really wanted the program to do. TOML is better than YAML or INI for a config file, but what's best is rolling your own format. It literally leads to better outcomes. The user can express what they want faster, easier, clearer, and the application doesn't have to struggle with alternate implementations of the spec for some poorly defined data serialization format.
- theknocker 3y ago
  [dead]
- majewsky 3y ago
  Where do you draw the line between config file format and data serialization format? The first paragraph of the TOML spec says: > TOML is designed to map unambiguously to a hash table. Sounds like a data serialization format to me. Besides the obvious differences in design and stylistic choices, it only really differs from JSON or YAML in that the top-level object of the data structure must be an object. If this type of constraint is what makes it a config file format, I am going to change the world with my revolutionary new config file format "JSON, but the first byte must be {". As another evidence for the blurriness of this line: Cargo uses TOML for its config file (Cargo.toml) which matches the usage that you describe. But then it also uses TOML for the serialization of its internal locking data structure in Cargo.lock, which is a file that humans are specifically not told to edit directly because it just contains serialized data. It just so happens to use TOML, probably because they already had a parser for that available at the time. > TOML is better than YAML or INI for a config file, but what's best is rolling your own format. It literally leads to better outcomes. Disadvantages of rolling your own format: - You now have to write your own parser, which costs at least a bit of time and is going to be more prone to bugs than a widely-used deserialization library like Serde. I see that you dunk on data serialization formats being "poorly defined", but if the popular formats are poorly defined, then certainly the average tool developer is not going to do a better job when they actually want to implement the tool and not the parsing for a config file format. - Users have to learn the format, and have to mentally switch between formats for each application. I just hate how every homegrown config format uses slightly different syntax, esp. which character starts a comment. With `config.yaml` or `config.toml`, I don't have to guess. - If the config file is YAML or TOML, I immediately have some basic syntax highlighting in my editor.

165 points by jonbaer 3y ago

The one thing I wish TOML had added was null. Today many TOML configs still have “nullable” keys which means you need to comment them out to unset them.

Guthur 3y ago
Well that would be null right? an absence of the thing.
- TylerE 3y ago
  Null is one way to represent such a value, but it’s hardly the only. Option-style algebraic types are another, and more or less strictly superior if working in a language with first class support for such, because they force you to explicitly handle the null/None/Empty/whatever case if you’re doing something with it that could fail (basically anything besides assignment or passing it along).
  - Guthur 3y ago
    Seems beyond the simplicity TOML was aiming for. Absence is absence in the file, in your programming language you can wrap in whatever trickier you like or is relevant.
    - TylerE 3y ago
      The trick is distinguishing empty string vs no value vs key not being present.
      - Guthur 3y ago
        Key not being present is no value, key present has value, empty string or otherwise, seems simple enough. You can add advanced semantics on load, the file does nothing by itself after all.
        TylerE 3y ago
        It’s that “or” that bites you if you aren’t real sure of the semantics in play.
        Guthur 3y ago
        There is no 'or' for a given file, it is declaratively there or it is not. The 'or' semantics are part of your language environment not the toml file.
lexicality 3y ago
The Billion Dollar Mistake lives on. Personally I'm happy it doesn't have explicit null and somewhat surprised it supports NaN (which I thought it didn't) To my mind, a configuration file using "this key doesn't exist" as one of several valid options seems deeply unsound.
- the_mitsuhiko 3y ago
  > The Billion Dollar Mistake lives on. I disagree. There is a big difference between configuration language formats and programming languages. > To my mind, a configuration file using "this key doesn't exist" as one of several valid options seems deeply unsound. I would agree, but in that case you now often need a secondary key to "turn something off" which is often not done.
  - lexicality 3y ago
    > There is a big difference between configuration language formats and programming languages. You're right! But in my opinion the difference is that a configuration file should have a default state that could be written to disk and loaded with no change in behaviour. eg you can set `feature = "on"` or `feature = "off"` and if you don't set either it's the same as writing `feature = "off"`. Having a secret third thing of `feature = null` should be outlawed and I'm glad TOML doesn't encourage it
    - the_mitsuhiko 3y ago
      There is a secret third thing called "feature not in file" and most TOML implementation encourage it as "key not in file" is typically returned as `null` or `None`. That's how so many TOML files in practice end up with all these secret values. It's also particularly odd in lists where I have seen `["foo", "bar", {}]` show up in the real world with `{}` being used as a replacement value for null.
      - lexicality 3y ago
        In my experience (Python and Rust) missing keys are errors, not silently converted to None: In [1]: example = {"foo": None} In [2]: print(example["foo"]) None In [3]: print(example["bar"]) --------------------------------------------------------------------------- KeyError Traceback (most recent call last) Cell In[3], line 1 ----> 1 print(example["bar"]) KeyError: 'bar' In [4]: print(example.get("bar", "sensible default")) sensible default If configuration values were arbitrarily nullable I would either need to deal with the possibility of nulls ending up anywhere in my program (back to The Billion Dollar Mistake) or have to have checks on parsing to say which fields are nullable or not. I acknowledge people have done silly things regarding missing keys already - but I think they should be discouraged from continuing to do so, not enabled.
arp242 3y ago
It's been discussed a few times over the years: https://github.com/toml-lang/toml/issues/921 https://github.com/toml-lang/toml/issues/921 https://github.com/toml-lang/toml/issues/803 https://github.com/toml-lang/toml/issues/803 https://github.com/toml-lang/toml/issues/802 https://github.com/toml-lang/toml/issues/802 https://github.com/toml-lang/toml/issues/146 https://github.com/toml-lang/toml/issues/146 https://github.com/toml-lang/toml/issues/30 https://github.com/toml-lang/toml/issues/30 Obviously there is a bit of a demand, and it certainly has valid use cases, but also has downsides.
- the_mitsuhiko 3y ago
  I'm aware of the discussions. I also gave up trying to get NULL into TOML and I think I'm reference in one of those issues. Personally I do not create TOML configs where absence of key has a meaning, but I'm also exposed to other people's TOMLs where this is an actual issue with all the consequences this causes. And that ignores the issue of nulls in arrays which are sometimes needed. For instance I have been exposed to parameterized SQL queries in TOML where you need to use the empty object to represent NULL, a particularly absurd incarnation of this.
  - arp242 3y ago
    I'm not saying there aren't reasonable and valid use cases, there obviously are. But there are also downsides, so the question is how it all balances out. Almost every feature that has ever been added to anything has reasonable and valid use cases, but clearly not everything should have all the features.

gscho 3y ago

In case anyone is interested I made an online toml syntax checker in the same vane as jsonlint.com and yamllint.com: https://toml-lint.com https://toml-lint.com

bruce511 3y ago

I'm puzzling over where I would use this sort of thing, over say Json, or simple xml, and I realise that it's mostly useful in cases where you're expected to edit the configuration in a text editor. That's not something I come across a lot - all my software (that I use) has a visual interface, and so raw editing of text/config files is not something we do. So context really comes into play here. I can see how this would be very useful for some programs, and near useless for others. And more specifically useful for those who consume user-edited configuration files.

zaebal 3y ago
It also reminds me the syntax of INI files.
lexicality 3y ago
Curious at to what technologies you work with? The work that I do and tools that I use have all manner of configuration file syntaxes and languages and a big headache for doing sysadmin work is trying to remember if a particular .rc file uses INI, YAML, JSON, TOML or some home rolled nonsense.
Sujeto 3y ago
I moved some config files from json to yaml. Maintaining the documentation became a lot easier, because before I had a document describing every config option, which I had to maintain separately, instead now I have comments in the config file itself.
- bonzini 3y ago
  Note that now you have "the Norway problem" instead. https://hitchdev.com/strictyaml/why/implicit-typing-removed/ https://hitchdev.com/strictyaml/why/implicit-typing-removed/ There's a whole spectrum of config formats including JSON, JSON5, strict YAML, YAML. It's quite messy.
  - GuB-42 3y ago
    I found yaml to be surprisingly complex and tricky to write for how simple it looks. It is made to be beautiful and easy for a human to read, but the trade-off are a lot of implicit rules you need to be aware of.

SPBS 3y ago

JSON is basically perfect if it allowed trailing commas and comments. TOML is not a replacement for JSON because of how badly it chokes on nested lists of objects (being both hard to read and hard to write), due to a misguided attempt to avoid becoming JSON-like[1]. [1] https://github.com/toml-lang/toml/issues/516 https://github.com/toml-lang/toml/issues/516

eviks 3y ago
still won't be prefect, all those extra quotes are bad for a heavily used human readable configs format
arp242 3y ago
That's already fixed; in the upcoming TOML 1.1 you can write: tbl = { hello = "world", } All the examples in the issue you linked should work. https://github.com/toml-lang/toml/pull/904 https://github.com/toml-lang/toml/pull/904
- bdudbdjsh 3y ago
  published data standards having version is so wrong. now if you see "config in yaml" you know nothing, zero, nada, about the format because all versions are so different and everyone implemented the version at the time and didn't bother to mention version. not to mention you can use a dozen syntaxes for yaml/toml and each application may not understand them all. all this is so silly. we will stick with json and informal-ini forever one way or another.
  - eviks 3y ago
    Since it's impossible to design anything great in tech on first attempt, what is the road to improvement without versions? (and mentioning version could be a requirement in a future version)
  - arp242 3y ago
    It's not ideal, I agree, but it solves real problems for people, so there's that. Many commonly-used standards today weren't created by a bunch of wise men in a room thinking how to bestow their wisdom upon the rest of us, they often originated in real-world applications, were refined over a period of time based pn real-world experience, and then became a standard. TOML is the same. I hope it will become an RFC some day. We just need to fix a few outstanding issues first. Most commonly used TOML parsers support 1.0; adding 1.1 support should be pretty easy as the changes aren't that large (I did it in the parser I maintain, and it's 10 lines of code or so that had to be changed, most of them quite trivial).

conaclos 3y ago

I pretty like the idea of a superset of JSON that supports (1) comments, (2) trailing commas, (3) unquoted properties, (4) optional {} for the root object, (5) multi-line strings, (6) number separator. JSON6 proposal [1] supports all of this, except (4). Unfortunately it also supports more and make the spec a bit too complex for my taste (JSON has a compact spec; any extension should honor this). Same issue with JSON5 proposal [2]. EKON [3] is certainly the best candidate. However, it didn't get any traction. [1] https://github.com/d3x0r/JSON6 https://github.com/d3x0r/JSON6 [2] https://json5.org/ https://json5.org/ [3] https://github.com/ekon-org/ekon https://github.com/ekon-org/ekon

4ad 3y ago
Among types and other useful properties, CUE supports everything you asked for. We're even considering adding a "data-only" mode to CUE which would be exactly what you asked for. https://cuelang.org https://cuelang.org
- conaclos 3y ago
  Is there any concise representation of the language spec? Similar to the spec of JSON [1]. [1] https://www.json.org/json-en.html https://www.json.org/json-en.html
  - 4ad 3y ago
    No, we do have a spec[1], but it cannot be as concise as the JSON link posted because the JSON link is mostly only about syntax, and our spec discusses both syntax and semantics. And of course, we are a much richer language than JSON, with many more features and computational behavior. That being said, it would be nice to use railroad diagrams to describe syntax in our spec. [1] https://cuelang.org/docs/references/spec https://cuelang.org/docs/references/spec
codeflo 3y ago
JSON as a computer-generated serialization format doesn't need any of those features. But for human-written configuration files, at the very least trailing commas and comments are basically a necessity. The question then is which features to pick beyond that (in my humble opinion, none), and how such a new standard would gain traction. JSON5, JSON6, JSONC all basically attempt to solve the latter problem by branding, but so far, mostly have failed. It doesn't help that there's an insane amount of bike-shedding going on in this space. The people who have forked JSON5 to create JSON6 for example have created an endless amount of confusion over which standard to use, harming both of them. I think they might want to think really hard whether the inclusion of octal literals was really worth the hostile fork.

deleted 3y ago

[deleted]

dist-epoch 3y ago

Funny thing: TOML author/inventor calls it a mistake: > TOML is a bad file format. It looks good at first glance, and for really really trivial things it is probably good. But once I started using it and the configuration schema became more complex, I found the syntax ugly and hard to read. > I personally abandoned TOML and as such, I'm not planning on adding any new features > If you're still reading, I'd like to put forth CSON as a format of choice. https://github.com/avakar/pytoml/issues/15#issuecomment-217739462 https://github.com/avakar/pytoml/issues/15#issuecomment-2177...

mhd 3y ago
> Funny thing: TOML author/inventor calls it a mistake You mean the author of this specific python implementation, or did Tom Preston-Werner ever disavow it?
detaro 3y ago
Someone called Martin is the "author/inventor" of "Tom's Obvious Minimal Language"?
- kps 3y ago
  It might be “Tom OR Martin's Language”.
lionkor 3y ago
Thats not the author/inventor of TOML.
esafak 3y ago
Tom (aka mojombo) is the founder of GitHub. https://github.com/mojombo https://github.com/mojombo

pachico 3y ago

My main issue with TOML is precisely described in the homepage example: [servers] [servers.alpha] is error prone. An sub heading shouldn't need to know about its parent, much less to have to copy it entirely.

pphysch 3y ago
But then you need new syntax to indicate subheadings, which is also (probably less) error prone because you might omit it.
petre 3y ago
How do you convert it into a nested object then? You could use deeply nested objects like JSON or YAML does, but then it defeats the purpose of an easy to read configuration format. This is also a feature of Java property files.
- mdaniel 3y ago
  > This is also a feature of Java property files. To the very best of my knowledge, Properties are merely Map<String,String> (see: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Properties.html#getProperty%28java.lang.String%29 https://docs.oracle.com/en/java/javase/11/docs/api/java.base... ) and any subsequent interpretation happens in the application (e.g. Spring treating `foo[1]=bar` as a list-ish assignment but the actual key returned from `getProperty` is the string literal as written `foo[1]`)
ryan-duve 3y ago
I think it is not needed? Or did you mean something other than this? Python 3.11.3 (main, May 4 2023, 05:53:32) [GCC 10.2.1 20210110] Type 'copyright', 'credits' or 'license' for more information IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help. In [1]: !cat /tmp/x.toml [servers.alpha] x = 10 In [2]: import tomllib ...: ...: with open("/tmp/x.toml", "rb") as f: ...: t = tomllib.load(f) ...: ...: t Out[2]: {'servers': {'alpha': {'x': 10}}}

mhd 3y ago

Not to be confused with the Tom Programming Language[0], a kind of early 2000s proto-Swift. [0]: http://www.gerbil.org/tom/ http://www.gerbil.org/tom/

camgunz 3y ago

Rather than a JSON replacement, I like TOML as a YAML replacement. It's a lot simpler, I'm not confused by things not having a preceding dash, indentation isn't significant, etc.

IshKebab 3y ago
I disagree. TOML is an INI replacement. They both suck at nested structures but are good if you just have a load of top level sections containing simple key-values. YAML on the other hand is just as good at any nesting depth. I mean it's pretty awful at all depths, but the act of nesting doesn't make it any more awful.
- matsemann 3y ago
  I spent a whole day debugging an issue because an extra - dash in yaml. I had something like rules: - name: Only allow X - condition: ip=123 The - in front of condition of course makes a new list element. So I now had 2 rules. One with no conditions and a name, and an unnamed one with conditions. Since they are or'ed together everything for some reason I couldn't figure out went through, as I thought I had a single rule.. I hate yaml.
  - IshKebab 3y ago
    Yeah I've had this happen to me too. Truly awful format. I think the only reason it has caught on is because it has multiline strings, which are usually not great in other formats. Of course YAML fucks that up too by having a gazillion different kinds of multiline strings, distinguished only by impossible to remember symbols. But it does at least have them.
- camgunz 3y ago
  Yeah I guess we're saying the same thing: TOML as an INI replacement is more of a YAML as an INI replacement. Re: nesting, I think TOML's dotting is pretty elegant as it flattens the nesting you so often need in configs. It took me a while to get used to the double brackets (and it is pretty confusing that you can put them anywhere, not just after their parent section--what are the edge cases that benefit from this...) but now I'm bought in, to the point where I don't know if I've just gotten used to the madness or what. I like that those two things essentially flatten the config--if you were using the new multi-line-inline-tables (from the forthcoming 1.1) you could actually nest things but now I don't know how much I even want that.

alpaca128 3y ago

TOML is nice as a kind of extended .ini format but I don't like the approach to avoiding nested elements. I prefer KDL. It's like XML but with almost zero syntax. Probably not always the best idea because it's so niche but it's just really simple while having all the basics like comments and multiline strings.

grncdr 3y ago
KDL (assuming you mean https://kdl.dev/ https://kdl.dev/) looks great, thanks for mentioning it.
- p0nce 3y ago
  See also: https://sdlang.org/ https://sdlang.org/

omoikane 3y ago

Previously (2020-09-10): https://news.ycombinator.com/item?id=24436550 https://news.ycombinator.com/item?id=24436550 There was also an earlier entry where it was "Tom's Own Markup Language" (2013-02-24): https://news.ycombinator.com/item?id=5272634 https://news.ycombinator.com/item?id=5272634

nicoburns 3y ago

If one is looking for an alternative config format, the best designed I've seen is https://github.com/gura-conf/gura https://github.com/gura-conf/gura

eviks 3y ago
Thanks for the tip, pity they mandate 4 spaces, that's 2 too many
rascul 3y ago
Gura doesn't look too bad to me at a glance. Not sure if I'll attempt to use it in the future but I'll bookmark it for now.

overgard 3y ago

Another alternative is HOCON: https://github.com/lightbend/config/blob/master/HOCON.md https://github.com/lightbend/config/blob/master/HOCON.md I'd describe it as JSON-like with much more flexible syntax and the ability to reference itself.

deleted 3y ago

[deleted]

vander_elst 3y ago

I wonder why text protobuf is not widely used for config. There is a formal description of the structure and the configuration structure can be statically checked.

skybrian 3y ago
Maybe because protobufs are specified in a language that’s different from everything else and needs its own parser and codegen? Also, there’s often more of an impedance mismatch with the basic datatypes of scripting languages. For example, people using protobufs will often specify int64 by default, when you don’t actually need 64 bits but need more than 32. JSON support is often in a language’s standard library, or close, and its ambiguity around what numbers you can actually use can be seen as a worse-is-better approach. (There are similar tradeoffs with not specifying a max length of a string. It’s awkward, nit-picky, and error-prone to pick a maximum, but there are practical reasons you don’t want to allow unlimited lengths in a UI.)

IshKebab 3y ago

Naming it "Obvious" doesn't actually make it true unfortunately. JSON is still by far the most obvious format, and JSON5 fixes all the big issues with using it as a configuration format. I wish it had wider support. It's just so obviously better than TOML and YAML.

friend_and_foe 3y ago

I use TOML and JSON for everything. I like them both quite a bit. TOML gets a lot of criticism for not being JSON and JSON gets a lot of criticism for... not being entirely fit for it's stated purpose. However, they're both fantastic if you use them for what they're good at: TOML for human readable configurations that you expect a user to modify, JSON for representing data that a machine will read all of the time but that you want the user to be able to look at and grasp. I don't think either of them really need to grow to be good at both, they're fine as they are, just pick the right tool for the job.

Simran-B 3y ago
I don't understand the appeal of TOML. Why not use YAML instead? Seems a lot more "obvious" to read and write to me. And it's the best I know that is strong in both, human and machine readability.
- rascul 3y ago
  YAML is more complex and difficult to write than TOML.
- OtomotO 3y ago
  Yaml is incredibly complex. KISS
- MrJohz 3y ago
  YAML has a lot of extra stuff going on that can cause accidents if you don't take care. The classic example is the "Norway problem" where "no" (the country code for Norway) is parsed as "false" instead. If "no" is used as a key, this can cause the Norwegian data to disappear or to throw strange errors on load. The other big issue is that, by default, it allows relatively unrestricted code execution in many environments. If a user is ever able to submit data that is interpreted by a YAML parser, they can run arbitrary code, and potentially even commands. The fixes for these are both fairly well known (always quote strings and keys, use safe_load or an equivalent API), but it's very easy to make a mistake. TOML, by contrast, is much simpler - strings are strings are strings, they must always be quoted but other than that behave pretty much as expected. Similarly, there is no mechanism by which a TOML file can instruct the parser to execute arbitrary code as part of deserialisation, which makes it (a) a lot simpler, and (b) a lot safer.

wkdneidbwf 3y ago

i’ve been eagerly awaiting multi-line support for inline tables for what feels like forever

kmfrk 3y ago

In terms of growing scope, isn't the general case usually something along the lines of: TOML > YAML > JSON > SQLite I'm lazy, so I wish I could just manage them all in Sublime Text.

mcdonje 3y ago
No. Different ideal use cases, though they can overlap. TOML is great for simple config files that don't need anything beyond key/value or arrays. YAML is great for config files with more complicated data structure requirements. JSON is great for self-describing serialized interchange. SQLite is in a whole other class of technology. I suppose you can store config information in a table if you're already using SQLite in your app. If you're exposing it the way you would with a normal config file, the non-text format introduces friction.

viksit 3y ago

we need another version of this for LLM Prompts that is highly space efficient. has anyone come across that?

mdswanson 3y ago

Looks like Microsoft's .INI file format.

Waterluvian 3y ago

Using JSON for configuration was and continues to be a terrible blunder. TOML fills that role wonderfully. And it isn’t full of complexity that turns Yaml into a mess.

eviks 3y ago
Bad nesting alone would make it a similar blunder for configuration
shaunpersad 3y ago
I've had the complete opposite experience. Neither YAML or TOML are very obvious to me. I always have to double and triple check that my syntax does what I assume it will do. I never have that problem with JSON.
- Waterluvian 3y ago
  Not adding comments to a config file is recipe for complete disaster. You can kinda fake it but it’s just a silly headache. Which is recognized given there’s now “JSONC” but now you’ve got even more confusion determining what systems can actually accept comments or not. That’s always fun. I’m curious what parts of TOML do people find confusing? It’s such a tiny markup language.

enriquto 3y ago

Graybeard opinion: all of toml, yaml, xml, json, kdl, etc., are ridiculously over-engineered for simple configuration files. You can nearly always fulfill all of your needs with a simple two-column text file of key=value pairs that can be parsed by a trivial call to fscanf(3) or whatever your laguage supports (yes, even correctly discarding comments). The world would be a better place if people just gron'd their jsons, xml2'd their xmls and so on, thus pruning the universal dependency tree by a considerable amount.

rfoo 3y ago
--flagfile gets me really far but to be fair it sucks when you need nested kv
- enriquto 3y ago
  just use longer keys (with an ad-hoc separator, if you want)
  - rfoo 3y ago
    What about lists? Something like: --allowed-downtime-0-start=20230523170000Z --allowed-downtime-0-end=20230523170500Z --allowed-downtime-1-start=... Or --allowed-downtime-start='20230523170000Z,...' I can live with them, but neither looks clean :( Genuine question, as I actually like the plain old flagfiles/Java-properties-ish style.
    - enriquto 3y ago
      The output produced by gron looks quite clean to me. You just write your file as if it was a list of js variable assignments. allowed-downtime[0][0] = 20230523170000Z allowed-downtime[0][1] = 20230523170500Z allowed-downtime[1][0] = ... or if you absolutely want the names, you can make a "list of dictionaries": allowed-downtime[0].start = ...
Rapzid 3y ago
How do you handle data structures in your key/value world? Parse it out of some adhoc key string schema?

crabbone 3y ago

It's another junk format in the sea of junk formats. It sucks for describing data, its structure is too primitive, has wrong primitives, doesn't allow extension and has no mitigations for that. Essentially, it's used because it's simple to parse and there are existing parsers. But the problems it creates are very complicated. So, here's one example of a popular use-case: pyproject.toml. I mean, PyPA aren't the brightest minds around, so, they've created this joke. The stated goal of pyproject.toml was unification of dependency / requirements specification. But, what they ended up doing is this: [tool.joke] tool.joke.requires=<requirements> [tool.clown_fiesta] tool.clown_fiesta.requires=<requirements> The problem here is that the goal was to unify, to specify requirements only once... but in practice, you still have to write them twice, and, of course, it's possible that "joke" and "clown_fiesta" don't use the same format to specify requirements. So, you have no way to specify requirements for both at once, and no way to process similar structures, to present them differently to different tools. And so, PyPA has created an piece of junk software that today has to be used by hundreds of thousands. Because we live in the world of worse is better.

eviks 3y ago
Could you list a couple of great formats? The various issues are indeed widespread across all the popular ones, and there are too many formats to find the greats (wish there were a comparison site with all the features so you could easily exclude those that fail at a specific task)
jborean93 3y ago
Can you share an actual example where you need to specify requirements twice. With pyproject.toml I know of 3 ways requirements are specified and they are all used by different things: 1. Build requirements - requirements needed to build your package from an sdist 2. Runtime requirements - requirements your library needs at runtime 3. Extra requirements - optional runtime requirements For example my library pyspnego https://github.com/jborean93/pyspnego/blob/main/pyproject.toml https://github.com/jborean93/pyspnego/blob/main/pyproject.to... has all 3: 1. Cython for Win32 needed to build the sdist and setuptools as the general build system - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L1-L6 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 2. cryptography as a runtime dependency - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L28-L30 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 3. optional extras kerberos and yaml - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L33-L38 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... Granted the toml format and what is used in pyproject.toml has its warts but I'm curious what your joke and clown_fiesta examples are actually from as from where I am standing each section currently serve different purposes.

throwawaaarrgh 3y ago

TOML is, indeed, a human friendly config format. Whereas INI is a simple config format. YAML/JSON/CSV/XML/etc are not config formats at all. They are data serialization formats. If you don't know the difference, you probably didn't get a CS degree, or read or understand the specs. Config formats should be tailored to the application, because the entire point of a config file is to make it easy for a human to configure one specific application. That's why there are 100 different config file formats for old UNIX programs. For each one, it's vastly easier to use that one config format for that one application, than it would be to force some arbitrary data serialization format to represent what the human really wanted the program to do. TOML is better than YAML or INI for a config file, but what's best is rolling your own format. It literally leads to better outcomes. The user can express what they want faster, easier, clearer, and the application doesn't have to struggle with alternate implementations of the spec for some poorly defined data serialization format.

theknocker 3y ago
[dead]
majewsky 3y ago
Where do you draw the line between config file format and data serialization format? The first paragraph of the TOML spec says: > TOML is designed to map unambiguously to a hash table. Sounds like a data serialization format to me. Besides the obvious differences in design and stylistic choices, it only really differs from JSON or YAML in that the top-level object of the data structure must be an object. If this type of constraint is what makes it a config file format, I am going to change the world with my revolutionary new config file format "JSON, but the first byte must be {". As another evidence for the blurriness of this line: Cargo uses TOML for its config file (Cargo.toml) which matches the usage that you describe. But then it also uses TOML for the serialization of its internal locking data structure in Cargo.lock, which is a file that humans are specifically not told to edit directly because it just contains serialized data. It just so happens to use TOML, probably because they already had a parser for that available at the time. > TOML is better than YAML or INI for a config file, but what's best is rolling your own format. It literally leads to better outcomes. Disadvantages of rolling your own format: - You now have to write your own parser, which costs at least a bit of time and is going to be more prone to bugs than a widely-used deserialization library like Serde. I see that you dunk on data serialization formats being "poorly defined", but if the popular formats are poorly defined, then certainly the average tool developer is not going to do a better job when they actually want to implement the tool and not the parsing for a config file format. - Users have to learn the format, and have to mentally switch between formats for each application. I just hate how every homegrown config format uses slightly different syntax, esp. which character starts a comment. With `config.yaml` or `config.toml`, I don't have to guess. - If the config file is YAML or TOML, I immediately have some basic syntax highlighting in my editor.