21 ms·
TOML – Tom's Obvious, Minimal Language
- admax88q 6y agoI've always had a soft spot for the classic .INI style configuration. Modding command and conquer through "rules.ini" was always an adventure.
- edoceo 6y agoIt's 2020 and I'm still using them for configs. Section, key, value seems to fit so nice for me. But! I'm only using INI for the simple (like bootstrap level) of the apps - once it's up start using like etcd, or other fancier stuff
- edflsafoiewq 6y agoOne nice thing about INI is it's really easy to modify programmatically (without eg. losing comments).
- csb6 6y agoI also like how it just does one thing: key/values. It doesn’t try to be a complex serialization format or its own full language - it’s just keys and their values. Granted, sometimes data needs to have more structure, but for settings/flags INI is very good.
- m463 6y agoI used python configparser and there's a grey area when configs are merged (like default stuff and user-config'd stuff)
- jasonpeacock 6y agoOf configuration file formats, TOML is my favorite. It hits the right balance of power and simplicity, it's been able to handle my needs with minimal effort and fuss. I tried Python's INI for configuration, but it was both too simple and too complex at the same time, very frustrating. YAML is a nightmare, the formatting is way too easy to get wrong and accidentally break your configuration. JSON is too strict/simple. You can't even have trailing commas in a list!
- ghayes 6y agoI recently wrote a TOML parser in Idris2. It's a cool language, but has some quirks. For me, it's that it's just a simple KV store with some sugar (that's nice!) but it also means that a lot of information is lost forever in parsing. For example, these two documents are strictly identical: [info] name = { first="bob", last="jones" } and info.name.first="bob" info.name.first="jones" This can be seen as a positive (it's a really simple document) or a negative (it's hard to machine-generate nice TOML, and it's easy to think there's more to the structure of a written document than there is).
- alecbz 6y agoCan you expand on the negatives? I don't think I understand what you mean.
- demurgos 6y agoI believe the main negative he is referring to is that there is no canonical representation for a given document. The input file can be structured to share common keys (in the example above it's `info`) or not. Since both inputs are parsed into the same value it means that writing a printer is harder: you now have the choice between different output representations. As a comparison, there's not much choice when serializing to JSON - the only variation is around whitespace. When serializing TOML you need more insight to decide what's the best representation for a human. A contributing factor to this issue is that TOML is mostly used for configuration files, so it often matters that the output is readable.
- booleandilemma 6y agoJSON is too strict/simple This is why I like JSON! I have seen the horrors of XSLT and I’m never going back.
- justaj 6y agoIMO it kind of sucks that you're not able to add comments to JSON easily. In TOML (or YAML) it's just as easy as prepending it with # at the beginning of the line.
- jiggawatts 6y agoTOML superficially looks clean, but it has many of the same problems other similar languages have. Such file formats are typically used for configuration files. Yet, a substantial amount of effort was expended on data types not commonly seen in configuration files, such as a date-time-offset. Meanwhile, much more common data types typically used in configuration files is missing, such as GUIDs, IP Addresses, and byte arrays. Most disappointingly, TOML has a "table" type, but unfortunately, just like other "RPC as Config" formats, the column names need to be repeated for every row, which is just crazy. It also makes no sense to talk about "arrays of tables", which are actually just "rows" in standard terminology. An array of rows... is a table. Here is a terse example of the type of thing that does come up very often in bulk provisioning of the type that needs config files full of data: @" Server,Subnet,IP db01,databases,10.1.2.3 db02,databases,10.1.2.4 user01,filers,10.3.10.1 user02,filers,10.3.10.2 "@ | ConvertFrom-Csv | New-VMBuildScript.ps1 I'm yet to see a config file format that can even approach this in terms of its readability and terseness.
- jasonpeacock 6y agoIt all depends on context. Honestly, I've rarely (never?) needed to store GUIDs, IP addresses, or byte arrays in a configuration file. So that's not as common of a situation as you're thinking - maybe for your experience, but there are many different uses for configuration. The example you give would suck in any configuration file format, and honestly I don't believe it belongs in a configuration file - I'd put it into a data file (e.g. a CSV as you show) and then in the configuration file I'd include a pointer to that data file to be loaded separately.
- DC-3 6y agoEvery server config ever requires an IP address entry which in itself is a non trivial proportion of all the user edited config files in the world.
- joshvm 6y agoBut surely strings are valid? Unless you want some kind of static type checking/linting for eg an ipv4 vs ipv6 key? Strings are very portable as well (POD). Plenty of real world applications use static hostnames and rely on DCHP to assign IPs. When you have systems that can fail hard and you need to replace a NIC, it saves having to update either a ton of config files, router configs, or your hosts file(s). Most networking library api calls also take strings as arguments for this reason. Not saying its a better solution, but there times when either an IP or a string hostname should be acceptable. I've never had to use it, but Qt does have a hostname class, for an example of what such a dtype looks like. The class explicitly handles conversion for you: https://doc.qt.io/qt-5/qhostaddress.html https://doc.qt.io/qt-5/qhostaddress.html
- bostonvaulter2 6y agoI hope this catches on some more. I'm tired of being bitten by all the issues that I encounter with yaml configs.
- MiroF 6y agoWhat are these issues? I've been using YAML comfortably for years
- alecbz 6y agohttps://noyaml.com/ https://noyaml.com/
- bostonvaulter2 6y agoThis link has a decent overview: https://www.arp242.net/yaml-config.html https://www.arp242.net/yaml-config.html There's also the NO/Norway problem (although it seems this instance of the problem may be fixed if you're always using newer versions): https://allan.reyes.sh/programming/2018/06/20/The-YAML-NOrway-Law.html https://allan.reyes.sh/programming/2018/06/20/The-YAML-NOrwa...
- axaxs 6y agoSame, but I doubt it. It's been around for many years. I had to migrate services away from it recently, even.
- mekster 6y agoJust bring up Toml to be used for your next project whether it's open source or company project. That's how things get more popular.
- rdlfo 6y agoI was constantly reading/writing config files and the structure and comments were what convinced me TOML was worth to try. I did end up going back to JSON for the schema validator bit, but I just found out that TOML is still working on it: https://github.com/toml-lang/toml/pull/116 https://github.com/toml-lang/toml/pull/116
- tasubotadas 6y agoIF you need schema validation, you should probably be using XML
- _jal 6y agoI'm personally sick of yet-another-config-formats. Why did designing metaformats become cool? They all have warts because using the same symbolic notation for data and structure leads to encoding issues that have cognitive overhead. So instead of getting used to the pratfalls involved in making mistakes in one format/library set, we can do it in a whole bunch of different ones. And of course if anyone uses the new format much much, we'll get multiple, slightly incompatible versions, all with different bugs. Thanks?
- JadeNB 6y ago> I'm personally sick of yet-another-config-formats. Why did designing metaformats become cool? I can't seem to find a date for it, but I think TOML is pretty old, so probably predates a lot of these YACFs. I think that the problem is that it's such an easy workflow: 1. No existing config format is perfect. Look at this should-be easy thing I want to do that isn't easy! 2. Create a new format that makes the desired should-be easy thing actually easy. If you have one or two pain points, then it's really easy to address them, and your life gets so much better for a little while. If you're lucky and you're a good YACF designer, then your life gets so much better for quite a while—and, by the time you realise why the design isn't perfect, you're already invested in it.
- shakna 6y ago> I can't seem to find a date for it, but I think TOML is pretty old, so probably predates a lot of these YACFs. Not _that_ old. TOML's first release was in 2013. But as a testament to the config format, it was only about a year old when Rust's cargo adopted it fully.
- thayne 6y ago3. Realize that making a good general purpose config language that works well for a variety of different use cases is actually quite difficult.
- dang 6y agoIf curious see also a bit from 5 months ago https://news.ycombinator.com/item?id=22707387 https://news.ycombinator.com/item?id=22707387 2018 https://news.ycombinator.com/item?id=17513770 https://news.ycombinator.com/item?id=17513770 2013 https://news.ycombinator.com/item?id=5272634 https://news.ycombinator.com/item?id=5272634 https://news.ycombinator.com/item?id=5274025 https://news.ycombinator.com/item?id=5274025 Alternative formats to this alternative format: 2018 https://news.ycombinator.com/item?id=17765426 https://news.ycombinator.com/item?id=17765426 https://news.ycombinator.com/item?id=18023105 https://news.ycombinator.com/item?id=18023105
- flukus 6y agoAnother one I've been partial too lately is recutils (https://www.gnu.org/software/recutils/ https://www.gnu.org/software/recutils/), it's similar to sqlite but has the benefits of being minimalist and plain text. Generally I prefer actual code based configuration though, there's only a small subset of people than can read and edit these config formats but would be unable to configure something like DWM through real code (https://git.suckless.org/dwm/file/config.def.h.html https://git.suckless.org/dwm/file/config.def.h.html) and this approach gives far more flexibility without code bloat. Once customization through config is removed it's only machine-to-machine configuration that changes and these are easily handled by TOML, INI or even environment variables.
- throwaway4007 6y agoI still don't understand why the Python steering committee decided to switch to this format for package configuration instead of the perfectly serviceable and existing setup.cfg. Packaging is already a pain to setup, now you gotta learn three (3) different ways to do it (if only to translate from one to the next) because everyone's split. The choice doesn't even matter very much, just settle on something whatever it is
- RedShift1 6y agoThere's not much to learn. It's basically .ini on steroids.
- quotemstr 6y agoIt's not that it's hard. It's that it's different, and every difference has a cost. IMHO, and in the OP's opinion, the benefit of TOML for packages isn't worth the mental cost of having to know about three configuration methods.
- thayne 6y agoWhat confuses me is why there isn't a toml module in the standard library.
- nuclear_eclipse 6y agoI think the general feeling in the community is that the standard library is where packages go to die. Once something is in the standard library, it can't get meaningful features/updates outside of the yearly release cycle. pip and setuptools are already vendoring a TOML implementation, and once you have pip, you can pick from multiple great toml implementations that are all under active development. Maybe once the packages/standards for TOML are finalized and the only thing left to do is fix bugs, maybe then we'll see a TOML implementation in the standard library, but in the modern age where you can fetch dependencies at any time, there's honestly not that much incentive for CPython developers to take on more burden of maintenance and development for the already-massive standard library.
- blaisio 6y agoTOML sucks in so many ways. It is about 8x better than YAML, INI, or JSON though. It is good enough that I can ignore its flaws.
- tasogare 6y agoJSON should be banned for configuration, if only for the fact it doesn’t support comments.
- nuclear_eclipse 6y agoOr trailing commas in lists/objects. That is 90% of what burns me every time I write JSON by hand.
- dnautics 6y agoJSON is great, but not for config: It was never really meant to be written by humans.
- krapp 6y agoOf course it was meant to be written by humans - every programming language and text based data format was meant to be both written and read by humans. JSON is basically Javascript objects, and Javascript was definitely meant to be written by humans.
- 1_player 6y agoI agree totally with you, but designing a language that's not "meant for humans" is an _incredibly_ stupid idea. Of course humans will try and write any protocol by hand. Who's gonna stop me? If it looks like text, make it as easy as possible for humans to consume and write by hand. Period. That's a small but very important reason why HTTP rules the world.
- deleted 6y ago[deleted]
- 6y ago
- skywhopper 6y ago“Obvious” is subjective. I find TOML to be remarkably confusing and surprising and UNobvious, whereas YAML makes perfect sense to me, even though plenty of folks hate it. Essentially, the name is pretty obnoxious in multiple ways, given that it’s basically just an extension of the long-established INI format.
- quotemstr 6y agoYAML and TOML wouldn't have been necessary if JSON had been slightly more expressive. If JSON had * comments * bareword (not double quoted) object key names it'd be a much more suitable configuration file format for everything, and I don't think we'd see the motivation to make things like TOML and YAML.
- Anaminus 6y agoJSON5 pretty much addresses all the problems with JSON as a config format. I'd love for it to just replace JSON entirely. https://json5.org/ https://json5.org/
- quotemstr 6y agoGenerally looks good, but ugh, I dislike allowing single quotes. Allowing single quotes has close to zero ergonomic value but increases the number of ways to write the same fundamental thing. But that's a minor quibble. If everyone adopted this thing instead of legacy JSON, we'd be better-off.
- gridlockd 6y agoHaving two ways to quote is a useful feature, not just one way to do the same thing. Javascript already has this, so allowing single quotes simplifies copypasting object literals, which is a common use case.
- tornato7 6y agoThat looks really nice. I might start using it for configuration.
- ratbeard 6y agoYep. And trailing commas,
- afiori 6y agoor also no commas at all
- mixmastamyk 6y agoStrict yaml claims to have fixed most of the problems of yaml and other languages used for configuration. Basically it is yaml with only string as a basic datatype, and other complexities removed. Handling of data types is done on a higher schema layer. https://hitchdev.com/strictyaml/ https://hitchdev.com/strictyaml/ It's a smaller project so probably could use some support.
- gscho 6y agoSelf plug: I wrote a simple web app in the same vain as jsonlint and yamllint for toml files. https://toml-lint.com https://toml-lint.com
- brundolf 6y agohttps://xkcd.com/927/ https://xkcd.com/927/
- peter_d_sherman 6y agoUnicode handling seems nice: >" \uXXXX - unicode (U+XXXX) \UXXXXXXXX - unicode (U+XXXXXXXX) "
- sepples 6y agoTOML is great if you think of it as JSON, but optimized for being written by humans instead of machines. I hope no new first-class types or syntax features ever get added after the 1.0 release so it can have the same stability that makes JSON great. The vast majority of the proposals that I see in the repo's issues would collectively destroy the value of TOML, and I'm glad the current maintainer seems to have the sense and fortitude to resist such additions.
- shean_massey 6y agoI had tons of headaches trying to get toml to do what's trivial in Json (nested arrays of objects of arrays of ... etc). 1/10 would not try again. For very simple, hear "flat" config it checks out. But then, so does yaml or even ini files. Anything requiring composite types was just a nightmare. I'd even prefer xlm before using toml again.
- hoseja 6y agoI never had to work with it seriously but I still don't get why people hate XML so much.
- etripe 6y agoI would venture a guess that on the surface, it's about its verbosity and the pyramids of doom you can get when editing. On a deeper level, I think they might be frustrated with the ability to develop custom formats (XSDs) which effectively make XML not one format, but a gazillion formats.
- qayxc 6y agoXML is verbose, surprisingly complex, and first and foremost not designed to be written and consumed by humans. There's plenty of room for debate, e.g. element versus attribute and things get unwieldy pretty quickly [1] The "note"-example illustrates the issue quite well: the order of the elements matters and you end up writing every element identifier twice. Not to mention the bloat that comes with using an XML library that's actually compliant with the standard and includes all the bells 'n whistles. [1] https://www.w3schools.com/XML/xml_dtd_el_vs_attr.asp https://www.w3schools.com/XML/xml_dtd_el_vs_attr.asp
- AdmiralGinge 6y agoPersonally I find it a lot less readable than things like JSON or TOML.
- GoblinSlayer 6y agoHipster meme that only xml has problems. Ironically they now went full circle to yaml, which is much more obtuse than xml.
- 02020202 6y agoI use only yaml for humans and json for machines. i see many projects that use toml but i just hate it. maybe toml is not bad itself but i have never seen project that would have toml config really human-friendly. it always loks like a retard-child of some ini monstrosity. yaml all the way, hands down.
- wtf_is_up 6y agoI've always enjoyed using HOCON in Scala projects. I wish it saw more adoption.
- XorNot 6y agoI can't stand TOML. I find it's method of representing arrays and tables of objects so much harder to read and reason about than YAML.
- rco8786 6y agoWhoa cool. I remember seeing Tom’s initial proposal/spec for TOML on here years ago. Like maybe a decade even. Glad to see it’s still out there and actually been implemented
- bvinc 6y agoIt's used often in rust. Rust crates are configured with toml files.
- fbnlsr 6y agoI discovered TOML a couple of years ago when I started playing with Hugo, as it was Hugo's format of choice for configuration files. I honestly find TOML harder to read and more complicated to use than YAML. I tried to look into it but frankly I still haven't found a use case where it made more sense to use TOML.
- alwillis 6y agoThis has been my experience as well. There's been times I wanted YAML support in a particular app; I've never wanted TOML support. If anything, it's been "ugh, I have to use TOML".
- darthrupert 6y agoInteresting how differently competent people can perceive things. To me, YAML is a cruel joke, and I carry suspicions about the competence of anyone who opts to use it out of all possible options. I think I've even grown to like XML configuration over it.
- baby 6y agoI’ve had all sorts of issues with yaml due to indentation, so I don’t use yaml anymore.
- m463 6y agoI tried config files in the .INI type syntax of python's configparser but turns out there is a lot of hidden complexity where the "default config" intersects the "user config"
- deleted 6y ago[deleted]
- AlexandrB 6y agoAnd so the language churn continues. This time for configuration. TOML is fine, I guess. It doesn't seem to offer much benefit over Yaml or JSON. In the absence of those it would be great. But in a world where Yaml and JSON exist it's yet another structured data format you have to learn.
- metalliqaz 6y agoIt's not like TOML is new. I'm familiar with it because of Python's pip tool, and InfluxDB/Telegraf. Apparently it's also used in Rust's Cargo.
- BiteCode_dev 6y agoPeople keep confusing program configuration files and program stdin and are surprised that TOML is not a good fit for it. If you are provisioning servers from a TOML files, you're doing it wrong. That's your program main input, not config. If you have deeply nested values, chances are you are again confusing configuration vs main input. JSON and CSV are great main input formats. TOML is a great configuration format. YAML and XML try to do both, and end up doing average on either, but being abused for all of them (looking at you Ansible and Solr). So you should use each format for the correct purpose, as usual. A config file looks like ~/.ssh/config or /etc/nginx/nginx.conf, not like ~/.ssh/authorized_keys or /etc/nginx/site-available/default.conf. Just because we love to put a lot of data input in config folders (.config and /etc are full of input data) and call that configuration doesn't make it so. How to know if something is configuration and something is input? Conf data is usually only manually edited (authorized_keys changes with the life cycle of the program) and doesn't contain logic (default.conf if full of logic). Conf is also how the program is going to behave when performing its main task, the boostraping state, not the data used for it's main task (so not ip for provisioning servers, which is not meta, but the main course, or default.conf, which nginx uses to perform the main task). Conf rarely changes, main input often does. Conf is not piped or redirected to stdin.
- pornel 6y agoTOML used as an `.ini` file is nice, but it gets verbose and confusing pretty quickly if you try to store more complex data in it. `[[foo]]` syntax is not obvious at all. It's a bit weird that top-level declarations use ini-like syntax, but values can use JSON-like syntax, and there are multiple syntaxes to express the same data structure. It doesn't strike me as very elegant, or obvious. Still, it is less annoyingly-inflexible than JSON, less verbose than XML, and less footgunny than YAML.