13 ms·
Hey, Tom here (creator of TOML). Fun to see TOML on HN again! Since I first wrote a (mostly) joke proposal for TOML 5 years ago, TOML has been adopted by a numb
by mojombo 8y ago
Hey, Tom here (creator of TOML). Fun to see TOML on HN again! Since I first wrote a (mostly) joke proposal for TOML 5 years ago, TOML has been adopted by a number of prominent projects such as Cargo, Hugo, Pipenv, and others.
TOML is especially well suited for projects that need a simple configuration file that maps unambiguously to a hash table. There are still some weaknesses in TOML that make it non-optimal for large, complex config, but I'm hoping to address that in a later version of the spec (perhaps 2.0).
Happy to answer any questions you all have about TOML!
- nixpulvis 8y agoHi Tom, huge fan of this project. 1. Would a file path type make any sense to add? TOML is ideal for configurations, and so many configuration options use file paths. I know there are questions about platform dependency, and I don't have all the right answers for how that would work, but I could see it being possible somehow? 2. Data versioning is a big deal to me. For now there's always an option of just adding a "version" key and updating it, but can you think of a better way to do that with TOML? I'm always overly critical of JSON (despite overall being a fan) because it doesn't specify bit sizes of numbers. I notice that TOML has specified 64 bit numbers. It's nice to see that.
- mojombo 8y ago1. How would the file path type differ from a regular string? Would you want specific semantic validation for each platform? 2. This is still a big question, and we've been having a robust argument about it on GitHub [1]. It's a problem that so far has evaded an elegant solution, but usually when that's the case, it just means we haven't been creative enough about it yet! Or did you mean versioning of the data IN the config file? Can you elaborate? [1] https://github.com/toml-lang/toml/issues/522 https://github.com/toml-lang/toml/issues/522
- nixpulvis 8y ago1. Yea the goal would be some kind of validation at parsing time. I'm not super familiar with TOML parsing, but there must be error cases in the implementations, so it seems like using `\` on UNIX systems, for example could return an error. The only hard part about this I see is distinguishing paths from strings syntactically without making TOML overly complex. 2. So basically, the majority of people don't want to type version numbers (I see where they are coming from, but respectfully disagree). I'm not sure if there exists a solution in that case. As for my comment, I was really talking about both kinds of versioning. Mainly that, if my data is version 2.3.1, I'd expect that to also lock the version of TOML, requiring a new version of my data for bumps to the underlying data structure. Again syntax is the hard part, as I'm having trouble imagining a way to actually do this. Furthermore I've seen data formats in the past that let you version "parts" of a format, I personally like this, but don't think it's a good fit for TOML.
- bqe 8y agoBackslashes are valid characters in a POSIX filesystem, but they don't indicate a directory.
- curun1r 8y agoYeah...rather than validating paths, the better use case is probably normalizing them so that you don't need platform-specific configuration files. Being able to write: static-files = path/to/static/files And have it work on all platforms gives a distinct advantage over specifying paths as strings.
- earenndil 8y agoThat was my first thought, but imo such a functionality belongs in the language's file access library.
- Stratoscope 8y agoEvery modern operating system accepts forward slash paths. It's been a long time since there was any need for things like Python's os.path.join() or special file path knowledge in config files to convert slashes to backlashes on Windows. You can just use forward slashes everywhere and not worry about it. The one exception on Windows may be paths in a CMD.EXE command line, where / may be confused for a switch character, but I would think a config file should not be passing paths on a command line but instead passing them directly into a program - and then the forward slash will work fine.
- KirinDave 8y agoHi Tom. Dave here. I like a lot of the features in TOML, but I'm curious, can I have a pony? (Folks downvoting, PLEASE JUST LET PEOPLE HAVE JOKES EVERY NOW AND THEN)
- mojombo 8y agoHi Dave! Ponies come standard in TOML, you just have to know where to look. =)
- nixpulvis 8y agoNo soup for you.
- kemitche 8y agoToml is better than a pony. Proof: http://toml.versus.horse http://toml.versus.horse
- btilly 8y agoYou aren't going to change what people around here like. And complaining about downvotes usually just attracts more. If you think that your brand of funny is worth braving people's disapproval, do it knowing what the reaction is going to be. If you don't like people disapproving, then go elsewhere, or engage here in a way that people prefer.
- KirinDave 8y agoI made the joke for Tom. It has special significance for him.
- mojombo 8y agoDave and I worked together at Powerset many years ago (pre-GitHub) and wrote a bunch of Erlang together. We have a lot of inside jokes from those days. LEAVE HIM ALOOOOOOOOOOOONE!
- btilly 8y agoThat's fine then. Make the joke for Tom. Ignore the inevitable downvotes. Nobody else has to understand or like it.
- stevefan1999 8y agoThis project reminds me of the good old INI files.
- mojombo 8y agoAs it should! It was inspired by INI, but with a desire for a proper spec that unambiguously maps the config file to a hash table.
- CMay 8y agoThis was the first thing I noticed as well. It was always somewhat alienating when encountering newer formats that were less friendly than what came before. I think TOML does a good job of taking what came before and bringing it forward. Over the many years of the usage of INI I think occasionally we've seen developers add their own touches to how they support types, sections, groups, multi-dimensional arrays or other structures within config files that certainly indicated a need for some more standardized advancement here. I really hope this gains a lot of traction, because there are so many scenarios where things like XML or JSON are forced into roles they really don't belong in. One thing that I would say is that it might be worth indicating a standard header that can be used at the very least in cases when it is not stored in a file with a .toml extension. It would not surprise me at all to see TOML widely used with extensions that describe better the intent of the file rather than the format.
- echohack5 8y agoHey Tom, I just wanted to say thank you real quick. We use TOML in [Habitat](https://github.com/habitat-sh/habitat https://github.com/habitat-sh/habitat) to define default and overridable configuration. It's super easy and expressive. The [Habitat Core Plans](https://github.com/habitat-sh/core-plans https://github.com/habitat-sh/core-plans) have quite a few examples about how it gets used in the project. If you are looking for examples of larger configuration files with .toml, then that might be a good place to look for some real world use. Thank you again!
- mojombo 8y agoAmazing, thanks for the heads up! I'll add it to the list of projects using TOML [1] and take a look at your example configs to get some more real-world uses cases to look at when evaluating future changes to the spec. [1] https://github.com/toml-lang/toml/wiki https://github.com/toml-lang/toml/wiki
- yuchi 8y agoQuick link to a complex one: https://github.com/habitat-sh/habitat/blob/master/components/sup/tests/fixtures/sample_configs/complex_config.toml https://github.com/habitat-sh/habitat/blob/master/components...
- deleted 8y ago[deleted]
- jhabdas 8y agoThe one problem I've had, and perhaps I'm just doing something incorrectly, is going from a `[[something]]` back to a `[thing]` without TOML thinking that `thing` is part of `something` it's not.
- mojombo 8y agoCurious. In order to be part of one of the tables in the [[something]] array, a table would need to be named [something.X]. What parser are you using? it's possible that it's behaving improperly in this case. See the relevant part of the spec here: https://github.com/toml-lang/toml#user-content-array-of-tables https://github.com/toml-lang/toml#user-content-array-of-tabl...
- hyperpape 8y agoI'm a pretty big fan of TOML, though I've mostly only used it for Cargo. What are the limitations for large complex configs, and do you think there's currently a better configuration format for them? Feel free to link elsewhere if there's a canonical location for this discussion.
- mojombo 8y agoThe limitations are mainly around nested arrays of tables. As has been mentioned here already, the [[array-of-tables]] syntax that currently allows this can be super confusing, and is definitely the worst part of TOML right now. Even so, you can often work around this weakness by reworking your config file to pick up on a specific naming scheme (perhaps every "server-*" table is interpreted as a member of an array) or maybe there's an array of strings that contains the names of the tables, each of which becomes an array member in that order. But these are hacks, and it would be nice to have a first-class way of dealing with nested large tables that was easy to scan and not too repetitive. That said, the alternative right now in other formats is basically a { bunch { of { braces } } which is also not the most amazing thing for long nested tables.
- hyperpape 8y agoThanks, is there a strawman proposal out there for fixing this, or is it something for the distant future?
- alokitr 8y ago> The limitations are mainly around nested arrays of tables. As has been mentioned here already, the [[array-of-tables]] syntax that currently allows this can be super confusing, and is definitely the worst part of TOML right now. Good to know this is recognized. Looking forward to a better syntax to an indeed difficult problem
- Groxx 8y agois this e.g. due to the verbosity of re-declaring field names every time? or something else? e.g. CSVs handle 2D tables reasonably efficiently, since you just specify values: field_a, field_b, field_c 1, 2, 3 3, 4, 5 but I imagine TOML would benefit from something that lets you define nested tables without re-defining the table every time... which probably conflicts with the "obvious" goal. Hmm.
- ksec 8y ago> There are still some weaknesses in TOML that make it non-optimal for large, complex config May be the config shouldn't be large and complex in the first place? My biggest question is, why hasn't TOML used much more widely or becoming de facto standard? Am I missing something obvious?
- mojombo 8y agoI'm always in favor of small config files, but sometimes for larger projects that's just not possible (or desirable). TOML has always been about being as simple as possible while still solving a wide range of problems. But I think of it as the 80% solution, where 20% of projects might need something more powerful to solve their more complex needs, and that's ok. We still need a super simple config file format for the 80%. As for adoption, it takes a long time for something like TOML to be adopted. Inertia is powerful in the development space. I also haven't had as much time as I would like to push TOML forward and evangelize it. My hope is that at some point there will be a tipping point and you'll see most projects start with a TOML config and only change to something else when TOML can't meet the project's needs.
- simcop2387 8y agoYes I've got a project and config that I tried to stuff into TOML and even make my own DSL for it, but I haven't gotten anything satisfactory still. I've settled on some semiugly yaml for now until I try again https://github.com/perlbot/App-EvalServerAdvanced/blob/master/skel-sandbox/etc/seccomp.yaml https://github.com/perlbot/App-EvalServerAdvanced/blob/maste... The rest of the sandbox is configured with TOML however. Just seccomp rules that I haven't figured out the perfect way yet.
- dustingetz 8y agoTom, have you heard of EDN and Transit? Like JSON, but: 1. Extensible, custom types 2. Streamy, efficient, compressed and fast, leverages platform's existing native/optimized JSON parsers 3. Wide platform reach 4. Made by Rich Hickey http://blog.cognitect.com/blog/2014/7/22/transit http://blog.cognitect.com/blog/2014/7/22/transit
- mojombo 8y agoI have not, and while it looks cool, it seems to be optimized for data transfer between applications, not a human readable config format.
- KingMob 8y agoTransit is more about data transfer, but EDN is mostly meant to be a better JSON (useful date types, doesn't care about commas, has comments, etc)
- svacko 8y agoThere is also a similar project - Hjson, a user interface for JSON- https://hjson.org/ https://hjson.org/
- epage 8y agoFor me, hjson takes on some of the warts of yaml (bare strings) which is why I prefer json5 if I'm going for a json variant.
- DonHopkins 8y agoSo like YAML, TOML ain't markup language either, despite the recursive acronym and the "ML" suffix. If your name was Mark instead of Tom, would you have named it MARKML, and would it still not be markup language?
- adrianmalacoda 8y agoThis does not claim to be a markup language. According to the repo history, it did... 5 years ago, until someone noticed and corrected it.
- gregn610 8y agoA killer feature of TOML compared to JSON is that it allows comments. A config file without comments and examples ain't great. I found the double square bracket syntax useful and understandable. Agreed it's not obviously .INI or perfectly elegant but it certainly works and has its use-cases. Anyways, thank you @mojombo!
- alokitr 8y ago> I found the double square bracket syntax useful and understandable Really? I had kind of the inverse reaction to it. Indeed useful, but definitely not understandable
- adrusi 8y agoIt's kind of similar to the syntax of uri query strings and the php syntax for appending to arrays, in that an extra pair of square brackets indicates appending: In php: $a[] = 1; $a[] = 2; # $a == array(1, 2) In URIs: http://example.org/?a[]=1&b[]=2 Compare to TOML [[a]] item = 1 [[a]] item = 2
- majewsky 8y agoThat URI syntax is just a convention used by some web frameworks. You can just as well write https://example.org?foo=bar&foo=baz
- TimothyBJacobs 8y agoNot for PHP at least, maybe others, the last query arg will win.
- kolpa 8y agoJson allows comments, but they have to be in string fields of an object so they survive reserialization.
- 8y ago
- liquidgecka 8y agodamn it.. of course you would update the spec right before I release my config parser. =) I have been working on a TOML reader/writer for golang that supports read/change/write and format (like go fmt) that sometime I will get enough time to actually finish and release. =)
- mojombo 8y agoHah, sorry! You can always release an 0.4 compatible first, and then add the stuff from 0.5. Sounds like a great project!
- elcritch 8y agoThat TOML started out tongue-in-cheek makes it all that much better! Just a curious thought: Sometimes it’s handy to have a text file for small bits of numerical data. Would it be possible to extend TOML to have a “csv” section (array of arrays)? [x,y,z] 1,2,3 4,5,6 7,8,9 Or perhaps it’d have to be: [data: x,y,z] 1,2,3 4,5,6 7,8,9
- 11235813213455 8y agoTOML needs anchors (like yaml)
- zifnab06 8y agoTom here (not the creator or TOML). Every time I have a "what are we using for configuration" discussion at work for a new project, I get to say "let's use TOML, it's obviously superior, just look at it's name!". It's a horrible joke and I've yet to actually use it, but it makes me happy :)