4 ms·
> Unlike binary formats or database dumps, these files don't hide their meaning behind layers of abstraction. They're built for clarity, for resilience, and for
by IanCal 1y ago
> Unlike binary formats or database dumps, these files don't hide their meaning behind layers of abstraction. They're built for clarity, for resilience, and for people who like to know what's going on under the hood.
Csv files hide their meaning in external documentation or someone’s head, are extremely unclear in many cases (is this a number or a string? A date?) and is extremely fragile when it comes to people editing them in text editors. They entirely lack checks and verification at the most basic level and worse still they’re often but not always perfectly line based. Many tools then work fine until they completely break you file and you won’t even know. Until I get the file and tell you I guess.
I’ve spent years fixing issues introduced by people editing them like they’re text.
If you’ve got to use tools to not completely bugger them then you might as well use a good format.
- burnt-resistor 1y agoThey're standardized[0], so it's only stupid humans screwing them up. Maybe you need a database or an app rather than flat files. 0. https://www.ietf.org/rfc/rfc4180.txt https://www.ietf.org/rfc/rfc4180.txt
- IanCal 1y agoThat came far after csv files started being used and many parsers don’t follow the spec. Even if they do, editing the file manually can easily and silent break it - my criticisms are of entirely valid to the new spec files. The wide range of ways people make csvs is a whole other thing I’ve spent years fixing. It’s not about the stupidity of the humans, and if it was then planning for “no stupid people” is even stupider than those messing up the files. > Maybe you need a database or an app rather than flat files. Flat files are great. What’s needed are good file formats.
- burnt-resistor 1y agoTOML What's the problem?
- IanCal 1y agoWhat are you trying to ask? I don’t understand. I’m not talking about toml.
- burnt-resistor 1y agoI gave you a good text file format. You're acting like there are no good file formats. Either invent a domain-specific one, use a standard one, or use a different modality rather than complain that a utopia you won't bother to create doesn't exist.
- IanCal 1y agoCsv files are bad for many reasons, some of which are listed as positives in the article. I’m not talking about other formats.
- integralid 1y agoBut TOML is not a good file format. Quite the opposite actually. https://hitchdev.com/strictyaml/why-not/toml/ https://hitchdev.com/strictyaml/why-not/toml/
- ioasuncvinvaer 1y agoI found that post unconvincing. > It's very verbose. This is his example: https://github.com/crdoconnor/strictyaml/blob/master/hitch/story/map.story https://github.com/crdoconnor/strictyaml/blob/master/hitch/s... I think you shouldn't use yaml or toml for this. > TOML's hierarchies are difficult to infer from syntax alone True! The point of TOML is to flatten the hierarchical structures. I would argue your configuration files shouldn't have much nesting anyway. > Overcomplication: Like YAML, TOML has too many features Basically TOML has a date type and all associated problems and advantages. I think it's a reasonable thing to include. > Syntax typing I think this is a good thing. I want to know whether something is a string or a number.
- knome 1y agojson makes for a great flat file format these days, with jq around to munge the data in it. csv is pretty bad for errors. mostly use it to dump data when I need to pass it to someone that will want to shove it in excel.
- Someone 1y ago> They're standardized[0] From that article: “This memo […] does not specify an Internet standard of any kind” and “Interoperability considerations: Due to lack of a single specification, there are considerable differences among implementations. Implementors should "be conservative in what you do, be liberal in what you accept from others" (RFC 793 [8]) when processing CSV files”
- burnt-resistor 1y agoAre you AI? I was replying to a comment, not the article. Also, you're quoting me to myself: https://news.ycombinator.com/item?id=44837879 https://news.ycombinator.com/item?id=44837879
- fireflash38 1y agoIf you're reading in data, you need to parse and verify it anyway.
- IanCal 1y agoWhich you might not be able to do after it’s been broken silently.
- fireflash38 1y agoThat's still an issue with binary files too, and you can't even look at them to fix.
- IanCal 1y agoBinary files are not usually edited by hand, and should not get broken without a broken parser or writer. There can be many more ways of fixing them too because it may not be as ambiguous as broken sv files.