3 ms·
This is factually wrong about CSV, which can store any character including commas and even \0 (zero byte), provided it's implemented correctly (a rather large p
by rwmj 13y ago
This is factually wrong about CSV, which can store any character including commas and even \0 (zero byte), provided it's implemented correctly (a rather large proviso admittedly, but you should never try to parse CSV yourself). Here is a CSV parser which does get all the corner cases right:
https://forge.ocamlcore.org/scm/browser.php?group_id=113 https://forge.ocamlcore.org/scm/browser.php?group_id=113
- tbrownaw 13y agoyou should never try to parse CSV yourself Why? Writing a correct parser is not significantly harder than figuring out how to interface to an existing parser library, and allows cool things like heuristic parsing of malformed files. OTOH it's shocking how many people can't write a correct CSV generator, even after being explicitly told what they're doing wrong (which is always either "you need to put quotes around the data" or "you need to double any quotes that are part of the data") and given examples.
- rwmj 13y agoIf you knew enough about CSV to be able to write a correct parser, then you'd know enough not to write one lightly. Here are some surprising valid CSV files: https://forge.ocamlcore.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=csv/csv.git;a=tree;f=tests;h=b8d1a991cf7eb5c2290dc8ec027c0c50261667dc;hb=HEAD https://forge.ocamlcore.org/plugins/scmgit/cgi-bin/gitweb.cg... The test program in the same directory shows the semantic content of each.
- tbrownaw 13y agotestcsv6.csv at that link is malformed. DQUOT is used to (1) escape itself, and (2) enclose strings. It is not a generalized escape character the way backslash is in C-family languages. Interpreting it as a generalized escape causes two problems. One, if you generate files that way, they will be unreadable by parsers written according to the RFC. Two, if you read files that way, you will silently garble files generated by someone who forgot to escape the quotes that were part of their data.
- rwmj 13y agoI'm afraid you're wrong about this. Excel generates and parses "0 as a zero byte. The RFC doesn't discuss how CSV files work in the real world. This is exactly what I was talking about in my comment above.
- maxerickson 13y agoThere is no such thing as a valid CSV file. The RFC declares itself "informational" and says things like there is no formal specification in existence, which allows for a wide variety of interpretations of CSV files. This section documents the format that seems to be followed by most implementations: There are certainly reasonable arguments about the useful subset of rules.
- cjfont 13y agoThanks, I came here to make sure someone said this. Seriously, escaping special characters in CSV isn't all that difficult if you do it right from the start.
- jrochkind1 13y agoYou can find a CSV parser that gets nearly all, if not all, the corner cases right for just about any language. The problem is that the data you will be given to parse, by some third party agency, will quite likely not have been produced in such a way to get all the corner cases right. As anyone who routinely has to parse such data is unpleasantly aware of. So now you've got to manually fix data, or have a parser that _doesn't_ get the corner cases 'right' but instead uses heuristics to try to get what was intended out of your particular idiosyncratic and illegal data.