4 ms·
> There's nothing wrong with CSV as it is for basic data transfer. I disagree. CSV is horribly underspecified and many parsers have conflicting ideas on how t
by Lazare 8y ago
> There's nothing wrong with CSV as it is for basic data transfer.
I disagree. CSV is horribly underspecified and many parsers have conflicting ideas on how things should work. I've hit areas where, for example, an API was generating a CSV that another API could read, but NOT if it was opened and then re-saved by Excel first (just re-saving, not editing), because Excel was making some trivial change in the format that the consuming parser couldn't handle. Then there's issues with encoding, localisation, and escaping (see, eg, https://chriswarrick.com/blog/2017/04/07/csv-is-not-a-standard/ https://chriswarrick.com/blog/2017/04/07/csv-is-not-a-standa... although that's just scratching the surface).
That being said, this proposal seems worse in every way that existing CSVs, but I don't think the issue is "there's a well specified CSV standard with tons of parsers that won't change to support this new one", it's more that there is no standard, all the parsers are incompatible, and no meaningful improvements are possible. :(
- hodgesrm 8y agoTo your point it's hard to see how the referenced article qualifies as a standard. It does not deal with escape characters or missing values, two places where CSV implementations tend to vary in random ways.
- Lazare 8y agoMy comment may have been unclear, but I agree with this completely.
- wlesieutre 8y agoI've recently been dealing with some CSV data exported from Excel. One of the columns has lengths measured in inches or feet. A column that says 14" exports as , '"14"""', It feels downright silly to me.
- deathanatos 8y agoA cell w/ 14" would be ,"14""", And I verified that all four of Excel's CSV varieties write this, but this is also what RFC CSV writers would emit. Not saying you can't pick up oddities elsewhere, but as stated, it's not that bad. (Not that I condone use of CSV… it's an awful format.)
- wlesieutre 8y agoAh you're right, the spot I was looking at was after having read it into Python and split into a list on the commas, so the outer ' is my string container. Whoops.
- ken 8y agoWhat we need, I think, is an HTML5-like effort from the industry. That's the one format I can think of which went from "complete mess" to "parses the same everywhere". 1. Look at how everybody is extending CSV in the real world 2. Make a new format which does those things, as closely to all the existing implementations as is reasonable 3. Write a fully-specified new syntax for it, including a complete parser (state machine including error cases) 4. Make a complete set of tests, which make it very obvious when you don't pass, for shaming everyone into compliance 5. Get buy-in from a major standards organization, and all of the major corporate players I don't see any improvements/replacements succeeding if they don't hit all of these points. "CSV 1.1" here hits #1 and #2 only.