5 ms·

Why is CSV bad? I'm a huge fan of SQL, but curious what pitfalls CSV has.

tyingq 8y ago
A pretty good summary of the sort of "deceptive simplicity" around csv: http://thomasburette.com/blog/2014/05/25/so-you-want-to-write-your-own-CSV-code/ http://thomasburette.com/blog/2014/05/25/so-you-want-to-writ... And the related HN discussion: https://news.ycombinator.com/item?id=7796268 https://news.ycombinator.com/item?id=7796268
thomasfedb 8y ago
Not standardised. Fields that contain commas, newlines, or non-ASCII chars kill puppies.
- ainar-g 8y ago
  >Not standardised. Technically there is RFC4180[1], but yeah, CSV has a lot of flavours. I remember a story about how Excel couldn't correctly open a CSV file because it had a header that had "ID" as the first field name, and Excel's file format sniffing algorithm though that it's a different format. [1] https://tools.ietf.org/html/rfc4180 https://tools.ietf.org/html/rfc4180
  - ChrisSD 8y ago
    Technically RFC4180 isn't actually a standard but perhaps the world would be better if people could agree to treat it as such. However those living in countries that use the comma as a decimal separator may disagree. > This memo provides information for the Internet community. It does not specify an Internet standard of any kind.
    - _dps 8y ago
      As someone from one of those countries I've never understood this criticism of CSV. JSON doesn't let you choose your own separator either but no one levels this complaint at it. Defensively quoting values that may contain commas is easy, and nearly always a negligible cost. This really seems like a total non-issue to me and I've never understood how it has gained so much mindshare among CSV critics.
      - CodesInChaos 8y ago
        The problem is Excel's horrible CSV implementation, and opening the file in Excel is one of the main applications of CSV. The German version of Excel expects semicolon separated CSV, where comma serves are decimal point and the date format must also be German. I found no way to create a CSV file that opens in Excel, regardless of locale. (And there are a million other problems in Excel's CSV implementation)
- deleted 8y ago
  [deleted]
ChrisSD 8y ago
Perhaps because there are a multitude of, sometimes ill defined or ad hoc, specifications that overlap one another in both subtle and not so subtle ways. So you can't assume CSV files produced by one program can always be read by another.

A pretty good summary of the sort of "deceptive simplicity" around csv: http://thomasburette.com/blog/2014/05/25/so-you-want-to-write-your-own-CSV-code/ http://thomasburette.com/blog/2014/05/25/so-you-want-to-writ... And the related HN discussion: https://news.ycombinator.com/item?id=7796268 https://news.ycombinator.com/item?id=7796268

thomasfedb 8y ago

Not standardised. Fields that contain commas, newlines, or non-ASCII chars kill puppies.

ainar-g 8y ago
>Not standardised. Technically there is RFC4180[1], but yeah, CSV has a lot of flavours. I remember a story about how Excel couldn't correctly open a CSV file because it had a header that had "ID" as the first field name, and Excel's file format sniffing algorithm though that it's a different format. [1] https://tools.ietf.org/html/rfc4180 https://tools.ietf.org/html/rfc4180
- ChrisSD 8y ago
  Technically RFC4180 isn't actually a standard but perhaps the world would be better if people could agree to treat it as such. However those living in countries that use the comma as a decimal separator may disagree. > This memo provides information for the Internet community. It does not specify an Internet standard of any kind.
  - _dps 8y ago
    As someone from one of those countries I've never understood this criticism of CSV. JSON doesn't let you choose your own separator either but no one levels this complaint at it. Defensively quoting values that may contain commas is easy, and nearly always a negligible cost. This really seems like a total non-issue to me and I've never understood how it has gained so much mindshare among CSV critics.
    - CodesInChaos 8y ago
      The problem is Excel's horrible CSV implementation, and opening the file in Excel is one of the main applications of CSV. The German version of Excel expects semicolon separated CSV, where comma serves are decimal point and the date format must also be German. I found no way to create a CSV file that opens in Excel, regardless of locale. (And there are a million other problems in Excel's CSV implementation)
deleted 8y ago
[deleted]

ChrisSD 8y ago

Perhaps because there are a multitude of, sometimes ill defined or ad hoc, specifications that overlap one another in both subtle and not so subtle ways. So you can't assume CSV files produced by one program can always be read by another.