4 ms·

Why are people using CSV when better (and less fuzzily defined) solutions exist, such as JSON?

by mantrax5 12y ago

minimaxir 12y ago
CSV is far, far more ubiquitous and much more usable in non-web settings. (e.g. desktop data analysis programs)
- deleted 12y ago
  [deleted]
- jgalt212 12y ago
  true, and in those settings you largely don't see situations that trip up naive parsers such as newlines or delimiters inside fields.
jiggy2011 12y ago
Importing into excel is probably a big reason.
- mantrax5 12y ago
  If Excel compatibility is the goal, one should use libraries that read and produce Excel files. CSV is bullshit, it's not good for anything except scenarios where you control both the export process, and the parser (so you know what delimiter is used and so on).
  - minimaxir 12y ago
    The same could be said about exporting to JSON. (the JSON code on some of the big APIs does not parse intelligently and I have to spend a lot of time fixing it.)
    - deleted 12y ago
      [deleted]
  - mrweasel 12y ago
    CSV files are MUCH easier to search and inspect using tools like grep and less. It's the accounting people that want's Excel, but as a developer CSV is easier and more flexible. We process csv files containing payment information, export csv as product feeds for perhaps 10 different partners. It works, it's faster than XML or JSON and is easy for non-technical people to inspect using tools like Excel. Most cvs data exports are not something that's made up on the fly every day, it batch jobs that create the same type of CSV file each and every time. CSV is alright. XML is bullshit.
    - Roboprog 12y ago
      CSV is much simpler when the records are all of one type. It gets debatable which (XML vs CSV) is simpler when you get multiple record types dumping a hierarchical data structure. Obviously, you can make CSV files work for alternating record types, in the same way that the old mainframe files used to work with multiple record types in a file, with a type descriminator field in a know place, usually field 1, of each record. But it starts to get cluttered.
jasode 12y ago
In addition to aforementioned import/export data interop with MS Excel, there are tons of legacy systems (mainframes, etc) that import/export csv but not XML or JSON. The csv format is everywhere and will continue to be with us for decades. People will always look for a quality library (in whatever new programming language) that handles all tricky edge cases. A few months ago, I was trying to get some bulk data into ebay's proprietary TurboLister[1] program. Guess what, it can import csv but not JSON. SQLite[2] can import csv but not JSON. Google's terabyte ngram dataset[3] is csv (tsv) instead of JSON. I'm glad it's not JSON because it would have required extra disk space. ... plus tons of other real-world csv examples out in the wild. Unfortunately, the csv format is very easy for programs to write but it's very difficult for programs to properly read because of the tricky parsing. [1] http://pages.ebay.com/sellerinformation/sellingresources/turbolister.html http://pages.ebay.com/sellerinformation/sellingresources/tur... [2] http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles [3] http://storage.googleapis.com/books/ngrams/books/datasetsv2.html http://storage.googleapis.com/books/ngrams/books/datasetsv2....
jzwinck 12y ago
If your data are rectangular and you care about performance, CSV is better than JSON just because it avoids repetitive key names everywhere. Then again, if your data are rectangular and you really care about performance, you would not use any of these (you might use HDF5, which has support in many programming languages and will destroy the others in terms of speed).
- lukeschlather 12y ago
  JSON is almost a subset of CSV, with the understanding that you have to wrap every line in [], the document in [], and every field must be quoted. (And JSON doesn't have built-in support for headings, so you need to write a little loop instead of the library building a hash for you.) So no, if you control input and output, JSON is still easier to use than CSV, and just as performant. JSON stores straight arrays just fine. It's not the format's fault so many people choose to store hashes with it.
  - Someone 12y ago
    ...and quotes inside fields must be escaped with a backslash, newlines replaced by \n, etc. ...and, the moment you declare you handle json, people will send non-string data ("that is a number, of course it isn't quoted"), attempt to include nested data, leave out the opening and closing [] (because people will grep a file with one array per line to filter a json file; that is no way robust, but people will do it, anyways) The main advantage of json vs csv is that there is only one json, but quite a few different variant of csv. That is a huge advantage, but csv is engrained; there are many, many tools that handle csv but not json. And the "if you control input and output" case is not the interesting or the problematic one.
keeperofdakeys 12y ago
If you have simple data, why use something as complicated as JSON? For a recent project, I had a simple CSV file with an int and float per row; using JSON would probably double the datasize. I used a simple string.split(",") for the javascript decoder, because I control the data, and know it's safe. I don't need another javascript library (I'd probably do differently if I had a standard library, not a hodge-podge of scripts). Sometimes, simplicity is better for everyone.
- wiredfool 12y ago
  Why not use both? I've had to() parse json embedded in a field in a csv file. Unquoted of course. Until I explained to the other developer just how stupid that was.
xxs 12y ago
The idea that JSON is the substitute made me chuckle. JSON is more verbose to boot. CSV is a poor format but JSON is not panacea, actually personally I'd never use it for anything that's not web (browser) related.
chrismcb 12y ago
Because Excel doesn't export JSON. Because if you have a table, offering comma delimited field is easy. But really, people you work with give you csv files, and you don't have a choice.

Why are people using CSV when better (and less fuzzily defined) solutions exist, such as JSON?

minimaxir 12y ago

CSV is far, far more ubiquitous and much more usable in non-web settings. (e.g. desktop data analysis programs)

deleted 12y ago
[deleted]
jgalt212 12y ago
true, and in those settings you largely don't see situations that trip up naive parsers such as newlines or delimiters inside fields.

jiggy2011 12y ago

Importing into excel is probably a big reason.

mantrax5 12y ago
If Excel compatibility is the goal, one should use libraries that read and produce Excel files. CSV is bullshit, it's not good for anything except scenarios where you control both the export process, and the parser (so you know what delimiter is used and so on).
- minimaxir 12y ago
  The same could be said about exporting to JSON. (the JSON code on some of the big APIs does not parse intelligently and I have to spend a lot of time fixing it.)
  - deleted 12y ago
    [deleted]
- mrweasel 12y ago
  CSV files are MUCH easier to search and inspect using tools like grep and less. It's the accounting people that want's Excel, but as a developer CSV is easier and more flexible. We process csv files containing payment information, export csv as product feeds for perhaps 10 different partners. It works, it's faster than XML or JSON and is easy for non-technical people to inspect using tools like Excel. Most cvs data exports are not something that's made up on the fly every day, it batch jobs that create the same type of CSV file each and every time. CSV is alright. XML is bullshit.
  - Roboprog 12y ago
    CSV is much simpler when the records are all of one type. It gets debatable which (XML vs CSV) is simpler when you get multiple record types dumping a hierarchical data structure. Obviously, you can make CSV files work for alternating record types, in the same way that the old mainframe files used to work with multiple record types in a file, with a type descriminator field in a know place, usually field 1, of each record. But it starts to get cluttered.

jasode 12y ago

In addition to aforementioned import/export data interop with MS Excel, there are tons of legacy systems (mainframes, etc) that import/export csv but not XML or JSON. The csv format is everywhere and will continue to be with us for decades. People will always look for a quality library (in whatever new programming language) that handles all tricky edge cases. A few months ago, I was trying to get some bulk data into ebay's proprietary TurboLister[1] program. Guess what, it can import csv but not JSON. SQLite[2] can import csv but not JSON. Google's terabyte ngram dataset[3] is csv (tsv) instead of JSON. I'm glad it's not JSON because it would have required extra disk space. ... plus tons of other real-world csv examples out in the wild. Unfortunately, the csv format is very easy for programs to write but it's very difficult for programs to properly read because of the tricky parsing. [1] http://pages.ebay.com/sellerinformation/sellingresources/turbolister.html http://pages.ebay.com/sellerinformation/sellingresources/tur... [2] http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles [3] http://storage.googleapis.com/books/ngrams/books/datasetsv2.html http://storage.googleapis.com/books/ngrams/books/datasetsv2....

jzwinck 12y ago

If your data are rectangular and you care about performance, CSV is better than JSON just because it avoids repetitive key names everywhere. Then again, if your data are rectangular and you really care about performance, you would not use any of these (you might use HDF5, which has support in many programming languages and will destroy the others in terms of speed).

lukeschlather 12y ago
JSON is almost a subset of CSV, with the understanding that you have to wrap every line in [], the document in [], and every field must be quoted. (And JSON doesn't have built-in support for headings, so you need to write a little loop instead of the library building a hash for you.) So no, if you control input and output, JSON is still easier to use than CSV, and just as performant. JSON stores straight arrays just fine. It's not the format's fault so many people choose to store hashes with it.
- Someone 12y ago
  ...and quotes inside fields must be escaped with a backslash, newlines replaced by \n, etc. ...and, the moment you declare you handle json, people will send non-string data ("that is a number, of course it isn't quoted"), attempt to include nested data, leave out the opening and closing [] (because people will grep a file with one array per line to filter a json file; that is no way robust, but people will do it, anyways) The main advantage of json vs csv is that there is only one json, but quite a few different variant of csv. That is a huge advantage, but csv is engrained; there are many, many tools that handle csv but not json. And the "if you control input and output" case is not the interesting or the problematic one.

keeperofdakeys 12y ago

If you have simple data, why use something as complicated as JSON? For a recent project, I had a simple CSV file with an int and float per row; using JSON would probably double the datasize. I used a simple string.split(",") for the javascript decoder, because I control the data, and know it's safe. I don't need another javascript library (I'd probably do differently if I had a standard library, not a hodge-podge of scripts). Sometimes, simplicity is better for everyone.

wiredfool 12y ago
Why not use both? I've had to() parse json embedded in a field in a csv file. Unquoted of course. Until I explained to the other developer just how stupid that was.

xxs 12y ago

The idea that JSON is the substitute made me chuckle. JSON is more verbose to boot. CSV is a poor format but JSON is not panacea, actually personally I'd never use it for anything that's not web (browser) related.

chrismcb 12y ago

Because Excel doesn't export JSON. Because if you have a table, offering comma delimited field is easy. But really, people you work with give you csv files, and you don't have a choice.