5 ms·

CSV would be great if there were one CSV, and if you absolutely guarantee that nobody has "improved" the CSV by editing it with a spreadsheet program (including

by msla 3y ago

CSV would be great if there were one CSV, and if you absolutely guarantee that nobody has "improved" the CSV by editing it with a spreadsheet program (including opening it with a spreadsheet program) or their own deft little fingers in a text editor. For example: "Look, this contains \"quotes\"!",012345 Or: "Look, this contains ""quotes""!",012345 Or, for some degenerate examples: "Look, this contains "quotes"!",012345 Or: Look, this contains "quotes"!,012345 Or the spoor of a spreadsheet: "Look, this contains ""quotes""!",12345 Theoretically, JSON isn't immune to being hand-hacked into a semi-coherent mess. In practice, people don't seem to do that to JSON files, at least not that I've seen. Ditto number problems, in that in JSON, serial numbers and such tend to be strings instead of integers a "helpful" application can lop a few zeroes off of.

AshamedCaptain 3y ago
> CSV would be great if there were one CSV, and if you absolutely guarantee that nobody has "improved" the CSV by editing it with a spreadsheet program (including opening it with a spreadsheet program) Practically no formats actually pass those rules. Even plain text is bound to be "improved" by text editors frequently (uniformation of line endings, removal of data not in a known encoding, UTF BOM, UTF normalization, etc.) Just don't do that.
- taway1237 3y ago
  Just don't do what, edit your data? The problem is all JSON implementations agree what a properly encoded JSON file look like, and it's not true for CSV. Arbitrary CSV is not, in general, round-trippable.
  - gkbrk 3y ago
    > all JSON implementations agree what a properly encoded JSON file look like You clearly haven't had to deal with tools that cannot read and write a JSON file without messing up and truncating large numbers.
    - efreak 3y ago
      Json is JavaScript Object Notation. Javascript itself doesn't large numbers like 2^65 or accurate numbers like 2^55+75. The json spec doesn't limit numbers to 2^51 or 2^64, but also doesn't guarantee that they will be supported.
nuc1e0n 3y ago
Stick to files conforming to RFC-4180 then
ianburrell 3y ago
JSONL should replace CSV. It is standardized and the escapes mostly well specified. It is effectively CSV with "[" and "]" surrounding lines. Regular JSON would work fine for static file, and make Schema and links (JSON-LD) possible. But then the file could be any structure. JSONL works better for systems that assume line-based records, and are more likely to have consistent, simple records.
nly 3y ago
I mean in practice you open up the CSV file, figure out the escaping rules, change one line of your code, and get on with your life.
- stinos 3y ago
  You're lucky. In practice, you open the file a disappointed customer sends you because your software doesn't parse it. You resist yelling at whomever decided to ever do CSV support in your software because you know what it's like. But the customer is important, so then either write a one-off converter for them to a more proper CSV format, or you add - not change - several lines of code in mainline to be able to also parse customer's format. God forbid maybe also write. Repeat that a couple of times. See the mess increasing. Decide to use a so-called battle-tested CSV parser after all, only to find out it doesn't pass all tests. Weep. Especially because at the sametime you also support a binary format which is 100 times faster to read and write and has no issues whatsoever. I'm not blind for the benefits of CSV, but oh boy it would be nice if it would be, for starters, actually comma-separated.

by msla 3y ago

AshamedCaptain 3y ago

> CSV would be great if there were one CSV, and if you absolutely guarantee that nobody has "improved" the CSV by editing it with a spreadsheet program (including opening it with a spreadsheet program) Practically no formats actually pass those rules. Even plain text is bound to be "improved" by text editors frequently (uniformation of line endings, removal of data not in a known encoding, UTF BOM, UTF normalization, etc.) Just don't do that.

taway1237 3y ago
Just don't do what, edit your data? The problem is all JSON implementations agree what a properly encoded JSON file look like, and it's not true for CSV. Arbitrary CSV is not, in general, round-trippable.
- gkbrk 3y ago
  > all JSON implementations agree what a properly encoded JSON file look like You clearly haven't had to deal with tools that cannot read and write a JSON file without messing up and truncating large numbers.
  - efreak 3y ago
    Json is JavaScript Object Notation. Javascript itself doesn't large numbers like 2^65 or accurate numbers like 2^55+75. The json spec doesn't limit numbers to 2^51 or 2^64, but also doesn't guarantee that they will be supported.

nuc1e0n 3y ago

Stick to files conforming to RFC-4180 then

ianburrell 3y ago

JSONL should replace CSV. It is standardized and the escapes mostly well specified. It is effectively CSV with "[" and "]" surrounding lines. Regular JSON would work fine for static file, and make Schema and links (JSON-LD) possible. But then the file could be any structure. JSONL works better for systems that assume line-based records, and are more likely to have consistent, simple records.

nly 3y ago

I mean in practice you open up the CSV file, figure out the escaping rules, change one line of your code, and get on with your life.

stinos 3y ago
You're lucky. In practice, you open the file a disappointed customer sends you because your software doesn't parse it. You resist yelling at whomever decided to ever do CSV support in your software because you know what it's like. But the customer is important, so then either write a one-off converter for them to a more proper CSV format, or you add - not change - several lines of code in mainline to be able to also parse customer's format. God forbid maybe also write. Repeat that a couple of times. See the mess increasing. Decide to use a so-called battle-tested CSV parser after all, only to find out it doesn't pass all tests. Weep. Especially because at the sametime you also support a binary format which is 100 times faster to read and write and has no issues whatsoever. I'm not blind for the benefits of CSV, but oh boy it would be nice if it would be, for starters, actually comma-separated.