3 ms·

This is disingenuous. There isn't really a CSV standard that defines the precise grammar of CSV. But, suppose there was such a standard, then som of the proble

This is disingenuous. There isn't really a CSV standard that defines the precise grammar of CSV. But, suppose there was such a standard, then som of the problems you've described would become non-problems (eg. support across multiple tools). Potentially, if such a standard disallowed linebreaks, it would also make the use of grep easier. I actually really wish there was such a standard... Your other assertion about "squishy" formats is also... well, not really true. The format you listed became popular due to the data they encoded, not because of their quality. It's very plausible that a better format may exist, and they are probably out there. The problem is that we set the bar too low with popular formats, and that they often seem easy to implement, which results in a lot of junk formats floating around.

imtringued 4y ago
>There isn't really a CSV standard that defines the precise grammar of CSV. If you don't implement the pseudostandard that pretty much every library uses then whatever format you have can be called CSV but you can bet that it will be compatible with nothing else as you have implemented a bespoke file format that has no guarantee of compatibility. If it works at all it is an accident and it probably shouldn't.
Kinrany 4y ago
One problem that needs to be solved and standardized is escaping. It's extremely common for text formats to store text. Some characters always need to be escaped, but the set of characters is always unique to the host format.
jiggawatts 4y ago
> There isn't really a CSV standard that defines the precise grammar of CSV. Did you read the link going to a page literally titled: "Parsing JSON is a Minefield."? JSON has a "precise" grammar intended to be human readable. The end result is a mess, vulnerable to attacks due to dissimilarities between different implementations. Google put in significant engineering effort into "Ryu", a parsing library for double-precision floating point numbers: https://github.com/ulfjack/ryu https://github.com/ulfjack/ryu Why bother, you ask? Why would anyone bother to make floating point number parsing super efficient? JSON.
- fuklief 4y ago
  > Google put in significant engineering effort into "Ryu", a parsing library for double-precision floating point numbers: https://github.com/ulfjack/ryu https://github.com/ulfjack/ryu It's not a parsing library, but a printing one, i.e., double -> string. https://github.com/fastfloat/fast_float https://github.com/fastfloat/fast_float is a parsing library, i.e., string -> double, not by Google though, but was indeed motivated by parsing JSON fast https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/ https://lemire.me/blog/2020/03/10/fast-float-parsing-in-prac...
- crabbone 4y ago
  > Did you read the link going to a page literally titled: "Parsing JSON is a Minefield."? No, I didn't. But... why is this relevant to CSV? I think I might have read an article with a similar name before. And, in general, am not a big fan of JSON, but I didn't think this link contained a standard for CSV... Also, JSON has a defective grammar, with bad semantics, so, yeah, having a standard didn't save it... but it could have saved it. Not having a standard would've been even worse.
  - imtringued 4y ago
    JSON is well defined but hard to parse. CSV is not well defined but you still have to parse and print in such a way that is compatible with every other CSV implementation otherwise what is the point? Use xlsx Spreadsheets instead, those actually work even if they are a bit bloated.

imtringued 4y ago

>There isn't really a CSV standard that defines the precise grammar of CSV. If you don't implement the pseudostandard that pretty much every library uses then whatever format you have can be called CSV but you can bet that it will be compatible with nothing else as you have implemented a bespoke file format that has no guarantee of compatibility. If it works at all it is an accident and it probably shouldn't.

Kinrany 4y ago

One problem that needs to be solved and standardized is escaping. It's extremely common for text formats to store text. Some characters always need to be escaped, but the set of characters is always unique to the host format.

jiggawatts 4y ago

> There isn't really a CSV standard that defines the precise grammar of CSV. Did you read the link going to a page literally titled: "Parsing JSON is a Minefield."? JSON has a "precise" grammar intended to be human readable. The end result is a mess, vulnerable to attacks due to dissimilarities between different implementations. Google put in significant engineering effort into "Ryu", a parsing library for double-precision floating point numbers: https://github.com/ulfjack/ryu https://github.com/ulfjack/ryu Why bother, you ask? Why would anyone bother to make floating point number parsing super efficient? JSON.

fuklief 4y ago
> Google put in significant engineering effort into "Ryu", a parsing library for double-precision floating point numbers: https://github.com/ulfjack/ryu https://github.com/ulfjack/ryu It's not a parsing library, but a printing one, i.e., double -> string. https://github.com/fastfloat/fast_float https://github.com/fastfloat/fast_float is a parsing library, i.e., string -> double, not by Google though, but was indeed motivated by parsing JSON fast https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/ https://lemire.me/blog/2020/03/10/fast-float-parsing-in-prac...
crabbone 4y ago
> Did you read the link going to a page literally titled: "Parsing JSON is a Minefield."? No, I didn't. But... why is this relevant to CSV? I think I might have read an article with a similar name before. And, in general, am not a big fan of JSON, but I didn't think this link contained a standard for CSV... Also, JSON has a defective grammar, with bad semantics, so, yeah, having a standard didn't save it... but it could have saved it. Not having a standard would've been even worse.
- imtringued 4y ago
  JSON is well defined but hard to parse. CSV is not well defined but you still have to parse and print in such a way that is compatible with every other CSV implementation otherwise what is the point? Use xlsx Spreadsheets instead, those actually work even if they are a bit bloated.