2 ms·
I don't agree with the author's thesis, but you're wrong. 1. It is a very common use-case to load CSV data generated by an opaque system which did not specify
by einpoklum 5y ago
I don't agree with the author's thesis, but you're wrong.
1. It is a very common use-case to load CSV data generated by an opaque system which did not specify the format exactly to you.
2. CSV files are _difficult_ to process I: Fields are of variable width, which is not specified elsewhere. So you can't start processing different records in parallel, since you have to process previous lines to figure out where they end. You can't even go by some clear line termination pattern, since the quoting and escaping rules, and the inconsistencies the author mentions, make it so that you may not be able to determine with certainty that what you're reading is an actual end-of-record or end-of-field. Maybe it's all just a dream^H^H^H^H^H quoted string?
3. CSV files are _difficult_ to process II: For the same reasons as the above, it is non-trivial and time consuming to recover from CSV generation mistakes, corruptions during transmission, or missing parts of the data.
However - if you get some guarantees and abot field width and about the non-use of field and record separators in quoted strings, then your life becomes much easier.