3 ms·
CSV is a poorly defined format. It doesn't have a standard. No wonder it screws up when you start varying character encoding. The first practical lesson of CSV
by user5994461 10y ago
CSV is a poorly defined format. It doesn't have a standard. No wonder it screws up when you start varying character encoding.
The first practical lesson of CSV is CSV WILL fail miserably when changing character encoding, using non ASCII characters, having quoted string, varying end of line characters...
- hermitdev 10y agoIndeed. Differences in encoding or presence/lack of a BOM in UTF8 are a small set of issues. I've dealt with financial data from very large public banks and vendors that are horrible about vending proper CSV data. Either impromper or unescaped quotes (or delimiters). Takes weeks to get the data corrected, ao you manually correct the data or write a custom parser to try and navigate around it. One of my faves was a vendor that didn't escape the legal name of a company, so "Microsoft,Inc" would be 2 fields, rending most of that recors gibberish. I generally agree with the idiom "If you write your own cav parser, you're doing it wrong", but sometimes you have no choice to work around the idiocy of others.
- wolfgke 10y ago> CSV is a poorly defined format. It doesn't have a standard. Wrong: CSV is standardized in RFC 4180: > https://tools.ietf.org/html/rfc4180 https://tools.ietf.org/html/rfc4180
- chinpokomon 10y agoEx post facto. It's been a "standard" long before the RFC, so there are many CSV documents which need to be read but don't confirm to the RFC spec. You should create a CSV writer that conforms, but your reader needs to support much more. Then if you save back to the RFC spec, the document might break other downstream readers which were written long ago and which don't follow the new RFC spec. It's an imperfect file format.