3 ms·
1. I wouldn't bet Avro, Parquet, Arrow, or other formats that require a library to parse are going to _entirely_ replace CSV. Those are broader in scope, but ca
by hcarvalhoalves 5y ago
1. I wouldn't bet Avro, Parquet, Arrow, or other formats that require a library to parse are going to _entirely_ replace CSV. Those are broader in scope, but can't say are "replacements" if they don't share the simplicity of plain text data following a few conventions – much like how PDF is not a "replacement" for Markdown, but enables higher-fidelity content.
2. I would feel safer storing longer-term data in CSV than in a binary format w/ a complicated spec. Having to make sense of compressed, columnar padded data sounds worse than parsing CSV.
3. Although it's easy to point at corner cases, I don't remember the last time I couldn't figure out how to parse a file because of inconsistent quoting or exotic char encoding – and I've spent a good amount of the past 13 years exchanging CSVs and TSVs w/ 3rd parties full of horrible legacy. Asking those 3rd parties to send me an Avro/Parquet/whatever file would've made the project fail or take 10x longer.
There's a reason why CSV stuck around so long, and the alternatives make different trade-offs but miss the pros.