5 ms·
The thing is, I need a hefty library to write parquet. I can write a CSV with a couple fprintf statements and a loop. A parser isn't much harder if we can make
by Sanzig 5y ago
The thing is, I need a hefty library to write parquet. I can write a CSV with a couple fprintf statements and a loop. A parser isn't much harder if we can make assumptions about CSV dialect ahead of time.
If I'm trying to make an exportable format for a data logger with an SD card running on an ARM microcontroller, it doesn't get much easier than CSV. Sure, I could save space by rolling my own binary format, but then I have to provide a PC application to read it (and realistically, the user is probably just going to want that application to dump to a CSV anyway!).
I agree that for many use cases there are much better alternatives, but one of the reasons CSV is so popular is because it's so simple. It shouldn't be used for multi-gigabyte datasets, but for many simple use cases it works great.
- dingosity 5y agono. you can't write CSV with a couple of fprintf statements in a loop. If your data has a comma or a double-quote in it, the field needs to be surrounded by double quotes (and the double quote needs to be doubled). Ditto if you have leading or trailing spaces. that is... if you're following the RFC 4180 guidance for CSV. microsoft has another internal standard that occasionally gets referenced.