3 ms·
I wish this was a joke. I'm always trying to convince data scientists with a foot in the open source world that their life will be so much better if they use p
by PaulHoule 2y ago
I wish this was a joke. I'm always trying to convince data scientists with a foot in the open source world that their life will be so much better if they use parquet or Stata or Excel or any other kind of file but CSV.
On top of all the problems people mention here involving the precise definition of the format and quoting, it's outright shocking how long it takes to parse ASCII numbers into floating point. One thing that stuck with me from grad school was that you could do a huge number of FLOPS on a matrix in the time it would take to serialize and deserialize it to aSCII.
- jcattle 2y agoWhat advantages does excel give you over CSV?
- PaulHoule 2y agoAccurate data typing (never confuse a string with a number) Maybe be circular but: always loads correctly into Excel, if you want to load into a spreadsheet you can add text formatting and even formulas, checkboxes and stuff which can be a lot of fun.
- kec 2y agoThat is very much not true, Excel does type coercion, especially around things that happen to look like dates: https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates https://www.theverge.com/2020/8/6/21355674/human-genes-renam...
- PaulHoule 2y agoExcel does that type coercion if you import from CSV. If you export pandas data to XLSX it adds proper type information and then it imports properly into Excel and you avoid those problems.
- deleted 2y ago[deleted]