3 ms·
What isn't fun about CSV is quickly written parsers and serializers repeatedly making the common mistake of not handling, or badly handling, quoting. For a lon
by notatallshaw 2y ago
What isn't fun about CSV is quickly written parsers and serializers repeatedly making the common mistake of not handling, or badly handling, quoting.
For a long time I was very wary of CSV until I learnt Python and started using it's excellent csv standard library module.
- goatlover 2y agoWhy not Pandas, since you're working with tabular data anyway?
- FridgeSeal 2y agoBecause maybe they’re not doing something column oriented? Because it has a notoriously finicky API? A dozen other reasons?
- notatallshaw 2y agoWhen I first started, installing packages which required compiling native code on either my work Windows machine and the old Unix servers was not easy. So I largely stuck to the Python standard library where I could, and most of the operations I had at the time did not require data analysis on a server, that was mostly done in a database. Often the job was validating and transforming the data to then insert it into a database. As the Python packaging ecosystem matured and I found I could easily use Pandas everywhere it just wasn't my first thing I'd reach to. And occasionally it'd be very helpful to iterate through them with the csv module, only taking a few MBs of memory, vs. loading the entire dataset into memory with Pandas.
- codeulike 2y agoThats true, in recent years its been less of a disaster with lots of good csv libraries for various languages. In the 90s csv was a constant footgun, perhaps thats why they went crazy and came up with XML
- Macha 2y agoEven widely used libraries that you might expect get it right, don't. (Like Spark, which uses Java style backslash escaping)