4 ms·
> The CSV format itself is notoriously inconsistent So what? The format is so simple that the inconsistencies doesn't even matter and hell the format can be in
by TeeMassive 5y ago
> The CSV format itself is notoriously inconsistent
So what? The format is so simple that the inconsistencies doesn't even matter and hell the format can be inferred from the data and that can even be automated. Unlike JSON/XML, most software can automatically make sense of CSV; JSON needs a developer to understand it to transform it into workable data.
> CSVs often begin life as exported spreadsheets or table dumps from legacy databases, and often end life as a pile of undifferentiated files in a data lake, awaiting the restoration of their precious metadata so they can be organized and mined for insights.
CSVs are the de facto way to export spreadsheets and if you have to do this then the problem are that you are still using spreadsheets in your data flow, not CSVs. Same goes for legacy systems.
Also CSVs comes from places where having more sophisticated data serializers is near impossible and the time to code, compute, parse and analyze CSVs is trivial compared to any other format out there.
> I’ve spent many years battling CSVs in various capacities. As an engineer at Trifacta, a leading data preparation tool vendor, I saw numerous customers struggling under the burden of myriad CSVs that were the product of even more numerous Excel spreadsheet exports and legacy database dumps. As an engineering leader in biotech, my team struggled to ingest CSVs from research institutions and hospital networks who gave us sample data as CSVs more often than not.
Of course if you are working in biotech then you'll work with scientists (horrible coders most of the time) and legacy systems because it's a domain that's very adverse to change that might break stuff. And it's not the CSV format's fault if it's used as the wrong tool for the job.
> Another big drawback of CSV is its almost complete lack of metadata.
Again, emphasis on the right tool for the job. For simple tasks metada are not needed.
- Stratoscope 5y agoYou mistakenly replied on the wrong thread. Here's the one you meant to reply on: https://news.ycombinator.com/item?id=28221654 https://news.ycombinator.com/item?id=28221654 Your quote "The CSV format itself is notoriously inconsistent" comes from this page which the thread discusses: https://www.bitsondisk.com/writing/2021/retire-the-csv/ https://www.bitsondisk.com/writing/2021/retire-the-csv/
- TeeMassive 5y agoI am not a smart man (sometimes)
- Stratoscope 5y agoI agree, and disagree. You are smart, just not at every moment. Same as me. If anyone out there has never clicked on the wrong button, let them cast the first stone.