9 ms·
> It’s that libraries and tools to parse CSV tend to suck. Whereas JSON is the lingua franca of data. This isn't the case. An incredible amount of effort and
by derriz 2y ago
> It’s that libraries and tools to parse CSV tend to suck. Whereas JSON is the lingua franca of data.
This isn't the case. An incredible amount of effort and ingenuity has gone into CSV parsing because of its ubiquity. Despite the lack of any sort of specification, it's easily the most widely supported data format in existence in terms of tools and language support.
- hajile 2y agoCan you point me to a language with any significant number of users that does NOT have a JSON library? I went looking at some of the more niche languages like Prolog, COBOL, RPG, APL, Eiffel, Maple, MATLAB, tcl, and a few others. All of these and more had JSON libraries (most had one baked into the standard library). The exceptions I found (though I didn't look too far) were: Bash (use jq with it), J (an APL variant), Scratch (not exposed to users, but scratch code itself is encoded in JSON), and Forth (I could find implementations, but it's very hard to pin down forth dialects).
- derriz 2y agoI made no claim about JSON libraries. I contested the claim that "CSV libraries and tools suck". They do not.
- jstanley 2y agoYou claimed that CSV is "easily the most widely supported data format in existence in terms of tools and language support", which is a claim that CSV is better supported than JSON, which is a claim that JSON support is lacking.
- shawabawa3 2y agoCan you import .jsonl files into Google sheets or excel natively?
- freehorse 2y agoImporting csvs in excel can be a huge pain due to how excel handles localisation. It can basically alter your data if you are not mindful about that, and I have seen it happening too many times.
- codetrotter 2y agoFor example: Scientists rename genes because Microsoft Excel reads them as dates (2020) https://www.reddit.com/r/programming/comments/i57czq/scientists_rename_genes_because_microsoft_excel/ https://www.reddit.com/r/programming/comments/i57czq/scienti...
- TRiG_Ireland 2y agoI was so glad of that story. It gave me something to point to to get my boss off my back.
- phkahler 2y agoBut it handles it better than Json.
- freehorse 2y agoDepends on what you mean by "better". I would rather software not handle a piece of data at all, than handle it erroneously and changing the data without me realising and thus causing all sorts of issues after.
- FridgeSeal 2y agoCSV tooling has had to invest enormous amounts of effort to make a fragile, under-specified format half-useful. I would call it ubiquitous, I would call the tooling that we’ve built around it “impressive” but I would by no means call any of it “good”. I do not miss dealing with csv files in the slightest.
- dylan604 2y agoMicrosoft Windows has had to invest enormous amounts... Apple macOS has had to invest enormous amounts... Pick your distro of Linux has had to invest enormous amounts... None of them a perfect and any number of valid complaints can be said about any of them. None of the complaints make any of the things useless. Everyone has workarounds. Hell, JSON has had to invest enormous amounts of effort...
- FridgeSeal 2y agoI guess the point is that I can take a generic json parser and point it at just about any JSON I get my hands on, and have close to no issues parsing it. Want to do the same with csv? Good luck. Delimiter? Configurable. Encoding? Configurable. Misplaced comma? No parse in JSON, in csv: might still parse, but is now semantically incorrect and you possibly won’t know until it’s too late, depending on your parser. The list goes on.
- pbrumm 2y agoHere is a quick test The table of contents points to a single Json object that is 20ish gb compressed https://www.anthem.com/machine-readable-file/search/ https://www.anthem.com/machine-readable-file/search/ All stock libs will fail
- nukem222 2y ago[flagged]
- freehorse 2y ago> CSV tooling has had [...] to make a fragile, under-specified format half-useful You get this backwards. Tabular structured data to store are ubiquitous. Text as a file format is also ubiquitous because it is accessible. The only actual decisions are about whether to encode your variables as rows or columns, what is the delimiter, and other rules such as escaping etc. Vars as columns makes sense because it makes appending easier. There is a bunch of stuff that can be used for delimeters, commas being the most common, none is perfect. But from this point onwards, decisions do not really matter, and "CSV" basically covers everything from now on. "CSV" is basically what comes naturally when you have tabular datasets and want to store them in text. CSV tooling is developed because there is a need for this way of formatting data. Whether CSV is "good" or "ugly" or whatever is irrelevant, handling data is complicated as much as the world itself is. The alternatives are either not structuring/storing the data in a tabular manner, or non-text (eg binary) formats. These alternative exist and are useful in their own right, but don't solve the same problems.
- niccl 2y agoExcel. Before you dismiss it as 'not a language, people have argued that it is. And you can definitely program stuff in it, and so that surely makes it a language
- squeaky-clean 2y agoExcel can import and parse JSON, it's under the "Get Data" header. It doesn't have a direct GUI way to export to JSON, but it takes just a few lines in Office Scripts. You can even use embedded TypeScript to call JSON.stringify.
- otikik 2y agoMeanwhile, Excel exports to CSV as “semicolon separated values” depending on your OS locale
- matthewmacleod 2y agoAlbeit for fairly justifiable reasons
- notpushkin 2y agoJustifiable how?
- LoganDark 2y agoDecimal separators being commas in some locales?
- ideamotor 2y agoCommas are commonly used in text, too.
- susam 2y agoClearly they should have gone with BEL as the delimiter. printf "alice\007london\007uk\nbob\007paris\007france\n" > data.bsv I'm hoping no reasonable person would ever use BEL as punctuation or decimal separator.
- kmoser 2y agoOn the off chance you're not being facetious, why not ASCII 0 as a delimiter? (This is a rhetorical question.)
- defrost 2y agoThere are some decent arguments for BEL over NUL, however given you posed that as a rhetorical question I feel I can say little other than ding! ding! ding! winner winner, chicken dinner! Although BEL would drive me up the wall if I broke out any of my old TTY hardware.
- kentm 2y agoI’ve found that the number of parsers that don’t handle multiline records is pretty high though.
- consteval 2y agoIt's widely, but inconsistently, supported. The behavior of importers varies a lot, which is generally not the case for JSON.
- autoexec 2y ago> it's easily the most widely supported data format in existence in terms of tools and language support. Even better, the majority of the time I write/read CSV these days I don't need to use a library or tools at all. It'd be overkill. CSV libraries are best saved for when you're dealing with random CSV files (especially from multiple sources) since the library will handle the minor differences/issues that can pop up in the wild.
- nukem222 2y ago> An incredible amount of effort and ingenuity has gone into CSV parsing because of its ubiquity. Yea and it's still a partially-parseable shit show with guessed values. But we can and could have and should have done better by simply defining a format to use.
- recursive 2y ago> Despite the lack of any sort of specification People keep saying this but RFC 4180 exists.