4 ms·

I made no claim about JSON libraries. I contested the claim that "CSV libraries and tools suck". They do not.

by derriz 2y ago

jstanley 2y ago
You claimed that CSV is "easily the most widely supported data format in existence in terms of tools and language support", which is a claim that CSV is better supported than JSON, which is a claim that JSON support is lacking.
- shawabawa3 2y ago
  Can you import .jsonl files into Google sheets or excel natively?
  - freehorse 2y ago
    Importing csvs in excel can be a huge pain due to how excel handles localisation. It can basically alter your data if you are not mindful about that, and I have seen it happening too many times.
    - codetrotter 2y ago
      For example: Scientists rename genes because Microsoft Excel reads them as dates (2020) https://www.reddit.com/r/programming/comments/i57czq/scientists_rename_genes_because_microsoft_excel/ https://www.reddit.com/r/programming/comments/i57czq/scienti...
      - TRiG_Ireland 2y ago
        I was so glad of that story. It gave me something to point to to get my boss off my back.
    - phkahler 2y ago
      But it handles it better than Json.
      - freehorse 2y ago
        Depends on what you mean by "better". I would rather software not handle a piece of data at all, than handle it erroneously and changing the data without me realising and thus causing all sorts of issues after.
        d0mine 2y ago
        In practice, web browsers accept the tag soup that is sometimes called html and strict xml-based formats failed.
        afiori 2y ago
        The browser are not a database (unlike excel). Modifying data before showing it is reversible, modifying it before storing it is not.
    - xnx 2y ago
      Excel dropping leading zeros (as in ZIP codes) was a crazy design decision that has certainly cost many lifetimes of person-hours.
      - disgruntledphd2 2y ago
        And forcing 16+ digits to be floats, destroying information.
      - freehorse 2y ago
        Yeah have had similar struggles with social security numbers.
FridgeSeal 2y ago
CSV tooling has had to invest enormous amounts of effort to make a fragile, under-specified format half-useful. I would call it ubiquitous, I would call the tooling that we’ve built around it “impressive” but I would by no means call any of it “good”. I do not miss dealing with csv files in the slightest.
- dylan604 2y ago
  Microsoft Windows has had to invest enormous amounts... Apple macOS has had to invest enormous amounts... Pick your distro of Linux has had to invest enormous amounts... None of them a perfect and any number of valid complaints can be said about any of them. None of the complaints make any of the things useless. Everyone has workarounds. Hell, JSON has had to invest enormous amounts of effort...
  - FridgeSeal 2y ago
    I guess the point is that I can take a generic json parser and point it at just about any JSON I get my hands on, and have close to no issues parsing it. Want to do the same with csv? Good luck. Delimiter? Configurable. Encoding? Configurable. Misplaced comma? No parse in JSON, in csv: might still parse, but is now semantically incorrect and you possibly won’t know until it’s too late, depending on your parser. The list goes on.
    - pbrumm 2y ago
      Here is a quick test The table of contents points to a single Json object that is 20ish gb compressed https://www.anthem.com/machine-readable-file/search/ https://www.anthem.com/machine-readable-file/search/ All stock libs will fail
  - nukem222 2y ago
    [flagged]
- freehorse 2y ago
  > CSV tooling has had [...] to make a fragile, under-specified format half-useful You get this backwards. Tabular structured data to store are ubiquitous. Text as a file format is also ubiquitous because it is accessible. The only actual decisions are about whether to encode your variables as rows or columns, what is the delimiter, and other rules such as escaping etc. Vars as columns makes sense because it makes appending easier. There is a bunch of stuff that can be used for delimeters, commas being the most common, none is perfect. But from this point onwards, decisions do not really matter, and "CSV" basically covers everything from now on. "CSV" is basically what comes naturally when you have tabular datasets and want to store them in text. CSV tooling is developed because there is a need for this way of formatting data. Whether CSV is "good" or "ugly" or whatever is irrelevant, handling data is complicated as much as the world itself is. The alternatives are either not structuring/storing the data in a tabular manner, or non-text (eg binary) formats. These alternative exist and are useful in their own right, but don't solve the same problems.

I made no claim about JSON libraries. I contested the claim that "CSV libraries and tools suck". They do not.

by derriz 2y ago

I made no claim about JSON libraries. I contested the claim that "CSV libraries and tools suck". They do not.

You claimed that CSV is "easily the most widely supported data format in existence in terms of tools and language support", which is a claim that CSV is better supported than JSON, which is a claim that JSON support is lacking.

shawabawa3 2y ago
Can you import .jsonl files into Google sheets or excel natively?
- freehorse 2y ago
  Importing csvs in excel can be a huge pain due to how excel handles localisation. It can basically alter your data if you are not mindful about that, and I have seen it happening too many times.
  - codetrotter 2y ago
    For example: Scientists rename genes because Microsoft Excel reads them as dates (2020) https://www.reddit.com/r/programming/comments/i57czq/scientists_rename_genes_because_microsoft_excel/ https://www.reddit.com/r/programming/comments/i57czq/scienti...
    - TRiG_Ireland 2y ago
      I was so glad of that story. It gave me something to point to to get my boss off my back.
  - phkahler 2y ago
    But it handles it better than Json.
    - freehorse 2y ago
      Depends on what you mean by "better". I would rather software not handle a piece of data at all, than handle it erroneously and changing the data without me realising and thus causing all sorts of issues after.
      - d0mine 2y ago
        In practice, web browsers accept the tag soup that is sometimes called html and strict xml-based formats failed.
        afiori 2y ago
        The browser are not a database (unlike excel). Modifying data before showing it is reversible, modifying it before storing it is not.
  - xnx 2y ago
    Excel dropping leading zeros (as in ZIP codes) was a crazy design decision that has certainly cost many lifetimes of person-hours.
    - disgruntledphd2 2y ago
      And forcing 16+ digits to be floats, destroying information.
    - freehorse 2y ago
      Yeah have had similar struggles with social security numbers.

FridgeSeal 2y ago

CSV tooling has had to invest enormous amounts of effort to make a fragile, under-specified format half-useful. I would call it ubiquitous, I would call the tooling that we’ve built around it “impressive” but I would by no means call any of it “good”. I do not miss dealing with csv files in the slightest.

dylan604 2y ago
Microsoft Windows has had to invest enormous amounts... Apple macOS has had to invest enormous amounts... Pick your distro of Linux has had to invest enormous amounts... None of them a perfect and any number of valid complaints can be said about any of them. None of the complaints make any of the things useless. Everyone has workarounds. Hell, JSON has had to invest enormous amounts of effort...
- FridgeSeal 2y ago
  I guess the point is that I can take a generic json parser and point it at just about any JSON I get my hands on, and have close to no issues parsing it. Want to do the same with csv? Good luck. Delimiter? Configurable. Encoding? Configurable. Misplaced comma? No parse in JSON, in csv: might still parse, but is now semantically incorrect and you possibly won’t know until it’s too late, depending on your parser. The list goes on.
  - pbrumm 2y ago
    Here is a quick test The table of contents points to a single Json object that is 20ish gb compressed https://www.anthem.com/machine-readable-file/search/ https://www.anthem.com/machine-readable-file/search/ All stock libs will fail
- nukem222 2y ago
  [flagged]
freehorse 2y ago
> CSV tooling has had [...] to make a fragile, under-specified format half-useful You get this backwards. Tabular structured data to store are ubiquitous. Text as a file format is also ubiquitous because it is accessible. The only actual decisions are about whether to encode your variables as rows or columns, what is the delimiter, and other rules such as escaping etc. Vars as columns makes sense because it makes appending easier. There is a bunch of stuff that can be used for delimeters, commas being the most common, none is perfect. But from this point onwards, decisions do not really matter, and "CSV" basically covers everything from now on. "CSV" is basically what comes naturally when you have tabular datasets and want to store them in text. CSV tooling is developed because there is a need for this way of formatting data. Whether CSV is "good" or "ugly" or whatever is irrelevant, handling data is complicated as much as the world itself is. The alternatives are either not structuring/storing the data in a tabular manner, or non-text (eg binary) formats. These alternative exist and are useful in their own right, but don't solve the same problems.