3 ms·

I personally believe that at least SQLite matches all those criteria : "1) A truly open format is available" : sqlite is open-source, MIT-licensed, and well sp

I personally believe that at least SQLite matches all those criteria : "1) A truly open format is available" : sqlite is open-source, MIT-licensed, and well specified (even though I am usually not so happy with its weak typing approach, yet in this case this precisely enables a 100% correspondance between CSV and sqlite since CSV has also no typing at all...) "2) Applications have a speed increase from using csvs" : I think it should be obvious to everyone that this is the case... "3) The new format should not be grossly larger than the one it is replacing" : this is also the case

unnouinceput 5y ago
sqlite files are binary files, so a non-starter. If I open them in Notepad I can't read the values like I can with CSV.
- postalrat 5y ago
  "If I open them in Notepad" Well there's your problem.
  - munk-a 5y ago
    There is a definite demand for some import format that you can trivially edit - CSV excels at this right now and JSON isn't that bad. Binary dump formats definitely do have a time and a place but there is also a separate need for trivially human readable formats.
    - postalrat 5y ago
      I guess my point is that CSVs or any text files aren't really "human readable". You still need some application top view them.
      - HanaShiratori 5y ago
        So what's the alternative? Good old pen and paper?
        vertere 5y ago
        Doesn't count, you still need light to make it human-readable. /s
      - icelancer 5y ago
        I can - and do - read a CSV by opening it in Notepad++ to view basic data. This isn't possible with a SQLite file.
      - creata 5y ago
        In some incredibly pedantic sense, sure. But the difference is that every computer comes with what a human needs to read a CSV file. That's not true of an SQLite database.
        atatatat 5y ago
        Could it be?
        CRConrad 5y ago
        Sure, could be. In about the same sense that it could be that every computer comes with a free and open source operating system. Could be; it's not theoretically impossible -- but...
- euroderf 5y ago
  How about an app that opens an SQLite DB so that strings and numbers can easily be edited, and then on file save, SQLite's internal pointers and markers are updated ? Maybe an Emacs mode ?
anigbrowl 5y ago
It's worth mentioning that SQLite is committed by its publishers to be backward-compatible out to 2050 and is the recommended by the Library of Congress as a safe long-term storage format (as are XML, JSON, and CSV). https://www.sqlite.org/locrsf.html https://www.sqlite.org/locrsf.html For single tables a database is probably overkill, but it's nice to have around when you need something reasonably powerful without being overly complex or hard to get started with.
- dolni 5y ago
  I suppose if I was to recommend some "safe long-term storage formats" I, too, would choose things that are readable in plain ASCII and/or have open source roots. Shoutout to the Library of Congress for doing the real heavy lifting here?

unnouinceput 5y ago

sqlite files are binary files, so a non-starter. If I open them in Notepad I can't read the values like I can with CSV.

postalrat 5y ago
"If I open them in Notepad" Well there's your problem.
- munk-a 5y ago
  There is a definite demand for some import format that you can trivially edit - CSV excels at this right now and JSON isn't that bad. Binary dump formats definitely do have a time and a place but there is also a separate need for trivially human readable formats.
  - postalrat 5y ago
    I guess my point is that CSVs or any text files aren't really "human readable". You still need some application top view them.
    - HanaShiratori 5y ago
      So what's the alternative? Good old pen and paper?
      - vertere 5y ago
        Doesn't count, you still need light to make it human-readable. /s
    - icelancer 5y ago
      I can - and do - read a CSV by opening it in Notepad++ to view basic data. This isn't possible with a SQLite file.
    - creata 5y ago
      In some incredibly pedantic sense, sure. But the difference is that every computer comes with what a human needs to read a CSV file. That's not true of an SQLite database.
      - atatatat 5y ago
        Could it be?
        CRConrad 5y ago
        Sure, could be. In about the same sense that it could be that every computer comes with a free and open source operating system. Could be; it's not theoretically impossible -- but...
euroderf 5y ago
How about an app that opens an SQLite DB so that strings and numbers can easily be edited, and then on file save, SQLite's internal pointers and markers are updated ? Maybe an Emacs mode ?

anigbrowl 5y ago

It's worth mentioning that SQLite is committed by its publishers to be backward-compatible out to 2050 and is the recommended by the Library of Congress as a safe long-term storage format (as are XML, JSON, and CSV). https://www.sqlite.org/locrsf.html https://www.sqlite.org/locrsf.html For single tables a database is probably overkill, but it's nice to have around when you need something reasonably powerful without being overly complex or hard to get started with.

dolni 5y ago
I suppose if I was to recommend some "safe long-term storage formats" I, too, would choose things that are readable in plain ASCII and/or have open source roots. Shoutout to the Library of Congress for doing the real heavy lifting here?