5 ms·

To point 1, I'd argue that a SQLite database is a great next step beyond CSV despite being a binary format.

by cls59 5y ago

lmilcin 5y ago
SQLite also requires external dependency and special execution environment and are not as portable as CSV.
- srcreigh 5y ago
  SQLite is included in most operating systems by default. It's also on macOS and iOS. It also runs on all kinds of embedded devices in addition to personal computers and servers. https://en.wikipedia.org/wiki/SQLite#Operating_systems https://en.wikipedia.org/wiki/SQLite#Operating_systems
  - deleted 5y ago
    [deleted]
  - lmilcin 5y ago
    Just because most popular operating systems have libraries, doesn't yet mean it is portable. Can I click on it and open it in Excel? If not then it is not portable for me and for a lot of other people. The main reason I use CSV is to produce reports that I can either open myself or send to other people so that they can click on it and open themselves and immediately start hacking away. Excel is still corporate lingua franca and until this changes CSV is not going to retire. CSV means for me that I can send it to anybody and everybody will know how to use it. Try sending SQLite binary to people you work with, to your accountant, to your wife. Then share your experience.
    - wombatpm 5y ago
      Try using tabs. Still open in excel with less problems
      - lmilcin 5y ago
        I find files where the exact type of whitespace used is meaningful to be a sick joke. Not all editors make it easy for you to use a tab character. If only there was some other solution that would not require you to use a character that a lot of editors refuse to insert or even try to convert to spaces...
da_chicken 5y ago
No, SQLite's dynamic data types would silently coerce data just like opening a CSV directly with Excel does. The advantage of CSV is that it's as accurate as your plain text representation of your data can be. Since binary data can be represented by character data, that's 100% accurate. As soon as you introduce a storage format that has made assumptions about the type of data being stored, you've lost flexibility. SQLite is not intended for data serialization. It's intended for data storage to be read back by essentially the same application in the same environment.
- masklinn 5y ago
  > No, SQLite's dynamic data types would silently coerce data just like opening a CSV directly with Excel does. SQLite's "dynamic data types" coerce data on input, not output. Once the data is in sqlite the way you wanted it, excel has no interpretation to perform, except insofar as really really wanting dates. > The advantage of CSV is that it's as accurate as your plain text representation of your data can be. Yeah nah. > SQLite is not intended for data serialization. It's intended for data storage to be read back by essentially the same application in the same environment. That's completely, absolutely, utterly, objectively, nonsensical. Data analysis, exchange, container, and archiving, are literally all use cases listed as "appropriate uses for sqlite" in "Situations Where SQLite Works Well" on the official sqlite website: https://www.sqlite.org/whentouse.html https://www.sqlite.org/whentouse.html
  - edmundsauto 5y ago
    How is a csv not the most accurate representation of the data? If you trust the other agent encoded it properly in the db, then sure. Your flippant dismissal was inappropriate in tone and detracted from the rest of your opinion. Cockiness tells me that you’re insecure about your knowledge, not that you know more than GP.
    - masklinn 5y ago
      > How is a csv not the most accurate representation of the data? If you trust the other agent encoded it properly in the db, then sure. The idea that a CSV would be more likely to be correctly encoded than a DB is hilarious, thanks for the laugh. But that you were confident enough to seriously put it in writing shows how little experience you have with CSV.

To point 1, I'd argue that a SQLite database is a great next step beyond CSV despite being a binary format.

by cls59 5y ago

To point 1, I'd argue that a SQLite database is a great next step beyond CSV despite being a binary format.

lmilcin 5y ago

SQLite also requires external dependency and special execution environment and are not as portable as CSV.

srcreigh 5y ago
SQLite is included in most operating systems by default. It's also on macOS and iOS. It also runs on all kinds of embedded devices in addition to personal computers and servers. https://en.wikipedia.org/wiki/SQLite#Operating_systems https://en.wikipedia.org/wiki/SQLite#Operating_systems
- deleted 5y ago
  [deleted]
- lmilcin 5y ago
  Just because most popular operating systems have libraries, doesn't yet mean it is portable. Can I click on it and open it in Excel? If not then it is not portable for me and for a lot of other people. The main reason I use CSV is to produce reports that I can either open myself or send to other people so that they can click on it and open themselves and immediately start hacking away. Excel is still corporate lingua franca and until this changes CSV is not going to retire. CSV means for me that I can send it to anybody and everybody will know how to use it. Try sending SQLite binary to people you work with, to your accountant, to your wife. Then share your experience.
  - wombatpm 5y ago
    Try using tabs. Still open in excel with less problems
    - lmilcin 5y ago
      I find files where the exact type of whitespace used is meaningful to be a sick joke. Not all editors make it easy for you to use a tab character. If only there was some other solution that would not require you to use a character that a lot of editors refuse to insert or even try to convert to spaces...

da_chicken 5y ago

No, SQLite's dynamic data types would silently coerce data just like opening a CSV directly with Excel does. The advantage of CSV is that it's as accurate as your plain text representation of your data can be. Since binary data can be represented by character data, that's 100% accurate. As soon as you introduce a storage format that has made assumptions about the type of data being stored, you've lost flexibility. SQLite is not intended for data serialization. It's intended for data storage to be read back by essentially the same application in the same environment.

masklinn 5y ago
> No, SQLite's dynamic data types would silently coerce data just like opening a CSV directly with Excel does. SQLite's "dynamic data types" coerce data on input, not output. Once the data is in sqlite the way you wanted it, excel has no interpretation to perform, except insofar as really really wanting dates. > The advantage of CSV is that it's as accurate as your plain text representation of your data can be. Yeah nah. > SQLite is not intended for data serialization. It's intended for data storage to be read back by essentially the same application in the same environment. That's completely, absolutely, utterly, objectively, nonsensical. Data analysis, exchange, container, and archiving, are literally all use cases listed as "appropriate uses for sqlite" in "Situations Where SQLite Works Well" on the official sqlite website: https://www.sqlite.org/whentouse.html https://www.sqlite.org/whentouse.html
- edmundsauto 5y ago
  How is a csv not the most accurate representation of the data? If you trust the other agent encoded it properly in the db, then sure. Your flippant dismissal was inappropriate in tone and detracted from the rest of your opinion. Cockiness tells me that you’re insecure about your knowledge, not that you know more than GP.
  - masklinn 5y ago
    > How is a csv not the most accurate representation of the data? If you trust the other agent encoded it properly in the db, then sure. The idea that a CSV would be more likely to be correctly encoded than a DB is hilarious, thanks for the laugh. But that you were confident enough to seriously put it in writing shows how little experience you have with CSV.