25 ms·

My understanding is that SQLite doesn't impose any format either?

by millstone 6y ago

deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
mb7733 6y ago
I'm genuinely curious what you mean by this. Of course SQLite imposes a file format... That format is a SQLite database
- Dylan16807 6y ago
  > Of course SQLite imposes a file format... That format is a SQLite database Well this is in a context that rejects zip as being a format. Do you do that? If the answer is no then skip the rest of my post and just note that they're talking about a different definition of 'format'. - But in that context: The amount of structure imposed on you by the sqlite database format is not much more than the structure imposed on you by a zip. I think it's fair to rate them similarly as formats. A zip file is basically a key-value store. "Zip full of csvs", while awful to use, would impose about the same amount of structure as sqlite does: not much. And zip+csv is not much more elaborate than zip on its own.
  - samatman 6y ago
    In SQLite, I can have a table with a row which is `flag INTEGER NOT NULL CHECK flag == 0 or flag == 1 DEFAULT 0` Surely this is more structure than a ZIP file, which is merely a way of compressing a directory of files into a single entity, can provide? Sure the INTEGER part doesn't really do anything...
    - Dylan16807 6y ago
      That's a feature of the database engine, not the file format. You could attach a similar database engine to a zip backend.
      - samatman 6y ago
        This is an asinine objection.
        Dylan16807 6y ago
        I have a better one, if you want it. A configuration like that comes from the program using sqlite. Just adding sqlite into a system doesn't set up any data formatting like that. Sqlite itself gives you a blank canvas. And a blank canvas is not much of a data format.
        samatman 6y ago
        You're wrong about this. The SQLite data format includes the schema, in plain ASCII. This self-documenting nature makes it an excellent data format, I've taken advantage of it numerous times in making use of SQLite-based application file formats. SQLite is put forth as a basis for an application file format, and by definition it must be sufficiently flexible to accommodate any application. But by including the schema, it is self-documenting as to what the structure is, which ZIP isn't and can't be. QED.
- millstone 6y ago
  I hope to make the modest point that, if you package your data in a SQLite file or a zip file, then you have both packaging and data problems. MyCoolSQLApp may read and write a SQLite file with its own schema, but it can't handle an arbitrary SQLite file. Likewise MyCoolZipApp can't handle an arbitrary zip file.
  - mb7733 6y ago
    I'm not clear on the point here. You can make an application that handles arbitrary SQLite files, (for example: the SQLite shell, or a GUI database broswer), or you can make an application that only uses SQLite files with certain content/schema. Does this mean that SQLite does or does not define a file format? If not, what do you call the specification of how data is stored in a SQLite file besides a 'file format'?
dragonwriter 6y ago
> My understanding is that SQLite doesn't impose any format either? That's true that you can stuff any kind of string/blob data into any column of any table, so, yes, you still have to determine the data schema with sqlite much as you do with JSON, XML, or even CSV. I mean, I could have a CSV where each element is a base64-encoded ZIP containing sqlite database files that are each a single table with a single column of JSON files, each of which contains a JSON array of strings with XML documents in them. But that's usually not something people would mean if they said their app was using CSV as it's data storage format, nor is the version stripping out CSV on the top what people would mean if they say they are using SQLite. With ZIP, you have to decide the format(s) for the file(s) in the ZIP, their hierarchical structure, and, if the files aren't themselves the atomic data elements, the schema applicable to each file. Furthermore, in discussion of performance characteristics and other aspects of suitability, ZIP adds overhead, but you still also need to consider the access properties of the contained files.

My understanding is that SQLite doesn't impose any format either?

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

mb7733 6y ago

I'm genuinely curious what you mean by this. Of course SQLite imposes a file format... That format is a SQLite database

Dylan16807 6y ago
> Of course SQLite imposes a file format... That format is a SQLite database Well this is in a context that rejects zip as being a format. Do you do that? If the answer is no then skip the rest of my post and just note that they're talking about a different definition of 'format'. - But in that context: The amount of structure imposed on you by the sqlite database format is not much more than the structure imposed on you by a zip. I think it's fair to rate them similarly as formats. A zip file is basically a key-value store. "Zip full of csvs", while awful to use, would impose about the same amount of structure as sqlite does: not much. And zip+csv is not much more elaborate than zip on its own.
- samatman 6y ago
  In SQLite, I can have a table with a row which is `flag INTEGER NOT NULL CHECK flag == 0 or flag == 1 DEFAULT 0` Surely this is more structure than a ZIP file, which is merely a way of compressing a directory of files into a single entity, can provide? Sure the INTEGER part doesn't really do anything...
  - Dylan16807 6y ago
    That's a feature of the database engine, not the file format. You could attach a similar database engine to a zip backend.
    - samatman 6y ago
      This is an asinine objection.
      - Dylan16807 6y ago
        I have a better one, if you want it. A configuration like that comes from the program using sqlite. Just adding sqlite into a system doesn't set up any data formatting like that. Sqlite itself gives you a blank canvas. And a blank canvas is not much of a data format.
        samatman 6y ago
        You're wrong about this. The SQLite data format includes the schema, in plain ASCII. This self-documenting nature makes it an excellent data format, I've taken advantage of it numerous times in making use of SQLite-based application file formats. SQLite is put forth as a basis for an application file format, and by definition it must be sufficiently flexible to accommodate any application. But by including the schema, it is self-documenting as to what the structure is, which ZIP isn't and can't be. QED.
millstone 6y ago
I hope to make the modest point that, if you package your data in a SQLite file or a zip file, then you have both packaging and data problems. MyCoolSQLApp may read and write a SQLite file with its own schema, but it can't handle an arbitrary SQLite file. Likewise MyCoolZipApp can't handle an arbitrary zip file.
- mb7733 6y ago
  I'm not clear on the point here. You can make an application that handles arbitrary SQLite files, (for example: the SQLite shell, or a GUI database broswer), or you can make an application that only uses SQLite files with certain content/schema. Does this mean that SQLite does or does not define a file format? If not, what do you call the specification of how data is stored in a SQLite file besides a 'file format'?

dragonwriter 6y ago

> My understanding is that SQLite doesn't impose any format either? That's true that you can stuff any kind of string/blob data into any column of any table, so, yes, you still have to determine the data schema with sqlite much as you do with JSON, XML, or even CSV. I mean, I could have a CSV where each element is a base64-encoded ZIP containing sqlite database files that are each a single table with a single column of JSON files, each of which contains a JSON array of strings with XML documents in them. But that's usually not something people would mean if they said their app was using CSV as it's data storage format, nor is the version stripping out CSV on the top what people would mean if they say they are using SQLite. With ZIP, you have to decide the format(s) for the file(s) in the ZIP, their hierarchical structure, and, if the files aren't themselves the atomic data elements, the schema applicable to each file. Furthermore, in discussion of performance characteristics and other aspects of suitability, ZIP adds overhead, but you still also need to consider the access properties of the contained files.