3 ms·

Sadly they did not include bad sides: 1) Vulnerabilities: not only in SQLite, but also in wrappers like https://nvd.nist.gov/vuln/detail/CVE-2023-32697 https:/

by Lockal 3y ago

Sadly they did not include bad sides: 1) Vulnerabilities: not only in SQLite, but also in wrappers like https://nvd.nist.gov/vuln/detail/CVE-2023-32697 https://nvd.nist.gov/vuln/detail/CVE-2023-32697 2) Lack of transparency: zip with xml's contains only xml's; meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. Attempts to fix these issues removes benefits that were mentioned. 3) Lack of implementer support. It was one of the reasons for WebSQL deprecation many years ago. 4) Lack of standardization for file format. SQLite does not even promise forward compatibility, only backward one. Which means that new documents might not open in old software, or vendor should fork SQLite and only backport security patches.

jmull 3y ago
4) is enough for me, so I agree with your general point, but 1) 2) and 3) aren't really cons for SQLite. 1) Makes sense only if the average XML parsers and zip libraries in use have fewer vulnerabilities and are actively maintained as well. 2) You can store sensitive data in a SQLite database or XML file, there's no real difference. You can clean up a SQLite database pretty easily if you want and that doesn't take away all the benefits. 3) What does implementer support even mean? I believe they are open to custom work... WebSQL died because it doesn't make sense to pretend SQLite is some kind of standard -- that brings us back to 4), which is the valid reason to avoid SQLite. Actually, your 4) is worded too strongly. They say they're committed to forward compatibility as long as you don't use the new features. That makes forward compatibility the decision of the app: an app can have forward compatibility and not use newer features OR lose forward compatibility and use newer features.
internetter 3y ago
> Vulnerabilities: not only in SQLite, but also in wrappers like Yes, parsing encoded files tends to introduce vulnerabilities. ZIP parsers have had plenty of vulnerabilities. This is not exclusive to SQLite. > Lack of transparency: zip with xml's contains only xml's Both zips and sqlite cannot be read with a text editor. Both are open formats with widely available tools to read them. The sqlite binary might, in fact, be more widely available than unzipping tools. > meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. Elaborate? > Lack of implementer support. It was one of the reasons for WebSQL deprecation many years ago. I don't understand how this is relevant? > SQLite does not even promise forward compatibility, only backward one. Which means that new documents might not open in old software Neither does OpenDocument. SQLite is actually more solid in this regard – forwards compatibility is still a thing unless new features are used.
- naniwaduni 3y ago
  > Both zips and sqlite cannot be read with a text editor. Both are open formats with widely available tools to read them. Well, that's why your archive format of choice should be cpio, which is almost a text file except that modern implementations tend to 0-terminate the filename! Jokes aside, there are widely-distributed tools that can take in an almost-arbitrary zip file and account for every byte in it. The format is straightforward enough that, were you so inclined, you can do most of it (other than, like, decompression and crc-checking) manually in a text editor. The SQLite format is not like this. There is one implementation, and relatively easy to "hide" data in a database file that its tooling will not reveal.
- Lockal 3y ago
  > parsing encoded files tends to introduce vulnerabilities If we are talking about binary formats, now there are systematic solutions like https://github.com/google/wuffs https://github.com/google/wuffs that protect against vulnerabilities. But SQLite is not just a format - it's an evolving ecosystem with constantly added features. And the most prominent issue was not even in core, it was in FTS3. What will SQLite add next? More json-related functions? Maybe BSON? It is useful, but does not help in this situation. Regarding traces, there are many forensics tools and even books about forensic analysis of SQLite databases. In well-designed format such tools should not exist in the first place. This is hard requirement: if it requires rewriting the whole file - then so be it.
- mcpackieh 3y ago
  > > meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. > Elaborate? When you delete something from a SQLite database, it isn't necessarily actually removed from the file unless you VACUUM or have the secure_delete PRAMGA turned on. Either of these should solve the problem. VACUUM INTO is a good way to export sqlite databases from an application for this reason.

by Lockal 3y ago

jmull 3y ago

4) is enough for me, so I agree with your general point, but 1) 2) and 3) aren't really cons for SQLite. 1) Makes sense only if the average XML parsers and zip libraries in use have fewer vulnerabilities and are actively maintained as well. 2) You can store sensitive data in a SQLite database or XML file, there's no real difference. You can clean up a SQLite database pretty easily if you want and that doesn't take away all the benefits. 3) What does implementer support even mean? I believe they are open to custom work... WebSQL died because it doesn't make sense to pretend SQLite is some kind of standard -- that brings us back to 4), which is the valid reason to avoid SQLite. Actually, your 4) is worded too strongly. They say they're committed to forward compatibility as long as you don't use the new features. That makes forward compatibility the decision of the app: an app can have forward compatibility and not use newer features OR lose forward compatibility and use newer features.

internetter 3y ago

> Vulnerabilities: not only in SQLite, but also in wrappers like Yes, parsing encoded files tends to introduce vulnerabilities. ZIP parsers have had plenty of vulnerabilities. This is not exclusive to SQLite. > Lack of transparency: zip with xml's contains only xml's Both zips and sqlite cannot be read with a text editor. Both are open formats with widely available tools to read them. The sqlite binary might, in fact, be more widely available than unzipping tools. > meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. Elaborate? > Lack of implementer support. It was one of the reasons for WebSQL deprecation many years ago. I don't understand how this is relevant? > SQLite does not even promise forward compatibility, only backward one. Which means that new documents might not open in old software Neither does OpenDocument. SQLite is actually more solid in this regard – forwards compatibility is still a thing unless new features are used.

naniwaduni 3y ago
> Both zips and sqlite cannot be read with a text editor. Both are open formats with widely available tools to read them. Well, that's why your archive format of choice should be cpio, which is almost a text file except that modern implementations tend to 0-terminate the filename! Jokes aside, there are widely-distributed tools that can take in an almost-arbitrary zip file and account for every byte in it. The format is straightforward enough that, were you so inclined, you can do most of it (other than, like, decompression and crc-checking) manually in a text editor. The SQLite format is not like this. There is one implementation, and relatively easy to "hide" data in a database file that its tooling will not reveal.
Lockal 3y ago
> parsing encoded files tends to introduce vulnerabilities If we are talking about binary formats, now there are systematic solutions like https://github.com/google/wuffs https://github.com/google/wuffs that protect against vulnerabilities. But SQLite is not just a format - it's an evolving ecosystem with constantly added features. And the most prominent issue was not even in core, it was in FTS3. What will SQLite add next? More json-related functions? Maybe BSON? It is useful, but does not help in this situation. Regarding traces, there are many forensics tools and even books about forensic analysis of SQLite databases. In well-designed format such tools should not exist in the first place. This is hard requirement: if it requires rewriting the whole file - then so be it.
mcpackieh 3y ago
> > meanwhile SQLite contains by design all kinds of traces with sensitive information or empty blocks. > Elaborate? When you delete something from a SQLite database, it isn't necessarily actually removed from the file unless you VACUUM or have the secure_delete PRAMGA turned on. Either of these should solve the problem. VACUUM INTO is a good way to export sqlite databases from an application for this reason.