6 ms·

While I generally agree with the points they make I'm not sure I would want a (stock) sqlite being used for "less" trusted files. While sqlite has a very compr

While I generally agree with the points they make I'm not sure I would want a (stock) sqlite being used for "less" trusted files. While sqlite has a very comprehensive test suite there had been more than one attack with "manipulated" database files and as far as I know (I might be mistaken) the test suite(s) are not focused on testing that case much. Through I'm positive that it would be fine with a "hardened" sqlite where certain features are not compiled in at all and in turn the attack surface is reduced.

teraflop 6y ago
You could always sandbox it. This has actually already been done at least once: years ago, there used to be a "pure Java" JDBC driver for SQLite that was generated by compiling the original C code to a MIPS binary, and then translating the result into Java bytecode. http://web.archive.org/web/20080703234650/http://www.zentus.com/sqlitejdbc/ http://web.archive.org/web/20080703234650/http://www.zentus....
- ReactiveJelly 6y ago
  https://hacks.mozilla.org/2020/02/securing-firefox-with-webassembly/ https://hacks.mozilla.org/2020/02/securing-firefox-with-weba... Firefox did the same recently to sandbox Graphite (some font dependency) C/C++ --> WebAsm --> sandboxed AOT compiled native code
infogulch 6y ago
The article describes the current format as a zip of files where the main content of presentation slides is stored in a `content.xml` file within. Yes, sqlite may have had file format vulnerabilities in the past, but I cannot imagine trusting it less than any xml library in existence.
- vlovich123 6y ago
  You’d think so from reading [1]. [2] though paints a very different picture. > The attacker can submit a maliciously crafted database file to the application that the application will then open and query. I think it’s not impossible to secure SQLite better but it takes some work (like how Chrome sandboxes it in a separate process). [1] https://www.sqlite.org/security.html https://www.sqlite.org/security.html [2] https://www.sqlite.org/cves.html https://www.sqlite.org/cves.html
  - TylerE 6y ago
    I think his point is that if an attacker wants to give you bad data, there is already zero barrier to editing an XML file inside a zip.
  - moonchild 6y ago
    > [2] though paints a very different picture. I don't agree. That document states that the CVEs have historically required one of two preconditions: > The attacker can submit and run arbitrary SQL statements. > The attacker can submit a maliciously crafted database file to the application that the application will then open and query. If you look at the actual list of CVEs, all but one start with ‘Malicious SQL statement’. The single one that doesn't is suffixed with ‘The bug never appeared in any official SQLite release’. In other words, there has never been an official release of SQLite which was vulnerable when presented with a crafted database file.
    - ori_b 6y ago
      > If you look at the actual list of CVEs, all but one start with ‘Malicious SQL statement’. The single one that doesn't is suffixed with ‘The bug never appeared in any official SQLite release’. The list on the page is partial, and the language on that page is disturbingly dismissive. Reading leaves me more concerned that they're too arrogant to take security researchers seriously.
      - 0zymandiass 6y ago
        I'm sure they aren't perfect, but their test suite and extensive fuzz testing is far better than any XML or JSON parser I've ever seen https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html
        fpoling 6y ago
        I do not agree about JSON. As a format it is so trivial one can cover it with few test cases. SQL dialect that SQLite uses is vastly more complex.
- fpoling 6y ago
  At work we used a library that relied on Expat XML parser. After 20 years it still gets exploitable bugs. As the software was Windows-only we patched the library to use OS API for XML parsing to delegate to Microsoft the job of fixing security bugs.
remram 6y ago
Can you elaborate about this? What are the gotchas when opening an untrusted SQLite3 database? (I am developing a web application that will allow users to upload application files, which are SQLite3 databases)
- kibibyte 6y ago
  Section 1.2 here might be a good start. https://www.sqlite.org/security.html https://www.sqlite.org/security.html
- hansvm 6y ago
  I haven't taken the time to verify any of it, but a quick search for "sqlite code execution" doesn't look promising.
- ori_b 6y ago
  It's a multi-hundred-thousand line project (current count, excluding tests, is about 210,000). Like any project of a similar size, it has a history of security vulnerabilities. OpenBSD removed it because it was too much code to audit. OpenBSD dodged several issues like this RCE (https://research.checkpoint.com/2019/select-code_execution-from-using-sqlite/ https://research.checkpoint.com/2019/select-code_execution-f...) as a result. See https://www.cvedetails.com/vulnerability-list/vendor_id-9237/Sqlite.html https://www.cvedetails.com/vulnerability-list/vendor_id-9237... Hundreds of thousands of lines of code are simply going to have vulnerabilities and bugs. It's unavoidable with that level of complexity. I'm skeptical anything that complicated is a wise choice for a general file format -- especially when smaller, simpler, and more easily debuggable and independently implementable formats exist.
- mcint 6y ago
  The Full Text Search extension (FTS) (FTS4?) has been a repeated source of issues. If you stick with the core, and disable some of the shiny features, the track record is better. Though still not spotless, iirc.
Aeolun 6y ago
I think the same is true for less trusted MS office files though? E.g. VBA can do some crazy stuff.
yencabulator 6y ago
Further reading: https://research.checkpoint.com/2019/select-code_execution-from-using-sqlite/ https://research.checkpoint.com/2019/select-code_execution-f... I've never seen mention of this being fixed in any way. SQLite is not an acceptable format to transfer between computers.
- SQLite 6y ago
  That bug was fixed long ago, even before the referenced video was produced. The described attack is clever. It exploits the fact that an attacker might alter the schema so that it invokes an SQL function with side-effects when an app simply tries to read from a table. And depending on those side-effects, an exploit might be possible. In the video, there was a bug in a built-in SQL function that caused exploitable side-effects. But that bug was fixed long before the video was even produced. The examples in the video were from an older version of SQLite. They did not work for the latest SQLite release on the day that lecture was given. Since the checkpoint.com attack was described (in the video and elsewhere) new defense-in-depth features have been added to SQLite to make similar exploits increasingly unlikely. (1) Built-in SQL functions that have side effects cannot be used in the schema. Side-effect functions can only be invoked directly by the application. (2) When applications register their own custom SQL functions, they can now mark those functions as "direct-only", meaning that they are prohibited in the schema. (3) Run-time and compile-time options are available to prohibit the use of SQL functions in the schema that are not explicitly declared to be safe for use in the schema - that is, functions without side effects. This is for use in legacy applications that might have been created before the per-function flag that prohibited use within the schema was available. It is also an extra layer of defense for complex applications that might add hundreds or thousands of side-effect SQL functions - to ensure that the "direct-only" flag is not accidentally omitted from one of them. (4) Run-time options are available to disable triggers and views in applications that do not need them. This is not necessary to avoid an exploit, but it does provide an additional layer of defense. See https://sqlite.org/security.html https://sqlite.org/security.html and especially paragraph 1.2 item 8 for additional information.

teraflop 6y ago

You could always sandbox it. This has actually already been done at least once: years ago, there used to be a "pure Java" JDBC driver for SQLite that was generated by compiling the original C code to a MIPS binary, and then translating the result into Java bytecode. http://web.archive.org/web/20080703234650/http://www.zentus.com/sqlitejdbc/ http://web.archive.org/web/20080703234650/http://www.zentus....

ReactiveJelly 6y ago
https://hacks.mozilla.org/2020/02/securing-firefox-with-webassembly/ https://hacks.mozilla.org/2020/02/securing-firefox-with-weba... Firefox did the same recently to sandbox Graphite (some font dependency) C/C++ --> WebAsm --> sandboxed AOT compiled native code

infogulch 6y ago

The article describes the current format as a zip of files where the main content of presentation slides is stored in a `content.xml` file within. Yes, sqlite may have had file format vulnerabilities in the past, but I cannot imagine trusting it less than any xml library in existence.

vlovich123 6y ago
You’d think so from reading [1]. [2] though paints a very different picture. > The attacker can submit a maliciously crafted database file to the application that the application will then open and query. I think it’s not impossible to secure SQLite better but it takes some work (like how Chrome sandboxes it in a separate process). [1] https://www.sqlite.org/security.html https://www.sqlite.org/security.html [2] https://www.sqlite.org/cves.html https://www.sqlite.org/cves.html
- TylerE 6y ago
  I think his point is that if an attacker wants to give you bad data, there is already zero barrier to editing an XML file inside a zip.
- moonchild 6y ago
  > [2] though paints a very different picture. I don't agree. That document states that the CVEs have historically required one of two preconditions: > The attacker can submit and run arbitrary SQL statements. > The attacker can submit a maliciously crafted database file to the application that the application will then open and query. If you look at the actual list of CVEs, all but one start with ‘Malicious SQL statement’. The single one that doesn't is suffixed with ‘The bug never appeared in any official SQLite release’. In other words, there has never been an official release of SQLite which was vulnerable when presented with a crafted database file.
  - ori_b 6y ago
    > If you look at the actual list of CVEs, all but one start with ‘Malicious SQL statement’. The single one that doesn't is suffixed with ‘The bug never appeared in any official SQLite release’. The list on the page is partial, and the language on that page is disturbingly dismissive. Reading leaves me more concerned that they're too arrogant to take security researchers seriously.
    - 0zymandiass 6y ago
      I'm sure they aren't perfect, but their test suite and extensive fuzz testing is far better than any XML or JSON parser I've ever seen https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html
      - fpoling 6y ago
        I do not agree about JSON. As a format it is so trivial one can cover it with few test cases. SQL dialect that SQLite uses is vastly more complex.
fpoling 6y ago
At work we used a library that relied on Expat XML parser. After 20 years it still gets exploitable bugs. As the software was Windows-only we patched the library to use OS API for XML parsing to delegate to Microsoft the job of fixing security bugs.

remram 6y ago

Can you elaborate about this? What are the gotchas when opening an untrusted SQLite3 database? (I am developing a web application that will allow users to upload application files, which are SQLite3 databases)

kibibyte 6y ago
Section 1.2 here might be a good start. https://www.sqlite.org/security.html https://www.sqlite.org/security.html
hansvm 6y ago
I haven't taken the time to verify any of it, but a quick search for "sqlite code execution" doesn't look promising.
ori_b 6y ago
It's a multi-hundred-thousand line project (current count, excluding tests, is about 210,000). Like any project of a similar size, it has a history of security vulnerabilities. OpenBSD removed it because it was too much code to audit. OpenBSD dodged several issues like this RCE (https://research.checkpoint.com/2019/select-code_execution-from-using-sqlite/ https://research.checkpoint.com/2019/select-code_execution-f...) as a result. See https://www.cvedetails.com/vulnerability-list/vendor_id-9237/Sqlite.html https://www.cvedetails.com/vulnerability-list/vendor_id-9237... Hundreds of thousands of lines of code are simply going to have vulnerabilities and bugs. It's unavoidable with that level of complexity. I'm skeptical anything that complicated is a wise choice for a general file format -- especially when smaller, simpler, and more easily debuggable and independently implementable formats exist.
mcint 6y ago
The Full Text Search extension (FTS) (FTS4?) has been a repeated source of issues. If you stick with the core, and disable some of the shiny features, the track record is better. Though still not spotless, iirc.

Aeolun 6y ago

I think the same is true for less trusted MS office files though? E.g. VBA can do some crazy stuff.

yencabulator 6y ago

Further reading: https://research.checkpoint.com/2019/select-code_execution-from-using-sqlite/ https://research.checkpoint.com/2019/select-code_execution-f... I've never seen mention of this being fixed in any way. SQLite is not an acceptable format to transfer between computers.

SQLite 6y ago
That bug was fixed long ago, even before the referenced video was produced. The described attack is clever. It exploits the fact that an attacker might alter the schema so that it invokes an SQL function with side-effects when an app simply tries to read from a table. And depending on those side-effects, an exploit might be possible. In the video, there was a bug in a built-in SQL function that caused exploitable side-effects. But that bug was fixed long before the video was even produced. The examples in the video were from an older version of SQLite. They did not work for the latest SQLite release on the day that lecture was given. Since the checkpoint.com attack was described (in the video and elsewhere) new defense-in-depth features have been added to SQLite to make similar exploits increasingly unlikely. (1) Built-in SQL functions that have side effects cannot be used in the schema. Side-effect functions can only be invoked directly by the application. (2) When applications register their own custom SQL functions, they can now mark those functions as "direct-only", meaning that they are prohibited in the schema. (3) Run-time and compile-time options are available to prohibit the use of SQL functions in the schema that are not explicitly declared to be safe for use in the schema - that is, functions without side effects. This is for use in legacy applications that might have been created before the per-function flag that prohibited use within the schema was available. It is also an extra layer of defense for complex applications that might add hundreds or thousands of side-effect SQL functions - to ensure that the "direct-only" flag is not accidentally omitted from one of them. (4) Run-time options are available to disable triggers and views in applications that do not need them. This is not necessary to avoid an exploit, but it does provide an additional layer of defense. See https://sqlite.org/security.html https://sqlite.org/security.html and especially paragraph 1.2 item 8 for additional information.