5 ms·

EDIT: Deleted a controversial comment about the often praised SQLite test suite not actually being public and about locking behaviour in SQLite. References are

EDIT: Deleted a controversial comment about the often praised SQLite test suite not actually being public and about locking behaviour in SQLite. References are [1] and [2]. [1] https://www.sqlite.org/th3.html https://www.sqlite.org/th3.html [2] https://beets.io/blog/sqlite-nightmare.html https://beets.io/blog/sqlite-nightmare.html

rathel 7y ago
If I understand you correctly, there is another test suite in addition to the one you can download freely?
- liability 7y ago
  The TH3 test suite is proprietary: >"The TH3 test harness is a set of proprietary tests, written in C that provide 100% branch test coverage (and 100% MC/DC test coverage) to the core SQLite library. The TH3 tests are designed to run on embedded and specialized platforms that would not easily support TCL or other workstation services. TH3 tests use only the published SQLite interfaces. TH3 consists of about 69.4 MB or 948.1 KSLOC of C code implementing 44753 distinct test cases. TH3 tests are heavily parameterized, though, so a full-coverage test runs about 1.7 million different test instances. The cases that provide 100% branch test coverage constitute a subset of the total TH3 test suite. A soak test prior to release does hundreds of millions of tests. Additional information on TH3 is available separately." https://sqlite.org/testing.html https://sqlite.org/testing.html
- Tomte 7y ago
  > The TH3 test harness is a set of proprietary tests https://www.sqlite.org/testing.html#harnesses https://www.sqlite.org/testing.html#harnesses
liability 7y ago
D. Richard Hipp has always been pretty up front about his business model. He provides sqlite for free, public domain, and does not typically entertain third party contributions to keep the licensing clean. Additionally his business sells proprietary extensions and support for sqlite. Having that test suite uniquely positions them to do so, and due to this business model they expect to have the funding to continue providing a public domain sqlite to the world for decades to come.
- deleted 7y ago
  [deleted]
- liability 7y ago
  Resp to deleted comment: sqlite is considered such because it's open source and well tested. That doesn't seem to imply to me that the tests are also open source. I don't think Hipp is lying about his proprietary test suite, so I just can't convince myself to lose sleep over this.
  - quickquestion42 7y ago
    Yes, fair enough. Note that I never accused anybody of not telling the truth. My point was about how useful sqlite can be as the posterchild example of a good test suite when we can't see the secret sauce. Not that it doesn't exist. I deleted the other comment because after reading yours I realised it was needlessly controversial. I think you're right in that SQLite can be considered an example of having a good test suite when looked at from a different perspective, just not from the point of it (the test suite) being open source.

rathel 7y ago

If I understand you correctly, there is another test suite in addition to the one you can download freely?

liability 7y ago
The TH3 test suite is proprietary: >"The TH3 test harness is a set of proprietary tests, written in C that provide 100% branch test coverage (and 100% MC/DC test coverage) to the core SQLite library. The TH3 tests are designed to run on embedded and specialized platforms that would not easily support TCL or other workstation services. TH3 tests use only the published SQLite interfaces. TH3 consists of about 69.4 MB or 948.1 KSLOC of C code implementing 44753 distinct test cases. TH3 tests are heavily parameterized, though, so a full-coverage test runs about 1.7 million different test instances. The cases that provide 100% branch test coverage constitute a subset of the total TH3 test suite. A soak test prior to release does hundreds of millions of tests. Additional information on TH3 is available separately." https://sqlite.org/testing.html https://sqlite.org/testing.html
Tomte 7y ago
> The TH3 test harness is a set of proprietary tests https://www.sqlite.org/testing.html#harnesses https://www.sqlite.org/testing.html#harnesses

liability 7y ago

D. Richard Hipp has always been pretty up front about his business model. He provides sqlite for free, public domain, and does not typically entertain third party contributions to keep the licensing clean. Additionally his business sells proprietary extensions and support for sqlite. Having that test suite uniquely positions them to do so, and due to this business model they expect to have the funding to continue providing a public domain sqlite to the world for decades to come.

deleted 7y ago
[deleted]
liability 7y ago
Resp to deleted comment: sqlite is considered such because it's open source and well tested. That doesn't seem to imply to me that the tests are also open source. I don't think Hipp is lying about his proprietary test suite, so I just can't convince myself to lose sleep over this.
- quickquestion42 7y ago
  Yes, fair enough. Note that I never accused anybody of not telling the truth. My point was about how useful sqlite can be as the posterchild example of a good test suite when we can't see the secret sauce. Not that it doesn't exist. I deleted the other comment because after reading yours I realised it was needlessly controversial. I think you're right in that SQLite can be considered an example of having a good test suite when looked at from a different perspective, just not from the point of it (the test suite) being open source.