3 ms·

> SQLite’s test suite is proprietary This is literally the first time I've ever heard of this, for any project anywhere. I suppose Android is built a bit in th

> SQLite’s test suite is proprietary This is literally the first time I've ever heard of this, for any project anywhere. I suppose Android is built a bit in this way, but that's a whole other can of worms.

heisenbit 2y ago
Well, Java during its initial life was controlled to a degree through the control of the tests: https://en.wikipedia.org/wiki/Technology_Compatibility_Kit https://en.wikipedia.org/wiki/Technology_Compatibility_Kit .
- vitiral 2y ago
  Huh, I wasn't aware that Java was initially open source
  - karussell 2y ago
    I think Java/JDK was closed source initially, then went open source in 2006/2007 (?), but without the TCK. The TCK was never open sourced but the JCK is now kind of "open": https://openjdk.org/groups/conformance/JckAccess/ https://openjdk.org/groups/conformance/JckAccess/
galangalalgol 2y ago
It could be simply to prevent forks, but if it really is 100% branch coverage, why do they still have memory safety related CVE coming out? With asan turned on, and full static analysis, that should make such errors exceedingly rare. Part of the benefit of rust is that it makes coverage both easier to get due to its type system, and less necessary because of the guarantees it makes. But if they really went all the way to 100% branch coverage that should be almost as good if all the samitizers are running.
- int_19h 2y ago
  They claim 100% on https://en.wikipedia.org/wiki/Modified_condition/decision_coverage https://en.wikipedia.org/wiki/Modified_condition/decision_co... However, unless you can guarantee that every branch tested has been covered for all possibly relevant application states, that does not preclude CVEs.
cryptonector 2y ago
They have a test suite that is part of SQLite3 then public domain product, and they have a much bigger and better test suite that is proprietary.
ncruces 2y ago
Large chunks of the test suite are open source, committed to the repo and easy to run with a `make test`. Everytime a bug is reported in the forums, the open source tests are updated as part of the bug fix for everyone to see. There's a separate test suite that offers 100% coverage, that is proprietary, and which was created for certification for use in safety critical environments. HN loves to discuss business models for open source, but apparently has a problem with this one. Why?
froh 2y ago
they do not fully own said proprietary sql test suite. they've licensed it. that's why they can _run_ it but not publish it or share it. That's at least how I remember Richard Hick describing the situation at a talk.

heisenbit 2y ago

Well, Java during its initial life was controlled to a degree through the control of the tests: https://en.wikipedia.org/wiki/Technology_Compatibility_Kit https://en.wikipedia.org/wiki/Technology_Compatibility_Kit .

vitiral 2y ago
Huh, I wasn't aware that Java was initially open source
- karussell 2y ago
  I think Java/JDK was closed source initially, then went open source in 2006/2007 (?), but without the TCK. The TCK was never open sourced but the JCK is now kind of "open": https://openjdk.org/groups/conformance/JckAccess/ https://openjdk.org/groups/conformance/JckAccess/

galangalalgol 2y ago

It could be simply to prevent forks, but if it really is 100% branch coverage, why do they still have memory safety related CVE coming out? With asan turned on, and full static analysis, that should make such errors exceedingly rare. Part of the benefit of rust is that it makes coverage both easier to get due to its type system, and less necessary because of the guarantees it makes. But if they really went all the way to 100% branch coverage that should be almost as good if all the samitizers are running.

int_19h 2y ago
They claim 100% on https://en.wikipedia.org/wiki/Modified_condition/decision_coverage https://en.wikipedia.org/wiki/Modified_condition/decision_co... However, unless you can guarantee that every branch tested has been covered for all possibly relevant application states, that does not preclude CVEs.

cryptonector 2y ago

They have a test suite that is part of SQLite3 then public domain product, and they have a much bigger and better test suite that is proprietary.

ncruces 2y ago

Large chunks of the test suite are open source, committed to the repo and easy to run with a `make test`. Everytime a bug is reported in the forums, the open source tests are updated as part of the bug fix for everyone to see. There's a separate test suite that offers 100% coverage, that is proprietary, and which was created for certification for use in safety critical environments. HN loves to discuss business models for open source, but apparently has a problem with this one. Why?

froh 2y ago

they do not fully own said proprietary sql test suite. they've licensed it. that's why they can _run_ it but not publish it or share it. That's at least how I remember Richard Hick describing the situation at a talk.