5 ms·

How SQLite Is Tested

59 points by deegles 12y ago

voltagex_ 12y ago
Previous discussions: https://hn.algolia.com/?query=http:%2F%2Fwww.sqlite.org%2Ftesting.html&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=http:%2F%2Fwww.sqlite.org%2Fte...
NelsonMinar 12y ago
I wonder if this test suite will be the last known use of TCL in modern code?
- deleted 12y ago
  [deleted]
- hobo_mark 12y ago
  At least in chip design, and Electronic Design Automation in general, TCL is still heavily used.
- kijin 12y ago
  AFAIK the test suite for Redis is also written in TCL.
  - bch 12y ago
    Indeed - both antirez and drh are big Tcl users. drh is a Tcl Core Team (TCT) member emeritus, and antirez is responsible for this Tcl love letter: http://antirez.com/articoli/tclmisunderstood.html http://antirez.com/articoli/tclmisunderstood.html
- shanemhansen 12y ago
  openacs [http://openacs.org/ http://openacs.org/] is still alive. Openacs was my first web development framework. My first ever database training was from Greenspun's "TCL for web nerds" book. Those are some fond memories.
- mzs 12y ago
  Does expect count?
- bch 12y ago
  Tealeaf[0] collectors engine is Tcl on Linux. F5 and A10 appliances are Tcl at their core (see this[1] for whether they'd do it again). Cisco still embeds Tcl as far as I know. Fidessa[2] trading systems are Tcl[3][4], and on and on and on. [0] http://www-01.ibm.com/software/info/tealeaf/ http://www-01.ibm.com/software/info/tealeaf/ [1] https://devcentral.f5.com/articles/irules-concepts-tcl-the-how-and-why https://devcentral.f5.com/articles/irules-concepts-tcl-the-h... [2] https://www.fidessa.com/about-us https://www.fidessa.com/about-us [3] https://www.fidessa.com/products/training/training-courses https://www.fidessa.com/products/training/training-courses [4] http://www.services.fidessa.com/brochures/_training/VF043.pdf http://www.services.fidessa.com/brochures/_training/VF043.pd... EDIT: I only need to reference a footnote 1x, but need to reference ea. footnote once., update Fidessa references.
antonios 12y ago
This post is the most reliable karma generator I've seen on HN.
a3_nm 12y ago
It's interesting that, even though sqlite is free software (it is even public domain), some of the tests are proprietary: https://www.sqlite.org/th3.html https://www.sqlite.org/th3.html. I wonder what's the goal of this...
- acqq 12y ago
  How about the goal of allowing the author to actually earn something?
  - hueving 12y ago
    And how exactly would the author do that?
    - acqq 12y ago
      The author is selling the support to the companies that can request some changes. Having the best tests, he can claim that the modifications he introduces are "production ready" and charge for that too. Others can just say "we changed something and it works maybe." Just writing the code is significantly easier than having it verified and tested. See specifically: http://www.sqlite.org/th3.html http://www.sqlite.org/th3.html "SQLite itself is in the public domain and can be used for any purpose. But TH3 is proprietary and requires a license. Members of the SQLite Consortium get free and unlimited access to TH3. Others can contact the SQLite developers for information on how to obtain a license to access and use TH3."
michaelmcmillan 12y ago
Does anyone know how long time it takes to run these tests?
- SQLite 12y ago
  The minimum set of tests needed to provide 100% branch coverage takes about 3 minutes on a fast machine. To run all of the tests under various compile-time options and on all supported platforms usually takes two to three days, assuming everything works. A full-up automated test on just Linux x64 takes about 18 hours.
  - delinka 12y ago
    Does this include the tests that run on other database servers?
    - SQLite 12y ago
      The sqllogictest suite (https://www.sqlite.org/sqllogictest https://www.sqlite.org/sqllogictest) only has to be run on all the various servers when the tests themselves change, in order to validate the tests. Otherwise, those tests only need to be run on the system under test, since the tests themselves contain the correct answer to check against. The sqllogictest suite takes about 15 minutes to run three times against SQLite (once with all optimizations turned on, and twice more with various optimizations turned off) on a fast machine.

59 points by deegles 12y ago

Previous discussions: https://hn.algolia.com/?query=http:%2F%2Fwww.sqlite.org%2Ftesting.html&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=http:%2F%2Fwww.sqlite.org%2Fte...

NelsonMinar 12y ago

I wonder if this test suite will be the last known use of TCL in modern code?

deleted 12y ago
[deleted]
hobo_mark 12y ago
At least in chip design, and Electronic Design Automation in general, TCL is still heavily used.
kijin 12y ago
AFAIK the test suite for Redis is also written in TCL.
- bch 12y ago
  Indeed - both antirez and drh are big Tcl users. drh is a Tcl Core Team (TCT) member emeritus, and antirez is responsible for this Tcl love letter: http://antirez.com/articoli/tclmisunderstood.html http://antirez.com/articoli/tclmisunderstood.html
shanemhansen 12y ago
openacs [http://openacs.org/ http://openacs.org/] is still alive. Openacs was my first web development framework. My first ever database training was from Greenspun's "TCL for web nerds" book. Those are some fond memories.
mzs 12y ago
Does expect count?
bch 12y ago
Tealeaf[0] collectors engine is Tcl on Linux. F5 and A10 appliances are Tcl at their core (see this[1] for whether they'd do it again). Cisco still embeds Tcl as far as I know. Fidessa[2] trading systems are Tcl[3][4], and on and on and on. [0] http://www-01.ibm.com/software/info/tealeaf/ http://www-01.ibm.com/software/info/tealeaf/ [1] https://devcentral.f5.com/articles/irules-concepts-tcl-the-how-and-why https://devcentral.f5.com/articles/irules-concepts-tcl-the-h... [2] https://www.fidessa.com/about-us https://www.fidessa.com/about-us [3] https://www.fidessa.com/products/training/training-courses https://www.fidessa.com/products/training/training-courses [4] http://www.services.fidessa.com/brochures/_training/VF043.pdf http://www.services.fidessa.com/brochures/_training/VF043.pd... EDIT: I only need to reference a footnote 1x, but need to reference ea. footnote once., update Fidessa references.

antonios 12y ago

This post is the most reliable karma generator I've seen on HN.

a3_nm 12y ago

It's interesting that, even though sqlite is free software (it is even public domain), some of the tests are proprietary: https://www.sqlite.org/th3.html https://www.sqlite.org/th3.html. I wonder what's the goal of this...

acqq 12y ago
How about the goal of allowing the author to actually earn something?
- hueving 12y ago
  And how exactly would the author do that?
  - acqq 12y ago
    The author is selling the support to the companies that can request some changes. Having the best tests, he can claim that the modifications he introduces are "production ready" and charge for that too. Others can just say "we changed something and it works maybe." Just writing the code is significantly easier than having it verified and tested. See specifically: http://www.sqlite.org/th3.html http://www.sqlite.org/th3.html "SQLite itself is in the public domain and can be used for any purpose. But TH3 is proprietary and requires a license. Members of the SQLite Consortium get free and unlimited access to TH3. Others can contact the SQLite developers for information on how to obtain a license to access and use TH3."

michaelmcmillan 12y ago

Does anyone know how long time it takes to run these tests?

SQLite 12y ago
The minimum set of tests needed to provide 100% branch coverage takes about 3 minutes on a fast machine. To run all of the tests under various compile-time options and on all supported platforms usually takes two to three days, assuming everything works. A full-up automated test on just Linux x64 takes about 18 hours.
- delinka 12y ago
  Does this include the tests that run on other database servers?
  - SQLite 12y ago
    The sqllogictest suite (https://www.sqlite.org/sqllogictest https://www.sqlite.org/sqllogictest) only has to be run on all the various servers when the tests themselves change, in order to validate the tests. Otherwise, those tests only need to be run on the system under test, since the tests themselves contain the correct answer to check against. The sqllogictest suite takes about 15 minutes to run three times against SQLite (once with all optimizations turned on, and twice more with various optimizations turned off) on a fast machine.