5 ms·

Elegant tests with Truth Tables

28 points by pcreux 11y ago

brentvatne 11y ago
This type of functionality is built into cljs.test, for example: https://github.com/r0man/cljs-http/blob/master/test/cljs_http/test/util.cljs#L25-L73 https://github.com/r0man/cljs-http/blob/master/test/cljs_htt...
dward 11y ago
Seems more appropriate to call these Table Driven Tests: https://github.com/golang/go/wiki/TableDrivenTests https://github.com/golang/go/wiki/TableDrivenTests As they are not truth tables: http://en.wikipedia.org/wiki/Truth_table http://en.wikipedia.org/wiki/Truth_table
kjhughes 11y ago
The name "Truth Tables" in already firmly established in logic to refer to enumerations of all combinations of boolean inputs and outputs. I think a better name for these testing constructs would be Expectation Tables.
- pcreux 11y ago
  Naming is hard! Thanks for suggesting "Expectation Tables".
  - kjhughes 11y ago
    True, naming is hard. But coming from an EE/CE digital logic background, if it doesn't have 1's and 0's (or T's and F's) in it, it just ain't a Truth Table to me. Of course I could just as easily be criticized for "Expectation Tables" by someone in obstetrics, so what's fair is fair. I do like your design. Nice job.
nerdy 11y ago
This is a common practice and more often than not the data is reused, so it is provided by a "data provider", a method living in the test code, which returns a structure with inputs and expected results. For example, in PHPUnit: https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers https://phpunit.de/manual/current/en/writing-tests-for-phpun... It would actually be even more elegant if the data where extracted out of the test methods in my opinion. Though it would effectively double the number of methods, it cleans things up significantly within the test methods, provides an outlet for extension and the only expense is the addition of declaration & closing lines.
seivan 11y ago
I remember Gherkin (Cucumber for Ruby) had this! It does beat having a ton of assertions.
- kedean 11y ago
  You have that backwards. Gherkin is a spec language for writing tests, Cucumber is a library for Ruby and Java that interprets Gherkin specifications.
  - seivan 11y ago
    How is that backwards, that is essentially what I said :-/ ?
krishicks 11y ago
A more advanced version of this can be seen in the some of the tests for CloudFoundry's new Diego runtime: https://github.com/cloudfoundry-incubator/runtime-schema/blob/master/bbs/lrp_bbs/evacuation_test.go#L17 https://github.com/cloudfoundry-incubator/runtime-schema/blo... The 'table' consists of an array of calls to constructors of structs that contain fields which are used to set up the context for each test, set value expectations specific to that test, and whether certain tests in shared examples apply to that context. There is a certain level of cognitive overhead involved in understanding this method of writing tests, to be sure, but the benefits (concise expression of a huge matrix of variations) outweigh the costs (massive duplication, and therefore comprehension of thousands of more lines of test code). It's important to note that this style of test is really best for the situation where there is a large matrix of things you'd like to test. The example I linked included ~70 different contexts, each of which is being run through the large Test() function at the bottom: https://github.com/cloudfoundry-incubator/runtime-schema/blob/master/bbs/lrp_bbs/evacuation_test.go#L784 https://github.com/cloudfoundry-incubator/runtime-schema/blo...
kellyfelkins 11y ago
I love the concept of table driven tests. I created a gem so that you could put that data into tables -- called ATV for Ascii Table Values. Just more of the same idea and I think the resulting tests are easier to understand. Good writeup. Please take a look at my article: http://technology.indiegogo.com/2015/01/ascii-tables-for-clearer-testing/ http://technology.indiegogo.com/2015/01/ascii-tables-for-cle... And the gem: https://github.com/IndieGoGo/atv https://github.com/IndieGoGo/atv +------+-------+ | name | Kelly | +------+-------+

28 points by pcreux 11y ago

This type of functionality is built into cljs.test, for example: https://github.com/r0man/cljs-http/blob/master/test/cljs_http/test/util.cljs#L25-L73 https://github.com/r0man/cljs-http/blob/master/test/cljs_htt...

dward 11y ago

Seems more appropriate to call these Table Driven Tests: https://github.com/golang/go/wiki/TableDrivenTests https://github.com/golang/go/wiki/TableDrivenTests As they are not truth tables: http://en.wikipedia.org/wiki/Truth_table http://en.wikipedia.org/wiki/Truth_table

kjhughes 11y ago

The name "Truth Tables" in already firmly established in logic to refer to enumerations of all combinations of boolean inputs and outputs. I think a better name for these testing constructs would be Expectation Tables.

pcreux 11y ago
Naming is hard! Thanks for suggesting "Expectation Tables".
- kjhughes 11y ago
  True, naming is hard. But coming from an EE/CE digital logic background, if it doesn't have 1's and 0's (or T's and F's) in it, it just ain't a Truth Table to me. Of course I could just as easily be criticized for "Expectation Tables" by someone in obstetrics, so what's fair is fair. I do like your design. Nice job.

nerdy 11y ago

This is a common practice and more often than not the data is reused, so it is provided by a "data provider", a method living in the test code, which returns a structure with inputs and expected results. For example, in PHPUnit: https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers https://phpunit.de/manual/current/en/writing-tests-for-phpun... It would actually be even more elegant if the data where extracted out of the test methods in my opinion. Though it would effectively double the number of methods, it cleans things up significantly within the test methods, provides an outlet for extension and the only expense is the addition of declaration & closing lines.

seivan 11y ago

I remember Gherkin (Cucumber for Ruby) had this! It does beat having a ton of assertions.

kedean 11y ago
You have that backwards. Gherkin is a spec language for writing tests, Cucumber is a library for Ruby and Java that interprets Gherkin specifications.
- seivan 11y ago
  How is that backwards, that is essentially what I said :-/ ?

krishicks 11y ago

A more advanced version of this can be seen in the some of the tests for CloudFoundry's new Diego runtime: https://github.com/cloudfoundry-incubator/runtime-schema/blob/master/bbs/lrp_bbs/evacuation_test.go#L17 https://github.com/cloudfoundry-incubator/runtime-schema/blo... The 'table' consists of an array of calls to constructors of structs that contain fields which are used to set up the context for each test, set value expectations specific to that test, and whether certain tests in shared examples apply to that context. There is a certain level of cognitive overhead involved in understanding this method of writing tests, to be sure, but the benefits (concise expression of a huge matrix of variations) outweigh the costs (massive duplication, and therefore comprehension of thousands of more lines of test code). It's important to note that this style of test is really best for the situation where there is a large matrix of things you'd like to test. The example I linked included ~70 different contexts, each of which is being run through the large Test() function at the bottom: https://github.com/cloudfoundry-incubator/runtime-schema/blob/master/bbs/lrp_bbs/evacuation_test.go#L784 https://github.com/cloudfoundry-incubator/runtime-schema/blo...

kellyfelkins 11y ago

I love the concept of table driven tests. I created a gem so that you could put that data into tables -- called ATV for Ascii Table Values. Just more of the same idea and I think the resulting tests are easier to understand. Good writeup. Please take a look at my article: http://technology.indiegogo.com/2015/01/ascii-tables-for-clearer-testing/ http://technology.indiegogo.com/2015/01/ascii-tables-for-cle... And the gem: https://github.com/IndieGoGo/atv https://github.com/IndieGoGo/atv +------+-------+ | name | Kelly | +------+-------+