2 ms·

Barely tolerable. At one point, the Linux selftests were contemplating using TAP. I read the main example: 1..4 ok 1 - Input file opened not ok 2

by amluto 3y ago

Barely tolerable. At one point, the Linux selftests were contemplating using TAP. I read the main example: 1..4 ok 1 - Input file opened not ok 2 - First line of the input valid ok 3 - Read the rest of the file not ok 4 - Summarized correctly # TODO Not written yet and I had two immediate objections. a) Shouldn't tests have names? b) (more serious) To generate valid TAP output, I need to know up-front how many tests there are. What if I can run the tests in sequence but I don't know how to count them? This can happen if there is a tree of tests, where each node knows what children it has but not what children they have. There's no credible way to generate TAP output streamily, even if there is no parallelization of tests. No thanks.

duskwuff 3y ago
> a) Shouldn't tests have names? That's what the text following the test number is for. > b) (more serious) To generate valid TAP output, I need to know up-front how many tests there are. What if I can run the tests in sequence but I don't know how to count them? This can happen if there is a tree of tests, where each node knows what children it has but not what children they have. Typically you'd use subtests for this, e.g. 1..1 # Subtest: root 1..2 Subtest: branch 1 1..2 ok 1 - leaf 1 ok 2 - leaf 2 ok 1 - branch 1 ok 2 - leaf 3 ok 1 - root But if you really didn't know the number of tests up front for some reason, the test plan is also allowed to appear at the end of the file.
- stevekemp 3y ago
  I wrote a simple set of test-cases for my lisp implementation, and use the TAP format for the output. In my case I pipe to "tapview", by ESR, to visualize the results. The raw output is otherwise pretty basic: TAP version 14 ok tst:zero:1 ok tst:zero:2 .. ok union:1 ok union:2 ok year:len 1..215 Tapview is pretty portable, and works well as a simple output visualizer turning my view into: ./yal examples/lisp-tests.lisp | _misc/tapview ........................................................... 215 tests, 0 failures.
spc476 3y ago
a) They can have names (internally). At my previous job, our test suite required every test be a single file, and yet it still output TAP (you specified which tests you wanted via filenames). b) No, you don't have to know prior how many tests will run. If you read the spec, you'll see there's an option to print the final count at the end.

by amluto 3y ago

duskwuff 3y ago

> a) Shouldn't tests have names? That's what the text following the test number is for. > b) (more serious) To generate valid TAP output, I need to know up-front how many tests there are. What if I can run the tests in sequence but I don't know how to count them? This can happen if there is a tree of tests, where each node knows what children it has but not what children they have. Typically you'd use subtests for this, e.g. 1..1 # Subtest: root 1..2 Subtest: branch 1 1..2 ok 1 - leaf 1 ok 2 - leaf 2 ok 1 - branch 1 ok 2 - leaf 3 ok 1 - root But if you really didn't know the number of tests up front for some reason, the test plan is also allowed to appear at the end of the file.

stevekemp 3y ago
I wrote a simple set of test-cases for my lisp implementation, and use the TAP format for the output. In my case I pipe to "tapview", by ESR, to visualize the results. The raw output is otherwise pretty basic: TAP version 14 ok tst:zero:1 ok tst:zero:2 .. ok union:1 ok union:2 ok year:len 1..215 Tapview is pretty portable, and works well as a simple output visualizer turning my view into: ./yal examples/lisp-tests.lisp | _misc/tapview ........................................................... 215 tests, 0 failures.

spc476 3y ago

a) They can have names (internally). At my previous job, our test suite required every test be a single file, and yet it still output TAP (you specified which tests you wanted via filenames). b) No, you don't have to know prior how many tests will run. If you read the spec, you'll see there's an option to print the final count at the end.