3 ms·

If you are in a tribe not hating on pp-macros, you might find this approach for testing/debugging data structures (or even the surrounding pp-based "CTL"-like C

by cb321 2y ago

If you are in a tribe not hating on pp-macros, you might find this approach for testing/debugging data structures (or even the surrounding pp-based "CTL"-like C templating idea or abstract ways of dealing with the BST zoo) interesting: https://github.com/c-blake/bst/blob/main/test/bst-interact.c Essentially, you just write a very simple (some might say trivial) interactive command-shell that can include an auto-checking upon edit of (in that case tree) invariants keyed-off an environment variable. The only missing piece is some little tool to generate random inserts/deletes/etc. If the micro-shell has a pretty printer then it is easy to "replay" any failed series of edits with some "p" commands before & after invariant failures.

eqvinox 2y ago
This is incredibly funny to me, because that's (almost) exactly how some of our other tests work :D (Almost: we have a full-featured command shell and just use that for testing) Shell bindings: https://github.com/FRRouting/frr/blob/master/tests/ospf6d/test_lsdb.c https://github.com/FRRouting/frr/blob/master/tests/ospf6d/te... Input: https://github.com/FRRouting/frr/blob/master/tests/ospf6d/test_lsdb.in https://github.com/FRRouting/frr/blob/master/tests/ospf6d/te... Expected output: https://github.com/FRRouting/frr/blob/master/tests/ospf6d/test_lsdb.refout https://github.com/FRRouting/frr/blob/master/tests/ospf6d/te... Absolutely agree this is a great option to have for some kinds of tests.
- cb321 2y ago
  One reason to have a razor thin custom command shell (perhaps obeying similar conventions across data structures..) is that the parsing for such can be so fast/consistent that you can also use it for benchmarking/perf regression testing with a workload generator. You might also find & record "anomalous" workloads that way or write a little "translator" (some might say "compiler") from a log to such a workload or etc. I have done this for data structures even as fast as integer-keyed hash tables (though in that hyper-fast case you might need to try to measure & subtract off parser-loop/IO dispatch overhead and/or need statistical testing for "actually even different at all", perhaps along the lines of https://github.com/c-blake/bu/blob/main/doc/tim.md https://github.com/c-blake/bu/blob/main/doc/tim.md).

by cb321 2y ago

eqvinox 2y ago

This is incredibly funny to me, because that's (almost) exactly how some of our other tests work :D (Almost: we have a full-featured command shell and just use that for testing) Shell bindings: https://github.com/FRRouting/frr/blob/master/tests/ospf6d/test_lsdb.c https://github.com/FRRouting/frr/blob/master/tests/ospf6d/te... Input: https://github.com/FRRouting/frr/blob/master/tests/ospf6d/test_lsdb.in https://github.com/FRRouting/frr/blob/master/tests/ospf6d/te... Expected output: https://github.com/FRRouting/frr/blob/master/tests/ospf6d/test_lsdb.refout https://github.com/FRRouting/frr/blob/master/tests/ospf6d/te... Absolutely agree this is a great option to have for some kinds of tests.

cb321 2y ago
One reason to have a razor thin custom command shell (perhaps obeying similar conventions across data structures..) is that the parsing for such can be so fast/consistent that you can also use it for benchmarking/perf regression testing with a workload generator. You might also find & record "anomalous" workloads that way or write a little "translator" (some might say "compiler") from a log to such a workload or etc. I have done this for data structures even as fast as integer-keyed hash tables (though in that hyper-fast case you might need to try to measure & subtract off parser-loop/IO dispatch overhead and/or need statistical testing for "actually even different at all", perhaps along the lines of https://github.com/c-blake/bu/blob/main/doc/tim.md https://github.com/c-blake/bu/blob/main/doc/tim.md).