17 ms·
> a test that fails if a new API endpoint isn't mentioned [...] That's a great idea. A related thought I had was testing assertions about architecture by look
by dllthomas 5y ago
> a test that fails if a new API endpoint isn't mentioned [...]
That's a great idea. A related thought I had was testing assertions about architecture by looking at the graph of imports or calls.
From your blog post,
> if a change doesn’t update the relevant documentation, point that out in your review!
What you get at but don't seem to state explicitly is that finding the relevant documentation is hard, for both the author and reviewer. Finding any relevant documentation should be easy, but ideally we're finding all relevant documentation, and we need to be reaching sufficient confidence that we're missing little enough that we're able to hold back entropy enough to keep the docs useful.
Your tools address this for some cases! Doctest addresses this for other cases. From TFA here, it sounds like Swimm.io tries to address this for more cases (my gut says the article oversells it but I intend to look more closely).
To get further, an idea I've been toying with is to treat claims (implicit or explicit) in documentation as requiring citation, pointing not at sources but at tests. Ideally the test runner, when a test fails, can then surface all references to that test. In addition to highlighting portions of the docs that may need to change, this seems likely to also provide crucial context when fixing the code and/or the test.