3 ms·

I dislike the format itself (such is the nature of protocols/standards, that's fine) but I'm glad people managed to standardize somewhat on a format. I guess th

I dislike the format itself (such is the nature of protocols/standards, that's fine) but I'm glad people managed to standardize somewhat on a format. I guess the XML format from JUnit is also a de facto standard. I wish we had a unified format for for reporting benchmark results too. Now I think about it, I guess it should be possible to extend TAP formst to add arbitrary extra measures alongside pass/fail. I also wish testing tools would do more to force test authors to differentiate between different types of failures (the test thinks it detected a bug / the test setup failed / the test code itself divided by zero). Once your project has a corpus of tests that just "fail" it's quickly impossible to retroactively fix that problem.

eddmakesio 3y ago
> the test setup failed Most testing frameworks allow you to skip with a message if setup failed, which can help > the test code itself divided by zero I think this is a bug you do want to know and should be rare once code is merged into master (there's a bug in your code, just so happens it is test code!). However if for some reason you rely on a non-deterministic value that can make your test code fail in some way I have used Python decorators in the past to mark a test as such and raise a specific message.