4 ms·

We work with a rather large upstream scientific codebase in python. They have an innate distrust of anything that isn't written by them. Their testing system d

We work with a rather large upstream scientific codebase in python. They have an innate distrust of anything that isn't written by them. Their testing system depends on tests printing "OK" after every test. This means that in many cases, tests failing are indicated by the _absence_ of "OK" being printed. (We've attempted to isolate those parts and write our own stuff testing against upstream in pytest. We once presented a proposal to move them to pytest, offering to do any work and even wrote pytest plugins to seamlessly integrate with their current system. We got a - literal - "Thanks, but no thanks.")

popularonion 5y ago
That’s not the way I would do it, but is there really a problem here? Assuming the library itself isn’t littered with print statements that cause false positives. Experience has taught me that the “right” testing framework for a project is whatever the developers are happy and productive with.
chriswarbo 5y ago
> Their testing system depends on tests printing "OK" after every test. Oof. If they'd instead put "ok" before every test, they might have been accidentally compatible with TAP! https://testanything.org https://testanything.org
- rightbyte 5y ago
  I think relying on stout is kinda fine. I do tests with TCL/Expect like that. It is nice to not have to depend on the language runtime to do the test.
danuker 5y ago
There is a point where enough mismanagement makes relying on the said scientific codebase a liability rather than an advantage. You could even be a victim of "Embrace, Extend, Extinguish". My advice is to consider forking it, and poaching contributors, in the interests of common good.
oli5679 5y ago
Did they catch exceptions and then print 'not ok'?
- danuker 5y ago
  > tests failing are indicated by the _absence_ of "OK" being printed I guess they are counting OKs then bisect their test suite until they find the "not ok" test.
adenozine 5y ago
And you’re unwilling to name the library? I’d love to investigate this further.

popularonion 5y ago

That’s not the way I would do it, but is there really a problem here? Assuming the library itself isn’t littered with print statements that cause false positives. Experience has taught me that the “right” testing framework for a project is whatever the developers are happy and productive with.

chriswarbo 5y ago

> Their testing system depends on tests printing "OK" after every test. Oof. If they'd instead put "ok" before every test, they might have been accidentally compatible with TAP! https://testanything.org https://testanything.org

rightbyte 5y ago
I think relying on stout is kinda fine. I do tests with TCL/Expect like that. It is nice to not have to depend on the language runtime to do the test.

danuker 5y ago

There is a point where enough mismanagement makes relying on the said scientific codebase a liability rather than an advantage. You could even be a victim of "Embrace, Extend, Extinguish". My advice is to consider forking it, and poaching contributors, in the interests of common good.

oli5679 5y ago

Did they catch exceptions and then print 'not ok'?

danuker 5y ago
> tests failing are indicated by the _absence_ of "OK" being printed I guess they are counting OKs then bisect their test suite until they find the "not ok" test.

adenozine 5y ago

And you’re unwilling to name the library? I’d love to investigate this further.