4 ms·

Ward: Modern Test Framework for Python

tjpnz 5y ago
Not knocking this but what exactly is meant by "modern"?
- petters 5y ago
  Maybe (a) no test classes inheriting from Test case, (b) asyncio? It's a good question!
- spooneybarger 5y ago
  "recent" and doing "something new".
- say_it_as_it_is 5y ago
  Modern in tech usually is synonymous with new, but the authors also try to associate it with something better than established approaches. In the last ten years, many modern technologies have failed at the latter and instead offer something different with tradeoffs. No more.
anentropic 5y ago
I love the look of that output in the first screenshot... will be giving this a go :) I'm not so keen on the way that parameterised tests are defined though. Having a separate `each(...)` call per kwarg, which are then expected to have the same number of elements, feels a bit off. If I'd guessed how that would work before reading the docs I would have expected that form to test the product of those arg value sets against each other.
- nsdarren 5y ago
  Author of Ward here! Thanks for the feedback. It's a common piece of feedback I hear and I do agree. I haven't documented this yet, but due to the way Ward works you can actually write your tests inside a loop in order to parameterise them (see below). This is a little more explicit, and lets you build up your test data using things like itertools (if you like): for lhs, rhs, res in [ (1, 1, 2), (2, 3, 5), ]: @test("simple addition") def _(left=lhs, right=rhs, result=res): assert left + right == result I'm considering how/whether to add a pytest style decorator for parameterisation too. If you give Ward a try or just have any other general feedback, please let me know if you have any issues/suggestions on GitHub.
  - O5vYtytb 5y ago
    How do you represent the test ID for complex arguments? In Pytest, that's easily done with `pytest.param`: @pytest.parametrize("a,b", [ pytest.param(..., id="thing1"), pytest.param(..., id="thing2"), ]) def test_foo(a, b): ...
    - nsdarren 5y ago
      Probably the closest thing would be to add a unique tag to each instance of the test: for lhs, rhs, res, tag in [ (1, 1, 2, "thing1"), (2, 3, 5, "thing2"), (3, 4, 7, "thing3"), ]: @test("simple addition", tags=[tag]) def _(left=lhs, right=rhs, result=res): assert left + right == result You can then select individual tests using something like `ward --tags thing2` to only run tests with that tag (in my example, the 2nd of 3 tests). Since multiple tests can share the same tag in Ward, if you wanted to narrow things down to ensure you're only selecting from the tests generated in this loop you could do something like `ward --search 'simple addition' --tags thing2`. You can also select multiple instances at once with `ward --tags 'thing1 or thing3'`.
  - anentropic 5y ago
    FWIW I think this interface is slightly nicer than the pytest one https://pypi.org/project/parameterized/ https://pypi.org/project/parameterized/
- rgacote 5y ago
  Some of my tests have dozens of parameter sets with a half-dozen parameters. Getting those all aligned with individual 'each' statements would be interesting.
eirki 5y ago
Absolutely love the idea of using default params to identify fixtures. That part of pytest has always rubbed me the wrong way.
- petters 5y ago
  Yes, pytest makes navigating automatically from a test to the fixture it uses hard for some IDEs. It is also a bit too magical.
  - eirki 5y ago
    And some linters will complain if there is a collision between the names of a parameter and a function.
- nsdarren 5y ago
  Thanks :) That's one of my biggest gripes with pytest too, and was one of the main things I was looking to solve.
je42 5y ago
i wonder how does this compare to pytest ?
je42 5y ago
Just saw https://github.com/darrenburns/pytest-clarity https://github.com/darrenburns/pytest-clarity and also pytest fork on Darren's github profile. @Darren, did you consider extending/improving pytest over starting from scratch? if yes, what were you biggest drivers to start a new test framework
aequitas 5y ago
One of my first questions was how it compares to Pytest, that is answered here: https://wardpy.com/guide/pytest-comparison https://wardpy.com/guide/pytest-comparison

Not knocking this but what exactly is meant by "modern"?

petters 5y ago
Maybe (a) no test classes inheriting from Test case, (b) asyncio? It's a good question!
spooneybarger 5y ago
"recent" and doing "something new".
say_it_as_it_is 5y ago
Modern in tech usually is synonymous with new, but the authors also try to associate it with something better than established approaches. In the last ten years, many modern technologies have failed at the latter and instead offer something different with tradeoffs. No more.

I love the look of that output in the first screenshot... will be giving this a go :) I'm not so keen on the way that parameterised tests are defined though. Having a separate `each(...)` call per kwarg, which are then expected to have the same number of elements, feels a bit off. If I'd guessed how that would work before reading the docs I would have expected that form to test the product of those arg value sets against each other.

nsdarren 5y ago
Author of Ward here! Thanks for the feedback. It's a common piece of feedback I hear and I do agree. I haven't documented this yet, but due to the way Ward works you can actually write your tests inside a loop in order to parameterise them (see below). This is a little more explicit, and lets you build up your test data using things like itertools (if you like): for lhs, rhs, res in [ (1, 1, 2), (2, 3, 5), ]: @test("simple addition") def _(left=lhs, right=rhs, result=res): assert left + right == result I'm considering how/whether to add a pytest style decorator for parameterisation too. If you give Ward a try or just have any other general feedback, please let me know if you have any issues/suggestions on GitHub.
- O5vYtytb 5y ago
  How do you represent the test ID for complex arguments? In Pytest, that's easily done with `pytest.param`: @pytest.parametrize("a,b", [ pytest.param(..., id="thing1"), pytest.param(..., id="thing2"), ]) def test_foo(a, b): ...
  - nsdarren 5y ago
    Probably the closest thing would be to add a unique tag to each instance of the test: for lhs, rhs, res, tag in [ (1, 1, 2, "thing1"), (2, 3, 5, "thing2"), (3, 4, 7, "thing3"), ]: @test("simple addition", tags=[tag]) def _(left=lhs, right=rhs, result=res): assert left + right == result You can then select individual tests using something like `ward --tags thing2` to only run tests with that tag (in my example, the 2nd of 3 tests). Since multiple tests can share the same tag in Ward, if you wanted to narrow things down to ensure you're only selecting from the tests generated in this loop you could do something like `ward --search 'simple addition' --tags thing2`. You can also select multiple instances at once with `ward --tags 'thing1 or thing3'`.
- anentropic 5y ago
  FWIW I think this interface is slightly nicer than the pytest one https://pypi.org/project/parameterized/ https://pypi.org/project/parameterized/
rgacote 5y ago
Some of my tests have dozens of parameter sets with a half-dozen parameters. Getting those all aligned with individual 'each' statements would be interesting.

eirki 5y ago

Absolutely love the idea of using default params to identify fixtures. That part of pytest has always rubbed me the wrong way.

petters 5y ago
Yes, pytest makes navigating automatically from a test to the fixture it uses hard for some IDEs. It is also a bit too magical.
- eirki 5y ago
  And some linters will complain if there is a collision between the names of a parameter and a function.
nsdarren 5y ago
Thanks :) That's one of my biggest gripes with pytest too, and was one of the main things I was looking to solve.

je42 5y ago

i wonder how does this compare to pytest ?

je42 5y ago

Just saw https://github.com/darrenburns/pytest-clarity https://github.com/darrenburns/pytest-clarity and also pytest fork on Darren's github profile. @Darren, did you consider extending/improving pytest over starting from scratch? if yes, what were you biggest drivers to start a new test framework

aequitas 5y ago

One of my first questions was how it compares to Pytest, that is answered here: https://wardpy.com/guide/pytest-comparison https://wardpy.com/guide/pytest-comparison