8 ms·
Ask HN: What is your preferred Python 3 testing framework?
- bpicolo 9y agopytest: https://docs.pytest.org/en/latest/ https://docs.pytest.org/en/latest/ The dependency injection for fixtures is somewhat of a magical entity, but overall I've found it's the most efficient way to hammer out good tests on the standard unit/integration test spectrum. The default mode of operation doesn't even require importing pytest: Just write files named ending with `_test.py`, functions starting with `test`, and bare bones assertions. `yield_fixture` works well with the typical mock-and-assert-on python test pattern. Plenty of plugins out there for more exotic features, like asyncio, parallelization (though test parallelization tends to be much more problematic beyond "run tests at the same time"), output to <format>, etc too.
- orf 9y agoJust an FYI, normal fixtures can use 'yield', there is no need to use 'yield_fixture' anymore
- bpicolo 9y agoHah, good to know. Maybe I haven't bumped pytest in a while.
- sametmax 9y agoPretty much anybody with big modern project i know in the industry is using pytest. It make the barrier of entry of writting tests much lower. Given how annoying it is to write test, i'd say making it even a tiny bit easier is a win. Plus fixtures scale better than tearUp and down.
- allover 9y agoHmm I'm in the camp that believes setUp/tearDown and fixtures are not only equivalent, but both antipatterns. This article explains it better than I can [1] [1] https://robots.thoughtbot.com/lets-not https://robots.thoughtbot.com/lets-not
- bpicolo 9y agopytest fixtures don't have that issue, as they're only run when you include the fixture for DI
- allover 9y ago> pytest fixtures don't have that issue, as they're only run when you include the fixture for DI I don't see how that prevents any of the basic problems raised in the article.
- makmanalp 9y agopy.test avoids XUnit or unittest style class-based tests for this reason (though it will run them if you insist). It opts for simple test functions instead. Fixtures are passed in explicitly as parameters rather than implicitly as class members or whatnot. The result of all this is quite similar to what the author proposes.
- drcongo 9y agoPytest with Mixer [1] really lowers the barrier to entry. [1] https://mixer.readthedocs.io https://mixer.readthedocs.io
- gbog 9y agoMy favorite feature of pytest is pytest.mark.parametrize, which makes it easy to do table driven testing.
- amirouche 9y agowhat is "table driven testing"?
- makmanalp 9y agoI've never heard of that term being used but parametrize helps in situations where: a) You have a slew of test inputs that all need to be tested through the same function, but you don't want to duplicate code. b) You want something like "test with every combination of [x,y,z] for parameter a and [j,k,l] for parameter b". This is probably what the grandparent is referring to. c)You have an even more complicated scheme, which you can define. This is better than just having a for loop over a pre-generated list of parameters and calling a function with the same assertions, because pytest sees and handles them as separate but related test cases (e.g. when reporting errors, crash handling, fixtures, etc)
- gbog 9y agoYes. But what's striking me is that one woud ever want to test only one input state. If I have to test function f(x) and see if its output is as expected, I always want to test many inputs (including "silly" ones like wrong type, nulls, extremes). Writing one test for each of them is absurd, just give a list of inputs, a list of expected outputs and check them all.
- naspeh 9y agoAlso pytest have pytest-bdd[1] for BDD, with that in mind you can use pytest for unit tests and behave tests, with only one runner is just awesome (because you write your fixtures only once) Also good to mention pytest-xdist[2] for distributed testing... [1]: https://pypi.python.org/pypi/pytest-bdd https://pypi.python.org/pypi/pytest-bdd [2]: https://docs.pytest.org/en/3.0.0/xdist.html https://docs.pytest.org/en/3.0.0/xdist.html
- makmanalp 9y agoI'll enthusiastically second py.test. It's well documented, well supported, and well thought out. More importantly though, the details are just brilliantly done. For example: showing the values of local variables in an error traceback always saves me a ton of time. I don't know why this isn't the default option. Alternatively, you can pass --pdb to just run the debugger exactly when a test crashes, so you can inspect the contents. Another example is how it prints out details on assertions (e.g. "the left was this and the right was this") and not just "AssertionError". On top of that, for complex data structures, it will diff them and point out exactly what parts of the structure are matching and what parts are not matching, which again saves a ton of time from firing up a debugger and doing it by hand. For more esoteric options, I like this pycon talk: https://www.youtube.com/watch?v=jmsk1QZQEvQ https://www.youtube.com/watch?v=jmsk1QZQEvQ
- mylons 9y agopytest all day
- __s 9y agoimport unittest Used for CI in https://github.com/serprex/aespython https://github.com/serprex/aespython
- orf 9y agoPytest all the way.
- Walkman 9y agoHow is this a question? Pytest is so much better and more used than any other testing framework you don't even have to think about it.
- 0x54MUR41 9y agoThe company I work for uses standard unittest [0] library from Python. This library helps you to test defined functions in your Python program. For each defined function, you can set different cases to test (positive and negative cases). We also use coverage [1] to see code coverage and report. Report is usually reported to HTML because we can check the missing part of code (the flow of our logic program). [0]: https://docs.python.org/3/library/unittest.html https://docs.python.org/3/library/unittest.html [1]: https://pypi.python.org/pypi/coverage https://pypi.python.org/pypi/coverage
- guitarbill 9y agoThis. Simple, well documented, works for 99% of test cases. It's also how Django does it [0]. True, it has some warts, and you need to learn about e.g. `autospec`. But for better or worse, it's the "standard" way, and for me that low barrier to entry/universality is a bigger value-add than `assert` vs `self.assertEqual`, etc. [0] https://docs.djangoproject.com/en/1.11/topics/testing/ https://docs.djangoproject.com/en/1.11/topics/testing/
- orf 9y agoDjango has some different constraints than most projects, and the test suite has been around for a long long time. In most situations pyest is a superior choice that goes beyond 'assert' Vs 'self.assert*'.
- sidlls 9y agoHow is pytest superior?
- orf 9y agoIt's much more pythonic for starters, java-style camelcase is not great to look at. Fixtures are very powerful, as are parameterized tests. Assertion failures give you a lot of output to help you debug, like local variables and their values. There are a wide variety of plugins to help you test various frameworks (pytest-django, pytest-asyncio), various other plugins that plug into linters (pytest-flake8, pytest-isort). Super easy parallel tests. If you need to write something that has as few (or no) external dependencies then the standard unittest library is OK. All in all it's a bit like using urllib over requests. Sure, both work and get the job done, but one is just nicer.
- dbcurtis 9y agocurrently using good old unittest but with the addition of hypothesis. hypothesis is a real productivity booster.
- eru 9y agoUsing hypothesis makes Python feel almost like a typed language for me.
- dbcurtis 9y agoIf you want to really make Python feel like a typed language, and still be idiomatically Pythonic, learn to write constructors that raise ValueError in appropriate ways.
- geofft 9y agoIs there a good way to do that that doesn't involve a bunch of if-this-or-that-then-raise ~boilerplate in your constructors? e.g., is there some nice library that will let me put annotations like "must be a list of at least two items" or "must be an integer between 0 and 99" with a more concise syntax?
- dbcurtis 9y agoUmmm, maybe? There seem to be libraries for everything, but I do it manually. The constructors become "non-trivial", but not overly-complicated, and it pays off in simpler, more robust code elsewhere. So my code would be something like: class Foo: def __init__(self, x, y=None): if y is None: x_ = x.x y = x.y self.x = int(x_) if self.x < 0 or self.x > 99: raise ValueError('x must be in range 0..99') if len(y) < 2: raise ValueError('y must be iterable, length > 2') self.y = y Soo... You can say: f = Foo(1,['a','b']) f = Foo(1,('a','b', 3)) f = Foo("42", "bar") f2 = Foo(f) Calling int(x_) will take anything that implements __int__(), or raise ValueError. len() will take anything that implements the iterator protocol and has length >= 2, or raise. Calling Foo() on an instance of Foo will cause some excess copies, but it does have the effect of making sure you passed in something Foo-ish. Where "Foo-ish" means "has an x attribute that can be made into an integer between 0 and 99, and a y attribute that is an iterable of length >= 2". The above code lets some exceptions bubble up, but they could be wrapped in try..except to turn them into ValueError with specific messages. Note that this code accepts anything Foo-ish without calling isinstance(). Abstract base cases with isinstance() are a more suitable choice for complex cases of checking protocol conformance. So whether or not the performance impact and extra code are worth it probably varies from project to project.
- ri0t 9y agotox with py.test
- hprotagonist 9y agopytest and hypothesis.
- eru 9y agoHypothesis is a worthwhile addition to any Python testing framework. As a 'QuickCheck' implementation it even beats the original Haskell version in lots of respects.
- deleted 9y ago[deleted]
- ak217 9y agounittest with https://github.com/kislyuk/ensure https://github.com/kislyuk/ensure
- sidlls 9y agoThe built-in unittest library does the job well.
- Singletoned 9y agoAnything but pytest is good. Pytest is full of magic and can be horrendous for a newcomer to your team. You can't work out what is happening just by reading the tests, you have to know how pytest works. Also, when something breaks in pytest, all the magic means that it can be very hard to work out what is happening.
- luord 9y agoPytest, once I understood how the fixture system works, I never looked back. I wish JavaScript had something similar