3 ms·
Does anyone have tips on how to speed up test collection in pytest? In our django app (~5000 tests) it takes about 30s to run a single test with `-k testname` w
by cocoflunchy 4y ago
Does anyone have tips on how to speed up test collection in pytest? In our django app (~5000 tests) it takes about 30s to run a single test with `-k testname` when the actual time spent in the test is on the order of a few hundred milliseconds.
It hangs forever on the `collecting ...` phase.
- dmurray 4y agoRun the single test with "pytest tests/path/to/test_file.py::TestClass::test_method" instead of using pytest -k to parse all your test code looking for a method with the right name. Alternatively, your IDE or editor probably already has an extension that allows you to right click - run single test. VS Code at least has some plugin that collects the tests in the background.
- ssl232 4y agoTo add to this, the test ID pytest prints when a test has had a failure/warning is the same form you can call with pytest to run the test. For example, let's say pytest outputs this: =============================== warnings summary =============================== tests/parser/test_expressions.py::test_complex[+inf+1.23e7j-(inf+12300000j)] You can run just that particular test (with that particular parametrized input) with pytest tests/parser/test_expressions.py::test_complex[+inf+1.23e7j-(inf+12300000j)]
- BerislavLopac 4y agoTo add to that, pytest's parametrized IDs can be fully controlled in the test code [0], so your test can be something like: pytest tests/parser/test_expressions.py::test_complex[special_case_7] [0] https://docs.pytest.org/en/7.1.x/example/parametrize.html#different-options-for-test-ids https://docs.pytest.org/en/7.1.x/example/parametrize.html#di...
- masklinn 4y agoAlso pytest has flags like —lf and —sw to rerun failed tests or tackle test suites with lots of failures.
- cocoflunchy 4y agoThanks, it is a lot better (11s instead of 30)
- Jackevansevo 4y agoTimes I've profiled large tests suites / codebases in the past, this has been entirely down to import overheads. Quite a difficult thing to fix ... I'm hoping this PEP Proposal for lazy imports might solve things: https://peps.python.org/pep-0690/ https://peps.python.org/pep-0690/ Try profiling pytest with something like pyinstrument[1] and make sure you invoke it from python directly[2] with `--collect-only` as an argument. You'll essentially then just be profiling the import overheads before tests are ran. [1] - https://pyinstrument.readthedocs.io/en/latest/guide.html#profile-a-specific-chunk-of-code https://pyinstrument.readthedocs.io/en/latest/guide.html#pro... [2] - https://pyinstrument.readthedocs.io/en/latest/guide.html#profile-a-specific-chunk-of-code https://pyinstrument.readthedocs.io/en/latest/guide.html#pro...
- cocoflunchy 4y agoThanks, pyinstrument is really nice. You're right it looks like imports are taking a lot of time...
- rpep 4y agoWith a Django project, what it is actually doing is spinning up a new test database at this point which will be the cause of your slowness. If you haven’t made database schema changes you can run it with the flag “—reuse-db”. If you’ve got a large project that’s been going for a while, squashing migrations helps too. It can be a bit tricky to do this though.
- cocoflunchy 4y agoThanks, I run it with --nomigrations so it's not the migrations. Reusing db shaves about 2 seconds (I'm down to ~9 seconds when running a single test and specifying the full path)