3 ms·

> your example stops on the first one that fails, and does not evaluate the others after that. I don't think this is a big problem; trying to focus on multiple

> your example stops on the first one that fails, and does not evaluate the others after that. I don't think this is a big problem; trying to focus on multiple examples at once is difficult. It might be a problem if tests are slow and you are forced to work on all of them at once. But in that case I'd try to make the tests faster (getting rid of network requests, disk/DB access by faking them away or hoisting to the caller).

olex 4y ago
Hm. I think my main issue there is not the speed, but rather seeing the whole picture at once. You mentioned you use this pattern to test regular expressions; say you modify the regexp in question with some new feature requirement, and now the very first of a dozen test inputs fails. You fix it, but then each one of the following keeps failing, and you can only find an elegant solution that works for all of them after seeing all the failures, having ran the test and modified the code a dozen times. Wouldn't it be nicer to see all fails right away and be able to find a solution to all of them, instead of fixing the inputs one-by-one?
- danuker 4y ago
  In my experience, from doing some TDD Katas[0] and timing myself, I found coding slower and more difficult when focusing on multiple examples at once. I usually even comment out all the failing tests but the first one, after translating a bunch of specifications into tests, so I see the "green" when an example starts working. Maybe it would be easier to grok multiple regex examples than algorithmic ones, but at least for myself, I am skeptical, and I prefer taking them one at a time. [0] - https://kata-log.rocks/tdd https://kata-log.rocks/tdd
  - tsimionescu 4y ago
    In my own experience, this has often been a good way of going in circles, where I end up undoing and redoing changes as fixing one thing breaks another, until I take a step back to find the proper algorithm by considering multiple inputs. Of course, ymmv depending on how good your initial intuition is, and how tricky the problem is.
  - johtso 4y ago
    Again pytest makes things so much nicer in this regard. Having to comment things out sucks. With pytest you can use the -x flag to stop after the first test failure. Even better you can use that in combination with -lf to only run the last failed test.
    - masklinn 4y ago
      > With pytest you can use the -x flag to stop after the first test failure. > Even better you can use that in combination with -lf to only run the last failed test. Fwiw `--sw` is much better for that specific use-case. `--lf` is more useful to run the entire test suite, then re-run just the failed tests (of the entire suite). IIRC it can have some odd interactions with `-x` or `--maxfail`, because the strange things happen to the cached "selected set". Though it may also be because I use xdist a fair bit, and the interaction of xdist with early interruptions (x, maxfail, ...) seems less than perfect.
      - johtso 4y ago
        Oh nice, didn't know about that flag! Another option is to use a custom mark on the test you want to run, and then do something like "pytest -v -m onlyrunthis"
  - deleted 4y ago
    [deleted]

olex 4y ago

Hm. I think my main issue there is not the speed, but rather seeing the whole picture at once. You mentioned you use this pattern to test regular expressions; say you modify the regexp in question with some new feature requirement, and now the very first of a dozen test inputs fails. You fix it, but then each one of the following keeps failing, and you can only find an elegant solution that works for all of them after seeing all the failures, having ran the test and modified the code a dozen times. Wouldn't it be nicer to see all fails right away and be able to find a solution to all of them, instead of fixing the inputs one-by-one?

danuker 4y ago
In my experience, from doing some TDD Katas[0] and timing myself, I found coding slower and more difficult when focusing on multiple examples at once. I usually even comment out all the failing tests but the first one, after translating a bunch of specifications into tests, so I see the "green" when an example starts working. Maybe it would be easier to grok multiple regex examples than algorithmic ones, but at least for myself, I am skeptical, and I prefer taking them one at a time. [0] - https://kata-log.rocks/tdd https://kata-log.rocks/tdd
- tsimionescu 4y ago
  In my own experience, this has often been a good way of going in circles, where I end up undoing and redoing changes as fixing one thing breaks another, until I take a step back to find the proper algorithm by considering multiple inputs. Of course, ymmv depending on how good your initial intuition is, and how tricky the problem is.
- johtso 4y ago
  Again pytest makes things so much nicer in this regard. Having to comment things out sucks. With pytest you can use the -x flag to stop after the first test failure. Even better you can use that in combination with -lf to only run the last failed test.
  - masklinn 4y ago
    > With pytest you can use the -x flag to stop after the first test failure. > Even better you can use that in combination with -lf to only run the last failed test. Fwiw `--sw` is much better for that specific use-case. `--lf` is more useful to run the entire test suite, then re-run just the failed tests (of the entire suite). IIRC it can have some odd interactions with `-x` or `--maxfail`, because the strange things happen to the cached "selected set". Though it may also be because I use xdist a fair bit, and the interaction of xdist with early interruptions (x, maxfail, ...) seems less than perfect.
    - johtso 4y ago
      Oh nice, didn't know about that flag! Another option is to use a custom mark on the test you want to run, and then do something like "pytest -v -m onlyrunthis"
- deleted 4y ago
  [deleted]