5 ms·

It seems like you need to call this function :) and not surprisingly, your comment is totally incorrect, tests can be implemented independent of each other rega

It seems like you need to call this function :) and not surprisingly, your comment is totally incorrect, tests can be implemented independent of each other regardless of the quality of the code they are testing. This doesn't mean they can run in parallel, but every test should be running independently from each other no matter what.

sublinear 4y ago
Oh boy. The fact that tests test arbitrary things should be enough for you to reconsider what you're saying. You can't possibly know if I'm "totally incorrect". I'm not talking about sharing state between tests. I'm talking about the code being tested having global state that is deeply entrenched in how it works, it's unreasonable to refactor, yet a test suite must be written. What truly sucks is someone not even writing tests at all for the tangled parts which is extremely common to find because it breaks someone's precious mental model and they don't want to put their name on it.
- imran-iq 4y ago
  > I'm talking about the code being tested having global state that is deeply entrenched in how it works Could you give a specific/actual example of this? Technically a database is global state that is deeply entrenched in how an app works, but its trivial to make order independent tests that require certain records to exist.
- toolz 4y ago
  The way I see it, if your tests are dependent on mutable global state, they are bad. If they are not dependent on mutable global state, then other tests can't break your tests. I'm not confident enough to say there aren't exceptions here though.
  - andreareina 4y ago
    With mutable global state I wouldn't be too sure it's the tests that are the problem.
    - djur 4y ago
      If your tests aren't accounting for the effects of mutable global state on the code under test they are bad tests. Randomization helps catch bad tests.
- rgoulter 4y ago
  > The fact that tests test arbitrary things should be enough for you to reconsider what you're saying. You can't possibly know if I'm "totally incorrect". You'll have to explain, then, the significant difference between: test_one: lots of setup code for scenario one assertions test_two: lots of setup code for scenario two assertions where test_two only runs after test_one; and: test_one: lots of setup code for scenario one assertions test_two: lots of setup code for scenario one lots of setup code for scenario two assertions Of course, if the same "setup code for scenario one" running twice breaks things, then so would "running the suite twice", sure. > I'm talking about the code being tested having global state that is deeply entrenched in how it works "code being tested having global state that is deeply entrenched in how it works" does fall under "sharing state between tests".
- vore 4y ago
  I don't understand how this could be the case? If you need global state that carries over from another test and enforce it by order dependency, then the state is being shared. Otherwise, you can run the tests in any order.
- kissgyorgy 4y ago
  Having code which relies on global state is probably the worst possible case, that's why I mentioned you can STILL write independent tests from each other. You could even run them in parallel, but you would have to use locks, but just by running them serially, you don't even need to do that. You just have to make sure that the global state has been reset after each test case properly. If your project language is Python, pytest is providing all kinds of tools to setup and reset all kinds of global state (see the monkeypatch fixture). In other languages this can be solvable obviously, you might just need to write more helpers and might be more careful. If you don't write your tests independent from each other, you will have an awful time debugging any failure and will waste insane working hours which could be prevented by a little bit of discipline.
michaelt 4y ago
> tests can be implemented independent of each other regardless of the quality of the code they are testing. For example, if you have ten tests that must be run in a particular order, you can simply turn them into a single gigantic test.
- sublinear 4y ago
  This gave me a good chuckle. I imagine the test would then be named something vague and ultimately useless. Damn near every change breaks this test and I don't know what now! As long as it's all green checkmarks and I don't need to run all the tests every time then it must be ok right?
  - djur 4y ago
    If the tests are really interdependent to the degree that they cannot be reordered, they must be testing some kind of multi-step process. So the test should be named after that process.

sublinear 4y ago

Oh boy. The fact that tests test arbitrary things should be enough for you to reconsider what you're saying. You can't possibly know if I'm "totally incorrect". I'm not talking about sharing state between tests. I'm talking about the code being tested having global state that is deeply entrenched in how it works, it's unreasonable to refactor, yet a test suite must be written. What truly sucks is someone not even writing tests at all for the tangled parts which is extremely common to find because it breaks someone's precious mental model and they don't want to put their name on it.

imran-iq 4y ago
> I'm talking about the code being tested having global state that is deeply entrenched in how it works Could you give a specific/actual example of this? Technically a database is global state that is deeply entrenched in how an app works, but its trivial to make order independent tests that require certain records to exist.
toolz 4y ago
The way I see it, if your tests are dependent on mutable global state, they are bad. If they are not dependent on mutable global state, then other tests can't break your tests. I'm not confident enough to say there aren't exceptions here though.
- andreareina 4y ago
  With mutable global state I wouldn't be too sure it's the tests that are the problem.
  - djur 4y ago
    If your tests aren't accounting for the effects of mutable global state on the code under test they are bad tests. Randomization helps catch bad tests.
rgoulter 4y ago
> The fact that tests test arbitrary things should be enough for you to reconsider what you're saying. You can't possibly know if I'm "totally incorrect". You'll have to explain, then, the significant difference between: test_one: lots of setup code for scenario one assertions test_two: lots of setup code for scenario two assertions where test_two only runs after test_one; and: test_one: lots of setup code for scenario one assertions test_two: lots of setup code for scenario one lots of setup code for scenario two assertions Of course, if the same "setup code for scenario one" running twice breaks things, then so would "running the suite twice", sure. > I'm talking about the code being tested having global state that is deeply entrenched in how it works "code being tested having global state that is deeply entrenched in how it works" does fall under "sharing state between tests".
vore 4y ago
I don't understand how this could be the case? If you need global state that carries over from another test and enforce it by order dependency, then the state is being shared. Otherwise, you can run the tests in any order.
kissgyorgy 4y ago
Having code which relies on global state is probably the worst possible case, that's why I mentioned you can STILL write independent tests from each other. You could even run them in parallel, but you would have to use locks, but just by running them serially, you don't even need to do that. You just have to make sure that the global state has been reset after each test case properly. If your project language is Python, pytest is providing all kinds of tools to setup and reset all kinds of global state (see the monkeypatch fixture). In other languages this can be solvable obviously, you might just need to write more helpers and might be more careful. If you don't write your tests independent from each other, you will have an awful time debugging any failure and will waste insane working hours which could be prevented by a little bit of discipline.

michaelt 4y ago

> tests can be implemented independent of each other regardless of the quality of the code they are testing. For example, if you have ten tests that must be run in a particular order, you can simply turn them into a single gigantic test.

sublinear 4y ago
This gave me a good chuckle. I imagine the test would then be named something vague and ultimately useless. Damn near every change breaks this test and I don't know what now! As long as it's all green checkmarks and I don't need to run all the tests every time then it must be ok right?
- djur 4y ago
  If the tests are really interdependent to the degree that they cannot be reordered, they must be testing some kind of multi-step process. So the test should be named after that process.