5 ms·

I don’t think that’s right? The isolation comes from the test implementation not the framework. There isn’t any framework out there that can guarantee/give you

I don’t think that’s right? The isolation comes from the test implementation not the framework. There isn’t any framework out there that can guarantee/give you isolation. If I create a new in mem dB in the test there’s nothing stopping me from running it in parallel? Nothing about that “requires” isolation. It is isolation.

mrkeen 2mo ago
Maybe I misread. When I see a setUp() in a test suite, 9 times out of 10 it's to handle something shared and bulky, like a database. Otherwise you'd just do the thing - assert(expected, myService.run(input)); If I saw an extra myService.reset() or myService.setUp() I would suspect it's either in anticipation of a bad state (from a previous test) or to be a good neighbour for the next test which will run. In the cases where each unit test does have it's own individual Postgres or whatever, sure.
- aleksiy123 2mo ago
  I guess it all depends, isolation isn’t binary. you can share some things and be isolated across other dimensions. You could share a Postgres dB connection but just isolate the data logically. You can isolate or share across individual tests, across suites, across envs, across runs, across time. More isolation is generally better unless the complexity or cost is too high. But I guess I was just confused as to how a test requires isolation rather than have it.
locknitpicker 2mo ago
> The isolation comes from the test implementation not the framework. There isn’t any framework out there that can guarantee/give you isolation. I'm not sure what you mean by that. All the frameworks I ever used were designed with test isolation as the primary design goal. Even when you set shared test fixtures and setup/teardown code, all they provide is a way to share code across tests, which are by themselves independent and isolated. BDD-style frameworks are the notable exception in breaking away from the pattern of having isolation as a fundamental design trait. In fact, the whole reason why BDD-style tools require special support from testing frameworks is that they need to work around test isolation in order to share context across steps.
- solarengineer 2mo ago
  In JUnit, tests run sequentially in a single thread by default [1]. Parallel execution must be explicitly enabled. When enabled, JUnit uses a fork-join thread pool, so tests may run concurrently on different worker threads. Because these threads are reused, a ThreadLocal value left behind by one test could be visible to a later test that happens to run on the same thread. Setup and teardown methods can be used to create and clean up test-specific state. However, developers must still ensure that test data is unique to the test so that concurrently running tests do not interfere with each other. This uniqueness is unfortunately called "isolated", and has led to much confusion like in this thread. Certainly, the Test Execution Framework cannot guarantee data-isolation. Parallel and randomized test execution can also help expose application-side problems involving shared or order-dependent state. Such failures may only appear when tests happen to exercise the application under the relevant ordering or concurrency conditions. When I was a junior developer teaching myself Java Servlets in 2000, I had to learn this lesson the hard way. A Test Execution Framework would not be able to guarantee any "isolation" of test data and of workflows if the server-side state is mis-managed by the tech stack and/or by the developer. [1] https://docs.junit.org/6.1.3/writing-tests/parallel-execution.html https://docs.junit.org/6.1.3/writing-tests/parallel-executio...
  - aleksiy123 2mo ago
    This
  - locknitpicker 2mo ago
    > Parallel execution must be explicitly enabled. When enabled, JUnit uses a fork-join thread pool, so tests may run concurrently on different worker threads. Because these threads are reused, a ThreadLocal value left behind by one test could be visible to a later test that happens to run on the same thread. I don't understand your comment. JUnit docs describe parallel execution as an experimental feature that still has gotchas that you need to be mindful to avoid. So you have a test framework designed with isolation in mind for a specific execution mode, but when you go out of your way to try another execution mode that is described as experimental then you can stumble upon very corner cases where isolation is not ensured. What point did you wanted to convey?

mrkeen 2mo ago

Maybe I misread. When I see a setUp() in a test suite, 9 times out of 10 it's to handle something shared and bulky, like a database. Otherwise you'd just do the thing - assert(expected, myService.run(input)); If I saw an extra myService.reset() or myService.setUp() I would suspect it's either in anticipation of a bad state (from a previous test) or to be a good neighbour for the next test which will run. In the cases where each unit test does have it's own individual Postgres or whatever, sure.

aleksiy123 2mo ago
I guess it all depends, isolation isn’t binary. you can share some things and be isolated across other dimensions. You could share a Postgres dB connection but just isolate the data logically. You can isolate or share across individual tests, across suites, across envs, across runs, across time. More isolation is generally better unless the complexity or cost is too high. But I guess I was just confused as to how a test requires isolation rather than have it.

locknitpicker 2mo ago

> The isolation comes from the test implementation not the framework. There isn’t any framework out there that can guarantee/give you isolation. I'm not sure what you mean by that. All the frameworks I ever used were designed with test isolation as the primary design goal. Even when you set shared test fixtures and setup/teardown code, all they provide is a way to share code across tests, which are by themselves independent and isolated. BDD-style frameworks are the notable exception in breaking away from the pattern of having isolation as a fundamental design trait. In fact, the whole reason why BDD-style tools require special support from testing frameworks is that they need to work around test isolation in order to share context across steps.

solarengineer 2mo ago
In JUnit, tests run sequentially in a single thread by default [1]. Parallel execution must be explicitly enabled. When enabled, JUnit uses a fork-join thread pool, so tests may run concurrently on different worker threads. Because these threads are reused, a ThreadLocal value left behind by one test could be visible to a later test that happens to run on the same thread. Setup and teardown methods can be used to create and clean up test-specific state. However, developers must still ensure that test data is unique to the test so that concurrently running tests do not interfere with each other. This uniqueness is unfortunately called "isolated", and has led to much confusion like in this thread. Certainly, the Test Execution Framework cannot guarantee data-isolation. Parallel and randomized test execution can also help expose application-side problems involving shared or order-dependent state. Such failures may only appear when tests happen to exercise the application under the relevant ordering or concurrency conditions. When I was a junior developer teaching myself Java Servlets in 2000, I had to learn this lesson the hard way. A Test Execution Framework would not be able to guarantee any "isolation" of test data and of workflows if the server-side state is mis-managed by the tech stack and/or by the developer. [1] https://docs.junit.org/6.1.3/writing-tests/parallel-execution.html https://docs.junit.org/6.1.3/writing-tests/parallel-executio...
- aleksiy123 2mo ago
  This
- locknitpicker 2mo ago
  > Parallel execution must be explicitly enabled. When enabled, JUnit uses a fork-join thread pool, so tests may run concurrently on different worker threads. Because these threads are reused, a ThreadLocal value left behind by one test could be visible to a later test that happens to run on the same thread. I don't understand your comment. JUnit docs describe parallel execution as an experimental feature that still has gotchas that you need to be mindful to avoid. So you have a test framework designed with isolation in mind for a specific execution mode, but when you go out of your way to try another execution mode that is described as experimental then you can stumble upon very corner cases where isolation is not ensured. What point did you wanted to convey?