7 ms·

Your test fixtures are introducing tighter coupling between the tests, than the code they are testing! In this scenario (a mock DB with data that a test relies

by immibis 11mo ago

Your test fixtures are introducing tighter coupling between the tests, than the code they are testing! In this scenario (a mock DB with data that a test relies on, which is incompatible with a new test you want to add), duplicating the fixture is correct. Different tests with incompatible requirements on state should use different mock data. In this specific case, however, it's also true that one of them can be modified to make the tests compatible. If you do red-green-refactor then you can duplicate first, and then coalesce by changing the first test. "assert_equal names, names.sort" is a wrong answer. It would accept an empty collection.

bluGill 11mo ago
That depends on what state the fixture sets up and when you run them. That state becomes something you expect all tests to handle, if you change the fixture state and test breaks you should be fixing the production code - not the test code - to handle that change. Of course in reality I can well believe it is a tests data conflict in most cases (someone with the name "test user one" already is in the database at a different address..) and this is something you need to ensure doesn't happen. I have a fixture that sets our database to the initial install state. This works for me because we are an embedded system where every month we ship a bunch more new systems and so code needs to see that initial install state, if we change the initial state (which we do all the time) and a test breaks we want to know and fix that since customers will see that situation. However if you run on a server in a data center I could well believe you will never again see any specific state and so a fixture probably isn't right. Maybe ideally every test would take a snapshot of your current production database and test against that (with whatever additional data you add for the test) - if a customer enters data that breaks a test that is a "all hands on deck" to fix the code before customers hit that code path. Maybe - I don't work in this space and so I'm just speculating what you need.
- immibis 11mo ago
  Different unit tests work on different state. One unit test says that if I start from a blank database, add two users, now there are two users. A different unit test says that if I start from a database with two users, and list the users, I get them. If you rely on the add-user test running before the list-users test, you introduce bad coupling between tests. However, you could run them all in order in one test - it would be a more complex test. You could then run that on a real database and call it an integration test. Tests at all levels of complexity are useful. You could run end-to-end tests on a production e-commerce system, from signup to payment with a real credit card and delivery of a real physical product, if you wanted. Backup power systems are tested by shutting off the power a few times a year. But this article is clearly about unit tests. You shouldn't run unit tests against the production database and you should aim to minimize their dependencies.

by immibis 11mo ago

bluGill 11mo ago

That depends on what state the fixture sets up and when you run them. That state becomes something you expect all tests to handle, if you change the fixture state and test breaks you should be fixing the production code - not the test code - to handle that change. Of course in reality I can well believe it is a tests data conflict in most cases (someone with the name "test user one" already is in the database at a different address..) and this is something you need to ensure doesn't happen. I have a fixture that sets our database to the initial install state. This works for me because we are an embedded system where every month we ship a bunch more new systems and so code needs to see that initial install state, if we change the initial state (which we do all the time) and a test breaks we want to know and fix that since customers will see that situation. However if you run on a server in a data center I could well believe you will never again see any specific state and so a fixture probably isn't right. Maybe ideally every test would take a snapshot of your current production database and test against that (with whatever additional data you add for the test) - if a customer enters data that breaks a test that is a "all hands on deck" to fix the code before customers hit that code path. Maybe - I don't work in this space and so I'm just speculating what you need.

immibis 11mo ago
Different unit tests work on different state. One unit test says that if I start from a blank database, add two users, now there are two users. A different unit test says that if I start from a database with two users, and list the users, I get them. If you rely on the add-user test running before the list-users test, you introduce bad coupling between tests. However, you could run them all in order in one test - it would be a more complex test. You could then run that on a real database and call it an integration test. Tests at all levels of complexity are useful. You could run end-to-end tests on a production e-commerce system, from signup to payment with a real credit card and delivery of a real physical product, if you wanted. Backup power systems are tested by shutting off the power a few times a year. But this article is clearly about unit tests. You shouldn't run unit tests against the production database and you should aim to minimize their dependencies.