4 ms·

Nah I'm about done working on rails codebases with 2+ hour CI cycles because developers insist on hammering a database in unit tests.

by mvdtnz 3y ago

abdusco 3y ago
There's value in having database tests. It lets you assert the logic embedded in SQL in repos, which you cannot test without actually running the query on an actual database.
hardware2win 3y ago
Full green tests when app doesnt even wake up due to db issues sucks too
jasfi 3y ago
I can see a two-tier approach working. First run tests with mocks that runs very quickly, because no DB or API calls are made, they are mocked. Then run a full-on test that includes DB calls, because this will fully test your app better than mocks ever could. This way if a mocked test fails you find out much faster.
- squeaky-clean 3y ago
  This is my take. I have one test suite that should be able to run upon file-save and be completed in about 5 seconds. Then another suite that takes about 20 minutes that I can run in CI or locally. I suppose unit vs integration, but I don't worry about conforming to those definitions too strongly. Basically tests in the first suite should be fast and able to run in parallel. If they touch the file system or talk to a local db that's fine as long as it's fast and safe in parallel. The second suite can do whatever.
joeblubaugh 3y ago
If you can use sqlite in tests it’s often a good middle ground
dan_mctree 3y ago
And I'm done with developers who proudly boast of 95% test coverage using mocked tests because "we don't want to test the database" and then have database changes cause production crashes. "But my tests were all green!" Databases, external APIs, dependency version updates, etc are among the most likely places where unintended changes get introduced. Why would you not test that when catching unintended breaking changes is one of the primary purposes of tests? Your simple business logic layer function converting one object into another is not going to be the problem, the database is
- StackOverlord 3y ago
  Get the best of both world by mocking the database API itself. Use the mocks to memoize the db output. Disable the mocks if you need long thorough tests. Determine if the mocks need to be disabled by comparing the date of the last thorough check with the date of the last db schema change. Instead of something global like this, one could come up with something local to the test file, using a system of cassettes. Doing something like this would require to make the database a value and memoize according to its current state, but I think it's doable.
ecshafer 3y ago
2+ hour CI means something else is going wrong. at shopify, our CI is like ~10 minutes and we typically avoid mocks.
- mvdtnz 3y ago
  That may be the wall clock time but how parallelized is that?

Nah I'm about done working on rails codebases with 2+ hour CI cycles because developers insist on hammering a database in unit tests.

by mvdtnz 3y ago

Nah I'm about done working on rails codebases with 2+ hour CI cycles because developers insist on hammering a database in unit tests.

abdusco 3y ago

There's value in having database tests. It lets you assert the logic embedded in SQL in repos, which you cannot test without actually running the query on an actual database.

hardware2win 3y ago

Full green tests when app doesnt even wake up due to db issues sucks too

jasfi 3y ago

I can see a two-tier approach working. First run tests with mocks that runs very quickly, because no DB or API calls are made, they are mocked. Then run a full-on test that includes DB calls, because this will fully test your app better than mocks ever could. This way if a mocked test fails you find out much faster.

squeaky-clean 3y ago
This is my take. I have one test suite that should be able to run upon file-save and be completed in about 5 seconds. Then another suite that takes about 20 minutes that I can run in CI or locally. I suppose unit vs integration, but I don't worry about conforming to those definitions too strongly. Basically tests in the first suite should be fast and able to run in parallel. If they touch the file system or talk to a local db that's fine as long as it's fast and safe in parallel. The second suite can do whatever.

joeblubaugh 3y ago

If you can use sqlite in tests it’s often a good middle ground

dan_mctree 3y ago

And I'm done with developers who proudly boast of 95% test coverage using mocked tests because "we don't want to test the database" and then have database changes cause production crashes. "But my tests were all green!" Databases, external APIs, dependency version updates, etc are among the most likely places where unintended changes get introduced. Why would you not test that when catching unintended breaking changes is one of the primary purposes of tests? Your simple business logic layer function converting one object into another is not going to be the problem, the database is

StackOverlord 3y ago
Get the best of both world by mocking the database API itself. Use the mocks to memoize the db output. Disable the mocks if you need long thorough tests. Determine if the mocks need to be disabled by comparing the date of the last thorough check with the date of the last db schema change. Instead of something global like this, one could come up with something local to the test file, using a system of cassettes. Doing something like this would require to make the database a value and memoize according to its current state, but I think it's doable.

ecshafer 3y ago

2+ hour CI means something else is going wrong. at shopify, our CI is like ~10 minutes and we typically avoid mocks.

mvdtnz 3y ago
That may be the wall clock time but how parallelized is that?