4 ms·

> Maybe I've just got unlucky in the past, but in most projects I worked on a lot of developer time was wasted on waiting for PRs to go green. Many runs end up

by mrkeen 1y ago

> Maybe I've just got unlucky in the past, but in most projects I worked on a lot of developer time was wasted on waiting for PRs to go green. Many runs end up bottlenecked on I/O or availability of workers No, this is common. The devs just haven't grokked dependency inversion. And I think the rate of new devs entering the workforce will keep it that way forever. Here's how to make it slow: * Always refer to "the database". You're not just storing and retrieving objects from anywhere - you're always using the database. * Work with statements, not expressions. Instead of "the balance is the sum of the transactions", execute several transaction writes (to the database) and read back the resulting balance. This will force you to sequentialise the tests (simultaneous tests would otherwise race and cause flakiness) plus you get to write a bunch of setup and teardown and wipe state between tests. * If you've done the above, you'll probably need to wait for state changes before running an assertion. Use a thread sleep, and if the test is ever flaky, bump up the sleep time and commit it if the test goes green again.

zbentley 1y ago
> Instead of "the balance is the sum of the transactions", execute several transaction writes (to the database) and read back the resulting balance Er, doesn’t this boil down to saying “not testing database end state (trusting in transactionality) is faster than testing it”? I mean sure, trivially true, but not a good idea. I’ve seen lots of bugs caused by code that unexpectedly forced a commit, or even opened/used/committed a whole new DB connection, somewhere buried down inside a theoretically externally-transactional request handler. Bad code, to be sure, but common in many contexts in my experience.
- mrkeen 1y ago
  > I’ve seen lots of bugs caused by code that unexpectedly forced a commit, or even opened/used/committed a whole new DB connection, somewhere buried down inside a theoretically externally-transactional request handler. Yes! That's my current codebase you're describing! If you interweave the database all throughout your accounting logic, you absolutely can bury those kinds of problems for people to find later. But remember, one test at a time so that you don't accidentally discover that your the database transactions aren't protecting you nearly as well as you thought. In fact, screw database transactions. Pay the cost of object-relation impedance mismatch and unscalable joins, but make sure you avoid the benefits, by turning off ACID for performance reasons (probably done for you already) and make heavy use of LINQ so that values are loaded in and out of RAM willy-nilly and thereby escape their transaction scopes. The C# designers really leaned into the 'statements' not 'expression' idea! There's no transaction context object returned from beginTrans which could be passed into subsequent operations (forming a nice expression) and thereby clear up any "am I in a transaction?" questions. But yeah, right now it's socially acceptable to plumb the database crap right through the business logic. If we could somehow put CSS or i18n in the business logic, we'd need to put a browser into our test suite too!
bvrmn 1y ago
Nah. Tests could be run in N processes each with own database configured to skip full fsync. It resolves most of the issues and makes testing much much simpler.

by mrkeen 1y ago

zbentley 1y ago

> Instead of "the balance is the sum of the transactions", execute several transaction writes (to the database) and read back the resulting balance Er, doesn’t this boil down to saying “not testing database end state (trusting in transactionality) is faster than testing it”? I mean sure, trivially true, but not a good idea. I’ve seen lots of bugs caused by code that unexpectedly forced a commit, or even opened/used/committed a whole new DB connection, somewhere buried down inside a theoretically externally-transactional request handler. Bad code, to be sure, but common in many contexts in my experience.

mrkeen 1y ago
> I’ve seen lots of bugs caused by code that unexpectedly forced a commit, or even opened/used/committed a whole new DB connection, somewhere buried down inside a theoretically externally-transactional request handler. Yes! That's my current codebase you're describing! If you interweave the database all throughout your accounting logic, you absolutely can bury those kinds of problems for people to find later. But remember, one test at a time so that you don't accidentally discover that your the database transactions aren't protecting you nearly as well as you thought. In fact, screw database transactions. Pay the cost of object-relation impedance mismatch and unscalable joins, but make sure you avoid the benefits, by turning off ACID for performance reasons (probably done for you already) and make heavy use of LINQ so that values are loaded in and out of RAM willy-nilly and thereby escape their transaction scopes. The C# designers really leaned into the 'statements' not 'expression' idea! There's no transaction context object returned from beginTrans which could be passed into subsequent operations (forming a nice expression) and thereby clear up any "am I in a transaction?" questions. But yeah, right now it's socially acceptable to plumb the database crap right through the business logic. If we could somehow put CSS or i18n in the business logic, we'd need to put a browser into our test suite too!

bvrmn 1y ago

Nah. Tests could be run in N processes each with own database configured to skip full fsync. It resolves most of the issues and makes testing much much simpler.