3 ms·
I once worked in a company where there was an 80% code coverage requirement. Some enterprising contractor had a script that generated a single file with its own
by satnhak 4mo ago
I once worked in a company where there was an 80% code coverage requirement. Some enterprising contractor had a script that generated a single file with its own covering test suite the size of which could be tuned to achieve 80% over the whole codebase. Mostly the code was untested.
- atomicnumber3 4mo agoAnd thanks to AI, we could generate extremely convincing reams of code whose only purpose is to be fake unit tested. Amazing. I sincerely hope I never need to use this nuclear weapon.
- LPisGood 4mo agoOr better yet: effectively fake unit tests. It is almost never the case that tests written by AI detect actual issues. At most they detect that has changed.
- chamomeal 4mo agoYeah I’ve been thinking that LLM unit tests are basically snapshot tests. Just sorta ossify things in place. If they break, you just ¯\_(ツ)_/¯ and have the LLM fix them. It’s like they were never there! So it’s just like the olden days of everybody ignoring tests, but we give anthropic a ton of cash