4 ms·
The fact that tests are passing is not a useful metric to me - it’s easy to write low quality passing tests. I may be biased, I am working with a codebase writ
by thih9 2y ago
The fact that tests are passing is not a useful metric to me - it’s easy to write low quality passing tests.
I may be biased, I am working with a codebase written in copilot and I have seen tests that check if dictionary literals have the value that was entered in them, or that the functions with a certain type indeed return objects of that type.
- teo_zero 2y agoWe should have two distinct, independent LLMs: one generates the code, the other one the tests.
- eMPee584 2y agoperfect use case for GANs (generative adversarial network, consisting of (at least) a generator and a discriminator / judge) isn't it? (iiuc)
- TeMPOraL 2y agoDo you also hire two different kinds of programmers - one that never wrote a test in their life, and is not allowed to write anything other than production code, and second that never ever wrote anything other than tests, and is only ever allowed to write tests? It makes no sense to have "two distinct, independent LLMs" - two general-purpose tools - to do what is the same task. It might make sense to have two separate prompts.