3 ms·
> An LLM, specifically trained for test generation, consumes this specification. Its objective is to generate a diverse and comprehensive test suite that probes
by Jensson 2y ago
> An LLM, specifically trained for test generation, consumes this specification. Its objective is to generate a diverse and comprehensive test suite that probes the specified behavior from an external perspective.
If one of these tests are wrong though it will ruin the whole thing. And LLM are much more likely to make a math error (which would result in a faulty test) than to implement a math function the wrong way, so this probably wont make it better at generating code.
- MarcoDewey 2y agoI think this is a seriously excellent point. The bet that I am making is that the system reduces its error rate by splitting a broad task into two more focused tasks. However, it is possible that generating meaningful test cases is a harder problem (with a higher error rate) than producing code. If this is the case, then this idea I am presenting would compound the error rate.