3 ms·
The author begins by giving a number of test cases, like f(35)==6, for which the AI provides a candidate f, but the author doesn't actually implement the test c
by pphysch 4y ago
The author begins by giving a number of test cases, like f(35)==6, for which the AI provides a candidate f, but the author doesn't actually implement the test cases, is surprised when the AI is wrong, and laments the resulting burden of proofreading.
Just implement those tests, for heaven's sake!
- majormajor 4y agoImplementing a correct and comprehensive set of test cases for code is more time consuming and often more difficult than writing the code itself. (The comprehensive part especially - how do you prove to yourself you've covered all the necessary edge cases?) "Reading code is harder than writing it" "Debugging code is harder than writing it" These are pretty common statements, going back years. They should make the lack of a silver bullet here very obvious. (Yes, most software is already broken in various ways - my worry is that this will just make it more broken, e.g. non-technical management saying "why do we need to do as much QA, we have AI writing the code now!?")
- pphysch 4y agoYes, writing good tests is hard. My point is that he already has the relevant test cases. The hard work is done.
- fragmede 4y agoeven easier: ask the LLM for those test cases!
- hwayne 4y agoTo clarify, Bertrand Meyer is the one who gave the test cases and accepted the wrong answer without verifying them. I'm the one who lamented the resulting burden of proofreading.