3 ms·
The positive view: generate scenarios in which the code will fail. The negative view: write plausible-seeming explanations justifying the code as correct.
by mannykannot 3y ago
The positive view: generate scenarios in which the code will fail.
The negative view: write plausible-seeming explanations justifying the code as correct.
- mattsan 3y agoyep this is what I meant. hallucinating, justifying or missing bad stuff. additionally, similar to how large PRs are more likely to just be skimmed and replied with a "LGTM!", an LLM missing some bad stuff but still producing a seemingly thorough review would increase the chance of the bad stuff making its way in. allowing LLMs to write code would be fine if its truly verified by a human, but let another LLM hallucinate and cloud a persons judgement and you've got a problem