3 ms·
"A tool is only useful if I can use it in every situation". LLM's don't need to find every bug in your code - even if they found an additional 10% of genuine b
by Closi 3y ago
"A tool is only useful if I can use it in every situation".
LLM's don't need to find every bug in your code - even if they found an additional 10% of genuine bugs compared to existing tools, it's still a pretty big improvement to code analysis.
In reality, I suspect the scope is much higher than 10%.
- chrisco255 3y agoIf it takes you longer to vet hallucinations than to just test your code better, is it an improvement? If you accept a bug fix for a hallucination that you got too lazy to check because you grew dependent on AI to do the analysis for you, and the bug "fix" itself causes other unforeseen issues or fails to recognize why an exception in this case might be worth preserving, is it really an improvement?
- Closi 3y agoWhat if it takes you longer to vet false positives from a static analysis tool rather than just testing your code better?
- Cthulhu_ 3y agoWhat if indeed. Most static analyses tools (disclaimer: anecdotal) have very little false positives these days. This may be much worse in C/C++ land though, I don't know.