3 ms·
Yes, and for some cases no. The models are gotten very good, but I rather have an obviously broken pile of crap that I can spot immediately, than something tha
by Snuggly73 9mo ago
Yes, and for some cases no.
The models are gotten very good, but I rather have an obviously broken pile of crap that I can spot immediately, than something that is deep fried with RL to always succeed, but has subtle problems that someone will lgtm :( I guess its not much different with human written code, but the models seem to have weirdly inhuman failures - like, you would just skim some code, cause you just cant believe that anyone can do it wrong, and it turns out to be.
- minimaxir 9mo agoThat's what test cases are for, which is good for both humans and nonhumans.
- Snuggly73 9mo agoTest cases are great, but not a total solution. Can you write a test case for the add_numbers(a, b) function?
- minimaxir 9mo agoOf course you can. You can write test cases for anything. Even an add_numbers function can have bugs, e.g. you have to ensure the inputs are numbers. Most coding agents would catch this in loosely-typed languages.
- Snuggly73 9mo agoWell, for some reason it doesnt let me respond to the child comments :( The problem (which should be obvious) is that with a/b real you cant construct an exhaustive input/output set. The test case can just prove the presence of a bug, but not its absence. Another category of problems that you cant just test and have to prove is concurrency problems. And so forth and so on.