7 ms·
> Today’s coding benchmarks have established that models can write correct code. I wouldn't say that. > But as AI-generated code becomes the dominant path to
by einpoklum 5mo ago
> Today’s coding benchmarks have established that models can write correct code.
I wouldn't say that.
> But as AI-generated code becomes the dominant path to production
I really hope that's not the case.
- zakisaad 5mo agoHow do you define "correct" code?
- newsicanuse 5mo agoThe code that gets stuff done instead of beating around the bush making unxpected errors
- vanuatu 5mo agoi suspect this is highly dependent on what you're working on from my experience if you give the models a way to self-verify correctness they succeed basically 100% of the time
- maccard 5mo ago> from my experience if you give the models a way to self-verify correctness they succeed basically 100% of the time My experience is that if you can get the model to one shot the task, you'll do fine but if it has to iterate it leaves things worse than before and almost always requires human intervention after burning through an enormous amount of tokens