45 ms·
My experience with smaller models, in this case specifically GPT 5.4 Mini, is that they cannot two-shot moving a 10-20 line code change to another file without
by user43928 4mo ago
My experience with smaller models, in this case specifically GPT 5.4 Mini, is that they cannot two-shot moving a 10-20 line code change to another file without modifying it and introducing bugs.
I did not expect perfect reliability, but I thought they could at least get it right on the second attempt once you point out the difference. No such luck, it confidently tells you that now the code is the same, with yet another subtle bug added in the difference.
I don't know what work one would need to do where these garbage-class models would be adequate. Maybe they can masquerade as competent for a few minutes, but in the end the results simply are not right. At best they are suitable for a smarter search or autocomplete, in my opinion.
- what 4mo agoIs it not faster to just do that move yourself instead of asking the clanker to do it?
- user43928 4mo agoTyping "Create a branch for X and open a separate MR" is faster than me manually creating a branch, selecting and copying changes there, and then opening a MR.
- Applejinx 4mo agoRather than 'smarter search or autocomplete', maybe the better analogy is 'flexible information lookup about coding that's more responsive to search terms'? I see it not as an intelligence but as a wildly, spectacularly compressed knowledge base. You're trying to get search results that encompass almost any possible thing you could ask for, but rather than drawing from some textbook you're drawing from a distilled combination of ALL textbooks and everything web-scrapable since before the dotcom days. Of course this doesn't produce a useful person who always makes right choices, but isn't it interesting that you can compress that heavily and draw results out in such a casual way? Seems this remains relevant.