Basically yes, once the "problem" gets too big, the LLM stops being useful. As you say, it's great for automating away boring things; as a more complicated sea

Basically yes, once the "problem" gets too big, the LLM stops being useful. As you say, it's great for automating away boring things; as a more complicated search & replace, for instance. Or, "Implement methods so that it satisfies this interface", where the methods are pretty obvious. Or even "Fill out stub CRUD operations for this set of resources in the API". I've recently started asking Claude Opus 4 to review my patches when I'm done, and it's occasionally caught errors, and sometimes has been good at prompting me to do something I know I really should be doing. But once you get past a certain complexity level -- which isn't really that far - it just stops being useful. For one thing, the changes which need to be made often span multiple files, each of which is fairly large; so I try to think carefully about which files would need to be touched to make a change; after which point I find I have an idea what needs to be changed anyway. That said, using the AI like a "rubber duck" programmer isn't necessarily bad. Basically, I ask it to make a change; if it makes it and it's good, great! If it's a bit random, I just take over and do it myself. I've only wasted the time of reviewing the LLM's very first change, as nearly everything else I'd've had to do if I wrote the patch myself from scratch anyway. Furthermore, I often find it much easier to take a framework that's mostly in the right direction and modify it the way that I want, than to code up everything from scratch. So if I say, "Implement this", and then end up modifying nearly everything, it still seems like less effort than starting from scratch myself. The key thing is that I don't work hard at trying to make the LLM do something it's clearly having trouble with. Sometimes the specification was unclear and it made a reasonable assumption; but if I tell it to do something and it's still having trouble, I just finish the task myself.