To me, articles like this are not so interesting for the results. I'm not reading them to find out just exactly what the performance of AIs is, exactly. Obvious

To me, articles like this are not so interesting for the results. I'm not reading them to find out just exactly what the performance of AIs is, exactly. Obviously it's not useful for that, it's not systematic, anecdotal, unscientific... I think LLMs today, for all their goods and bads, can do some useful work. The problem is that there is still mystery on how to use them effectively. I'm not talking about some pie in the sky singularity stuff, but just coming up with prompts to do basic, simple tasks effectively. Articles like that are great for learning new prompting tricks and I'm glad the authors are choosing to share their knowledge. Yes, OP isn't saying the last word on prompting, and there's a million ways it could be better. But the article is still useful to an average person trying to learn how to use LLMs more productively. >the "Senior vs Junior Developer" narrative It sounds to me like just another case of "telling the AI to explicitly reason through its answer improves the quality of results". The "senior developer" here is better able to triage aspects of the codebase to identify the important ones (and to the "junior" everything seems equally important) and I would say has better reasoning ability. Maybe it works because when you ask the LLM to code something, it's not really trying to "do a good job", besides whatever nebulous bias is instilled from alignment. It's just trying to act the part of a human who is solving the problem. If you tell it to act a more competent part, it does better - but it has to have some knowledge (aka training data) of what the more competent part looks like.