When saying LLMs are "good at writing code" there's a distinction between "good at taking high-level English-language solutions to programming problems and tra

When saying LLMs are "good at writing code" there's a distinction between "good at taking high-level English-language solutions to programming problems and translating them into low-level implementations via a specific language / design patterns / etc" versus "good at finding solutions to programming problems." GPT-4 is indeed quite good at the former - and the former is what most enterprise programming work actually is. A lot of the hard part of professional software development is understanding the problem well enough to describe a solution in English: once you do that, writing the C# or Java is typically somewhat rote. Likewise LLMs are genuinely useful when you know exactly what you want to do with a 3rd-party library, but have to trawl through a bunch of API documentation to figure out the magic words. All that said, LLMs still really suck at the latter problem: https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmarks https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmar... OpenAI's benchmarks fall off quite badly with actual programming benchmarks, compared to simpler tests of code generation. If it involves managing state, creating novel data structures, counting to numbers higher than 3, etc, LLMs just aren't smart enough.