I was asked to help evaluate a couple code-generating LLMs at work, so I set out to build a couple toy apps in languages and frameworks I wasn't very familiar w

I was asked to help evaluate a couple code-generating LLMs at work, so I set out to build a couple toy apps in languages and frameworks I wasn't very familiar with. My experience was that it was about as useful as searching Google and Stack Overflow without knowing what a good answer looks like. You copy and paste a lot of code, a lot of it works, but inevitably, something breaks and you have no idea why. So you still have to go back to "the hard way" of reading the documentation and building an understanding. I'm sure things will improve over time, especially for products built specifically for code generation, but my first and second impression is that so far, LLMs don't actually lower the bar for junior programmers.