In my experience, if I use github copilot chat as a natural language interface to programming it's basically flawless. That is, I tell it exactly what I want th

In my experience, if I use github copilot chat as a natural language interface to programming it's basically flawless. That is, I tell it exactly what I want the code to he like but in english, and the llm just translates that into code for me. When using the inline github copilot, I find that whenever I am making changes that are obvious, it gets things 100% right. If there are no obvious mistakes or additions to my code, it makes up completely useless noise. I just ignore that. I have had some experiences where I did not know the language or library that I wanted to work with, and the results were disastrous. It hallucinates exactly the api that I want, but unfortunately it doesn't exist. As an aside -- that api SHOULD exist. The api design is very obviously missing this extremely important feature. I think llms may help people to figure out what kind of apis and features should be intuitively expected and increase software quality. But it still wasted a ton of my time. My conclusion is actually that llm copilots will be bad for beginners because they don't know enough to specify the request in enough detail, and they don't know enough to see any errors and be able to understand why they were generated.