> Learning to say what you want is a skill True, but “learning to figure out what someone wanted to say but wasn’t able to express themselves” is also a skill,

> Learning to say what you want is a skill True, but “learning to figure out what someone wanted to say but wasn’t able to express themselves” is also a skill, I expect LLMs will be able to learn that pretty well. Imagine you prompt ChatGPT4.5 and it doesn’t give you what you want. You click the thumbs down. ChatGPT says “Hold on, let me try again”. (Behind the scenes, OpenAI runs your prompt through their “prompt improver”, replaces your prompt with that prompt, and just shows you the output, no point showing the optimized prompt since it might be gibberish to a human). The new response actually is what you want, so you click thumbs up on the new output. That “thumbs down, thumbs up” pattern generates high quality labeled data for training the prompt improver, with very little cost.