4 ms·
The difference is that a human will tell you things like "I think", "I'm pretty sure" or "I don't know" in order to manage expectations. The LLM will very matte
by xienze 2y ago
The difference is that a human will tell you things like "I think", "I'm pretty sure" or "I don't know" in order to manage expectations. The LLM will very matter-of-factly tell you something that's not right at all, and if you correct them the LLM will go and very confidently rattle off another answer based on what you just said, whether your were telling it the truth or not. If a human acted that way more than a few times we'd stop asking them questions or at least have to do a lot of "trust but verify." LLMs do this over and over again and we just kind of shrug our shoulders and go "well they do pretty good overall."
- smw 2y agoI can't count the number of times I've had a support person confidently tell me something that is obviously not applicable to my problem and makes completely flawed assumptions about cs/physics/networking/logic. I get a lot of correct answers from llms, but sometimes they make shit up. Most of the time, it's some function in a library that doesn't actually exist. Sometimes even the wrong answers are useful because they tell me where to look in the reference docs. Ask it to search the web and cite sources, makes it easier to verify the answer. I don't appreciate what's going on with AI art and AI generated slop, but the idea that they aren't a useful tool is just wild to me.
- xienze 2y agoI'm not saying it's not useful, I'm saying that we hold humans giving us answers to a much higher standard than LLMs.