4 ms·
Not if you let your LLM grounds its truth in established facts. Otherwise they would be useless for programming for example.
by lukan 2mo ago
Not if you let your LLM grounds its truth in established facts. Otherwise they would be useless for programming for example.
- saghm 2mo agoI don't understand what this means. I use LLMs daily for my work in programming things, and they regularly will assert things that are not accurate.
- lukan 2mo agoWhen you let the the agent do a test, or tell it to read that doc first, you will ground it in reality. Doesn't mean they are 100% reliable. But without and on their own without access to grounding information, they halluzinate wildly.
- saghm 2mo agoThe parent comment said that these types of behaviors are inherent to how LLMs work, and your response was to "ground them in facts". From what I can tell, this does not meaningfully change the original point the parent comment was making that the flaw is inherent; throwing a bunch of extra context at it to try to make it happen less often is useful, but it's still just a best-effort mitigation for the behavior, not somehow a way of literally changing the inherent nature of it.
- stevage 2mo agoA lot like humans, really. People regularly cite something they read, or quote a stat that turns out to be just completely inaccurate. But if you look up the thing, then you have facts again.
- saghm 2mo agoSure, and by the same logic, if I was trying to cook a steak, I would not trust an arbitrary human to know the correct temperature off the top of their head; I'd want someone who I could trust had actual experience with the task I was trying to perform. The difference is that most humans are fully able to recognize whether they've cooked steak often enough to know the correct temperature off the top of their head, and they will say "I don't know" to most random arbitrary questions you ask them outside of their experience. I've yet to see an LLM product aimed at general usage for individuals be willing to say this without someone having to literally direct them to give that as an answer if they're not sure.
- lern_too_spel 2mo agoHis point was that, just like humans, if you ask the LLM to look it up, it will give you the correct answer. Just like humans, if you don't ask them to look it up, you don't know what you're going to get.
- saghm 2mo agoAnd my point is that I've never met a human who confidently asserts incorrect information in such a broad range of domains rather than just admitting that they don't know
- lern_too_spel 2mo agoThe point is that it's trivial to fix the problem of cooking the waterfowl or writing your code with existing tools. Just ask them to verify.
- saghm 2mo agoIf you have to say "don't make up something" for every possible question you ask in order for it not to make up something, that's a massive usability issue for regular people. If saying "don't make up something" will still result in it making up something up some of the time, that also might be a massive usability issue depending on if "some of the time" means 0.0001% or 1%. Having a natural language interface where you need to go out of your way to specify that you want an accurate answer rather than just a plausible one defeats the entire purpose of it being a natural language interface for normal people. In certain professional contexts, it can be useful, but I don't buy it at all that it makes sense to ask everyone in their everyday lives to go out of their way to specify that they actually want correct answers to their questions.
- lern_too_spel 2mo ago> If you have to say "don't make up something" for every possible question you ask You don't. It goes in the system prompt.
- elorant 2mo agoSo why isn't this the default mode then?
- lukan 2mo agoMore expensive.
- whstl 2mo agoThey're only useful for programming because version control, command+Z and "deploying a new version" exists. Take that away and you'll barely be able to make an app that display a pigeon riding a bicycle (or whatever you ppl are doing these days).