5 ms·
This largely seems like an alternative way of saying "you have to validate the results of an LLM." Is there any "premature closure" risk if you simply validate
by nonethewiser 1y ago
This largely seems like an alternative way of saying "you have to validate the results of an LLM." Is there any "premature closure" risk if you simply validate the results?
Premature closure is definitely a risk with LLMs but I think code is much less at risk because you can and SHOULD test it. But its a bigger problem for things you cant validate.
I might starting calling this "the original sin" with LLMs... not validating the output. There are many problems people have identified with using LLMs and perhaps all of them come back to not validating.
- bwfan123 1y ago> I might starting calling this "the original sin" with LLMs... not validating the output. I would rephrase it - the original sin of llms is not "understanding" what they output. By "understanding" it is meant the "the why of the output" - starting from the original problem and reasoning through to the output solution - ie, the causation process behind the output. What llms do is to pattern match a most "plausible output" to the input. But the output is not born out of a process of causation - it is born out of pattern matching. Humans can find meaning in the output of LLMs, but machines choke at it - which is why, LLM code looks fine at a first glance until someone tries to run it. Another way to put it is, LLMs sound persuasive to humans - but at the core are rote students who dont understand what they are saying.
- nonethewiser 1y agoI agree that its important to understand they are just predicting what you should expect to see. And that is a more fundamental truth about an LLM itself. But im thinking more in terms of using LLMs. That distinction doesn't really matter if the tests pass.
- wizzwizz4 1y agohttp://thecodelesscode.com/case/58 http://thecodelesscode.com/case/58