4 ms·
> There is no good reason this should not work everywhere else, in exactly the same way. Take for example a large retailer who has a large internal knowledge ba
by swatcoder 4y ago
> There is no good reason this should not work everywhere else, in exactly the same way. Take for example a large retailer who has a large internal knowledge base. Train an LLM on that corpus, ask the knowledge base any question.
Since LLM’s can’t scope themselves to be strictly true or accurate, there are indeed good reasons, like liability for false claims and added traditional support burden from incorrect guidance.
Everybody is getting so far ahead of the horse with this stuff, but we’re just not there yet and don’t know for sure how far we’re going to get.
- iandanforth 4y ago"LLM’s can’t scope themselves to be strictly true or accurate" This isn't true though the techniques to do so are 1. Not as yet widespread 2. Decrease the generality of the model and its perceived effectiveness.
- shawntan 4y agoI'm interested to hear what these techniques are. Decreasing the generality will help, but I fail to see how that scopes the output. At best that mitigates the errors to an extent.
- 256lie 4y agoconformal prediction
- shawntan 4y agoPredicting a set of answers to some confidence interval would still result in hallucinated answers.
- HDThoreaun 4y agolow probability answers get shot to a human and reviewed for model improvement.
- fulafel 4y agoRequiring the answers to automatically verifiable, or having answers be inputs to a reliable query system?
- le-mark 4y agoI’d rather use the reliable query system!
- astockwell 4y agoIf they are accurate for ~80% of the questions, they will be as accurate as any 1st or 2nd line help desk.
- bee_rider 4y agoHmm. Hypothetically if a human on first line help desk gives advice that is so completely bad as to be a crime, are they liable or the company? Because I guess a chat-bot would definitely not be liable.
- astockwell 4y agoHow often is your fast food order 100% correct?
- iudqnolq 4y agoCorrectness isn't one-dimensional. A wrong fast-food order might substitute or leave something out. There's essentially no chance the employee will swap in a random product from some other store. But in this example the AI could hallucinate a statement attributed to you it actually formed by putting together reddit comments.
- bee_rider 4y agoSometimes they accidentally include tomatoes but they rarely include bombs.
- fomine3 4y agoWrong food could be equivalent to bombs in worst case for who have allergy
- deleted 4y ago[deleted]
- perryizgr8 4y ago> Since LLM’s can’t scope themselves to be strictly true or accurate Bing tries to solve this and succeeds somewhat. It will insert Wikipedia style citations against each of its claims. You can visit them and verify the statement if you want. And I do it often. No reason why a future DocAI can't link to specific sections in internal documents whenever it answers a question.