3 ms·
LLMs don't only hallucinate because of mistaken statements in their training data. It just comes hand-in-hand with the model's ability to remix, interpolate, an
by mitthrowaway2 2y ago
LLMs don't only hallucinate because of mistaken statements in their training data. It just comes hand-in-hand with the model's ability to remix, interpolate, and extrapolate answers to other questions that aren't directly answered in the dataset. For example if I ask ChatGPT a legal question, it might cite as precedent a case that doesn't exist at all (but which seems plausible, being interpolated from cases that do exist). It's not necessarily because it drew that case from a TV episode. It works the same way that GPT-3 wrote news releases that sounded convincing, matching the structure and flow of real articles.
Training only on factual data won't solve this.
Anyway, I can't help but feel saddened sometimes to see our talented people and investment resources being drawn in to developing these AI chatbots. These problems are solvable, but are we really making a better world by solving them?
- CamperBob2 2y agoThese problems are solvable, but are we really making a better world by solving them? When you ask yourself that question -- and you do ask yourself that, right? -- what's your answer?
- mitthrowaway2 2y agoI do, all the time! My answer is "most likely not". (I assumed that answer was implied by my expressing sadness about all the work being invested in them.) This is why, although I try to keep up-to-date with and understand these technologies, I am not being paid to develop them.
- bosch_mind 2y agoI mean, comon man, _most_ tech people work on in general is also useless. FAANG has a retainer on talent whether they are spending their time wisely or not. LLM stuff is not in a different echelon
- dweinus 2y ago100% I think the author is really misunderstanding the issue here. "Hallucination" is a fundamental aspect of the design of Large Language Models. Narrowing the distribution of the training data will reduce the LLM's ability to generalize, but it won't stop hallucinations.
- sean_pedersen 2y agoI agree in that a perfectly consistent dataset won't completely stop statistical language models from hallucinating but it will reduce it. I think it is established that data quality is more important than quantity. Bullshit in -> bullshit out, so a focus on data quality is good and needed IMO. I am also saying LMs output should cite sources and give confidence scores (which reflects how much the output is in or out of the training distrtibution).
- rtkwe 2y agoI think the problem is you need an extremely large quantity of data just to get the machine to work in the first place. So much so that there may not be enough to get it working on just "quality" data.
- bbor 2y agoWhat’s a non-statistical language model? And I think looking to the training data for sources is a little silly - that’s the training data for intuitive language use, not true statements about the world. If you haven’t checked it out yet, two terms you’d love are “RAG” and “Manuel De Landa”
- antihipocrat 2y agoHow would confidence scores work? Multiple passthroughs and a % attached to each statement according to how often it appeared in the generated result? If so, building this could be quite complex depending on the domain. In the legal field even one simple word that is changed can have large consequences.
- mehrau 2y ago[dead]
- _venkatasg 2y agoMost sentences in the world are not about truth or falsity. Training on a high quality corpus isn’t going to fix ‘hallucination’. The complete separation of facts from sentences is what makes LLMs powerful.
- bosch_mind 2y agoAI noob, but instead of training and fine tuning the LLM itself, don’t more specific and targeted embeddings paired with the model help alleviate the hallucination where you incorporate semantic search context with the question?