19 ms·
Five minutes playing with any of these freely-available LLMs (and the commercial ones, to be honest) will be enough to demonstrate that they freely hallucinate
by wzdd 3y ago
Five minutes playing with any of these freely-available LLMs (and the commercial ones, to be honest) will be enough to demonstrate that they freely hallucinate information when you get into any detail on any topic at all. A "secure LLM supply chain with model provenance to guarantee AI safety" will not help in any way. The models in their current form are simply not suitable for education.
- dcow 3y agoObviously the models will improve. Then you’re going to want this stuff. What’s the harm in starting now?
- sieabahlpark 3y ago[dead]
- LordShredda 3y agoCitation on "will"
- tudorw 3y agoactually, are we sure they will improve, if there is emergent unpredicted behaviour in the SOTA models we see now, then how can we predict if what emerges from larger models will actually be better, it might have more detailed hallucinations, maybe it will develop its own version of cognitive biases or inattentional blindness...
- dcow 3y agoHow do we know the sun will rise tomorrow?
- muh_gradle 3y agoPoor comparison
- dcow 3y agoNo so! Either both the comments are meaningful, or both are meaningless.
- jchw 3y agoI don't understand why that is necessarily true.
- dcow 3y agoBecause they are both statements about the future. Either humans can inductively reason about future events in a meaningful way, or they can’t. So both statements are equally meaningful in a logical sense. (Hume) Models have been improving. By induction they’ll continue until we see them stop. There is no prevailing understanding of models that lets us predict a parameter and/or training set size after which they’ll plateau. So arguing “how do we know they’ll get better” is the same as arguing “how do we know the sun will rise tomorrow”… We don’t, technically, but experience shows it’s the likely outcome.
- jchw 3y agoIt's comparing the outcome that a thing that has never happened before will (no specified time frame), versus the outcome that a thing that has happened billions of times will suddenly not happen (tomorrow). The interesting thing is, we know for sure the sun will eventually die. We do not know at all that LLMs will ever stop hallucinating to a meaningful degree. It could very well be that the paradigm of LLMs just isn't enough.
- dcow 3y agoWhat? LLMs have been improving for years and years as we’ve been researching and iterating on them. “Obviously they’ll improve” does not require “solving the hallucination problem”. Humans hallucinate too, and we’re deemed good enough.
- ysavir 3y agoOriginally: very few input toggles with little room for variation and with consistent results. These days: Modern technology allows us to monitor the location of the sun 24/7.
- TheMode 3y agoBecause it has been the case for billions of years, and we adapted our assumptions as such. We have no strong reason to believe that we will figure out ways to indefinitely improve these chat bots. It may, but it may also not, at that point you are just fantasizing.
- dcow 3y agoWe’ve seen models improve for years now too. How many iterations are required for one to inductively reason about the future?
- arcticbull 3y agoHow many days does it take before the turkey realizes it’s going to get its head cut off on its first thanksgiving? Less glibly I think models will follow the same sigmoid as everything else we’ve developed and at some point it’ll start to taper off and the amount of effort required to achieve better results becomes exponential. I look at these models as a lossy compression logarithm with elegant query and reconstruction. Think JPEG quality slider. The first 75% of the slider the quality is okay and the size barely changes, but small deltas yield big wins. And like an ML hallucination the JPEG decompressor doesn’t know what parts of the image it filled in vs got exactly right. But to get from 80% to 100% you basically need all the data from the input. There’s going to be a Shannon’s law type thing that quantifies this relationship in ML by someone who (not me) knows what they’re talking about. Maybe they already have? These models will get better yes but only when they have access to google and bing’s full actual web indices.
- drw85 3y agoI don't think access to bing or google solves this problem. Right now, there are many questions that the internet gives unclear answers to. Try to find out if a plant is toxic for cats via google. Many times the results say both yes and no and it's impossible to assume which one is true based on the count of the results. Feeding the models more garbage data will not make the results any better.
- tudorw 3y agoone day it won't...
- mrguyorama 3y agoBecause we understand newtons laws of motion and the sun and earth seem to, with extreme certainty, follow such laws, and by evaluating those laws forward into the future we see the orbits will continue such that the sun rises. It's not magic. What rules and predictions can reliably describe how much machine learning will advance over time?
- krainboltgreene 3y ago> Obviously the models will improve Says who? The Hot Hand Fallacy Division?
- dcow 3y agoThe trend. Obviously nobody can predict the future either. But models have been improving steadily for the last 5 years. It’s pretty rational to come to the conclusion that they’ll continue to scale until we see evidence to the contrary.
- krainboltgreene 3y ago"the trend [says that it will improve]" followed by "nobody can predict the future either" is just gold. > It’s pretty rational No, that's why it's a fallacy.
- dcow 3y agoYou’re misunderstanding me. It’s also a fallacy to believe the sun will rise tomorrow. Everything is a fallacy if you can’t inductively reason. That’s the point, we agree.
- namaria 3y agoNonsense. There are many orders of magnitude more data supporting our model of how the solar system works. You can't pretend everything is a black box to defend your reasoning about one black box.
- dcow 3y agoI’m not pretending anything is a black box. The sun is going to run out of fuel. We have no idea when that will happen. There is a philosophical treatment for what I’m arguing, we just like to ignore it and let our egos convince us we’re a lot more certain about the world than we are.
- wzdd 3y agoEven if the models improve to the point where hallucinations aren't a problem for education, which is not obvious, then it's not clear that enforcing a chain of model provenance is the correct approach to solve the problem of "poisoned" data. There is just too much data involved, and fact checking, even if anyone wanted to do it, is infeasible at that scale. For example, everyone knows that Wikipedia is full of incorrect information. Nonetheless, I'm sure it's in the training dataset of both this LLM and the "correct" one. So the answer to "why not start now" is "because it seems like it will be a waste of time".
- ben_w 3y agoMostly agree, but: > So the answer to "why not start now" is "because it seems like it will be a waste of time". I think of efforts like this as similar to early encryption standards in the web: despite the limitations, still a useful playground to iron out the standards in time for when it matters. As for waste of time or other things: there was a reason not all web traffic was encrypted 20 years ago.
- emporas 3y agoAgree with most of your points, but a LargeLM, or a SmallLM for that matter, to construct a simple SQL query and put it in a database, they get it right many times already. GPT gets it right most of the time. Then as a verification step, you ask one more model, not the same one, "what information got inserted the last hour in the database?" Chances of one model to hallucinate and say it put the information in the database, and the other model to hallucinate again with the correct information, are pretty slim. [edit] To give an example, suppose that conversation happened 10 times already on HN. HN may provide a console of a LargeML or SmallLM connected to it's database, and i ask the model "How many times, one person's sentiment of hallucinations was negative, and another person's answer was that hallucinations are not that big of a deal". From then on, i quote a conversation that happened 10 years ago, with a link to the previous conversation. That would enable more efficient communication.
- bredren 3y agoMany sources of information contain inaccuracies, either known at the time of publication or learned afterward. Education involves doing some fact checking and critical thinking. Regardless of the strength of the original source. It seems like using LLMs in any serious way will require a variety of techniques to mitigate their new, unique reasons for being unreliable. Perhaps a “chain of model provenance” becomes an important one of these.
- csmpltn 3y ago> "Obviously the models will improve." Found the venture capitalist!
- dcow 3y agoI think people are conflating “get better” with “never hallucinate” (and I guess in your mind “make money”). They’re gonna get better. Will they ever be perfect or even commercially viable? Who knows.
- krater23 3y agoNo, a signature will not guarantee anything about if the model is trained with correct data or with fake data. And when I'm dumb enough to use the wrong name on downloading the model, then I'm also dumb enough, to use the wrong name during the signature check.
- z3c0 3y agoWhile I agree with them, I've found a lot of the other responses to not be conducive to you actually understanding where you misunderstood the situation. AI performance often decreases at a logarithmic rate. Simply put, it likely will hit a ceiling, and very hard. To give a frame of reference, think of all the places that AI/ML already facilitate elements of your life (autocompletes, facial recognition, etc). Eventually, those hit a plateau that render it unenthusing. LLMs are destined for the same. Some will disagree, because its novelty is so enthralling, but at the end of the day, LLMs learned to engage with language in a rather superficial way when compared to how we do. As such, it will never capture the magic of denotation. Its ceiling is coming, and quickly, though I expect a few more emergent properties to appear before that point.
- marricks 3y ago> Obviously the models will improve I mean, to some extent, but isn't reasonable to assume hallucination is a hard problem? Hallucination shows there's plenty of things they didn't actually learn, and are just good at seeming they learned. Like, if it gets exponentially harder to train them it's possible the level of hallucination will improve far worse than linearly even.
- tudorw 3y agoI agree, their needs to be human oversight, I find them interesting, but not sure beyond creative tasks, what I would actually use it for, I have no interest in replacing humans, why would I, so, augmenting human creativity with pictures, stories, music, yes, that works, it does it well. Education, law, medical, being in charge of anything, not so much.
- deleted 3y ago[deleted]
- LawTalkingGuy 3y ago"You're holding it wrong." A language model isn't a fact database. You need to give the facts to the AI (either as a tool or as part of the prompt) and instruct it to form the answer only from there. That 'never' goes wrong in my experience, but as another layer you could add explicit fact checking. Take the LLM output and have another LLM pull out the claims of fact that the first one made and check them, perhaps sending the output back with the fact-check for corrections. For those saying "the models will improve", no. They will not. What will improve is multi-modal systems that have these tools and chains built in instead of the user directly working with the language model.