5 ms·
People have made the point many times before, that “hallucination” is mainly what generative AI does and not the exception, but most of the time it’s a useful h
by Isamu 1y ago
People have made the point many times before, that “hallucination” is mainly what generative AI does and not the exception, but most of the time it’s a useful hallucination.
I describe it as more of a “mashup”, like an interpolation of statistically related output that was in the training data.
The thinking was in the minds of the people that created the tons of content used for training, and from the view of information theory there is enough redundancy in the content to recover much of the intent statistically. But some intent is harder to extract just from example content.
So when generating statically similar output, the statistical model can miss the hidden rules that were a part of the thinking that went into the content that was used for training.
- alan-crowe 1y agoThere is an interesting phenomenon with polynomial interpolation called Runge Spikes. I think "Runge Spikes" offers a better metaphor than "hallucination" and argue the point: https://news.ycombinator.com/item?id=43612517 https://news.ycombinator.com/item?id=43612517
- Hackbraten 1y ago> the statistical model can miss the hidden rules that were a part of the thinking that went into the content that was used for training. Makes sense. Hidden rules such as, "recommending a package works only if I know the package actually exists and I’m at least somewhat familiar with it." Now that I think about it, this is pretty similar to cargo-culting.
- delusional 1y agoAnd cargo-culting is in fact exactly what happens when people act as LLM's.
- qwertytyyuu 1y agoLLMs don’t really “know” though.mif you look at the recent Anthropic findings, they show that large language models can do math like addition but they do it weird way and when you asked the model how they arrive to the solution they provide method that is completely different to how they actually do it
- Hackbraten 1y agoThat's the point. It's one of the implicit, real-world rules that were underlying the training set.
- palmotea 1y ago> Now that I think about it, this is pretty similar to cargo-culting. In other news: the CTO just sent out an email stating we should be using AI every day as our "thought partner."
- Jensson 1y agoVC-culting actually works though, if you act like a unicorn people will invest.
- zwnow 1y ago"useful hallucination" so much AI glazing its crazy
- brookst 1y ago“Glazing” is such performative rhetoric its hilarious
- Izkata 1y agoIt's (I'm pretty sure) Gen Z slang. I only started hearing it sometime this year.
- reaperducer 1y ago"useful hallucination" so much AI glazing its crazy I'm still a fan of the standard term "lying." Intent, or a lack thereof, doesn't matter. It's still a lie.
- marcosdumay 1y agoIf intent doesn't matter, is it still lying when the reality happens to coincide with what the machine says? Because the OP's name seems way more descriptive and easier to generalize.
- diggan 1y agoSo you're saying deliberate deception, mistaken statements and negligent falsehoods should all be considered the same thing, regardless? Personally, I'd be scared if LLMs were proven to be deliberately deceptive, but I think they currently fall in the two later camps, if we're doing human analogies.
- codr7 1y agoHave you asked your LLMs if they're capable of lying? Did the answers strike you as deceptive?
- Alupis 1y ago
- richardw 1y agoOur brains killed us if we figured things out wrong. We’d get eaten. We learned to get things right enough, and how to be pretty right, fast, even when we didn’t know the new context (plants, animals, snow, lava). LLM’s are just so happy to generate enough tokens that look right ish. They need so many examples driven into them during training. The map is not the territory, and we’re training them on the map of our codified outputs. They don’t actually have to survive. They’re pretty amazing but of course they’re absolutely not doing what we do, because success for us and them look so different. We need to survive. (Please can we not have one that really wants to survive.)
- DonHopkins 1y agoEdible AI! Now there's an idea. That would force it to evolve.
- ToucanLoucan 1y ago> People have made the point many times before, that “hallucination” is mainly what generative AI does and not the exception, but most of the time it’s a useful hallucination. Oh yeah that's exactly what I want from a machine intelligence, a "best friend who knows everything about me," is that they just make shit up that they think I'd like to hear. I'd really love a personal assistant that gets me and my date a reservation at a restaurant that doesn't exist. That'll really spice up the evening. The mental gymnastics involved in the AI community are truly pushing the boundaries of parody at this point. If your machines mainly generate bullshit, they cannot be serious products. If on the other hand they're intelligent, why do they make up so much shit? You just can't have this both ways and expect to be taken seriously.
- simonw 1y agoOne of the main reason LLMs are unintuitive and difficult to use is that you have to learn how to get useful results out of fundamentally unreliable technology. Once you figure out how to do that they're absurdly useful. Maybe a good analogy here is working with animals? Guide dogs, sniffer dogs, falconry... all cases where you can get great results but you have to learn how to work with a very unpredictable partner.
- ToucanLoucan 1y ago> One of the main reason LLMs are unintuitive and difficult to use is that you have to learn how to get useful results out of fundamentally unreliable technology. Name literally any other technology that works this way. > Guide dogs, sniffer dogs, falconry... Guide dogs are an imperfect solution to an actual problem: the inability for people to see. And dogs respond to training far more reliably than LLMs respond to prompts. Sniffer dogs are at least in part bullshit and have been shown in many studies to respond to the subtle cues of their handlers far more reliably than anything they actually smell. And the best of part of them is they also (completely outside their own control mind you) ruin lives when falsely detecting drugs on cars that look a way the officer handling them thinks means they have drugs inside. And falconry is a hobby.
- worik 1y ago> that “hallucination” is mainly what generative AI does Me too