3 ms·

I really didn't think anyone was applauding the "hallucinations" but maybe I just hang around people who aren't visionaries.

by glasss 3y ago

ethanbond 3y ago
I've seen more and more of it over the past few weeks. I work in clinical trials and just recently saw the claim that hallucinations can help researchers "expand the search space for molecules they didn't consider." I'm doubtful...
- 0xfae 3y ago
  Interesting. If the computer can simulate things that a human wouldn't have thought up, great, but if it's considering things that any sane person knows wouldn't work and passing them off as correct, framing it that way is laughable. That's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature. But then again, many discoveries have been made because someone trying something that shouldn't work.
  - mrbungie 3y ago
    Just a thought. Hypothesis making may be interpreted as interpolation/extrapolation in a hypothesis space plus some heuristics to reduce that search space based on previous knowledge/valid hypothesis, how much weight you give to said knowledge and evidence, and some soft and hard logic rules. That is in part what allows (some, not counting Dunning–Kruger here) humans how certain to be about what they're arguing/talking about. Maybe if the LLM is refeed with how likely (i.e. how many samples/tokens support it's response) is the output in its datasets, it may reevaluate its confidence and rephrase its answer. In the end, the real problem of hallucinations in LLMs is about its confidence in the correctness/plausability of its own output. But that is something 1) humans can also be guilty of; and 2) that is no purely negative, as it can be exploited to generate new knowledge when applying robust hypothesis validation and testing to said ideas. As you say in your last paragraph, people who've made discoveries in some areas have been treated as insane when tackling problems from a new perspective or when disregarding previous knowledge. If they weren't so strongheaded about their ideas, maybe we wouldn't even be posting in this forum right now. PS: Still, I agree LLMs commit laughable mistakes sometimes ;)
  - nomel 3y ago
    > knows wouldn't work and passing them off as correct, framing it that way is laughable. I don't think this completely true. Considering something that wouldn't work often still leads to ideas, because it kicks your brain of a rut it may be in. This is why it can be incredibly useful to brainstorm with someone that doesn't have expertise in a topic. They'll say zany things that can be inspiring! Random word sequences are a pretty common way to get inspiration. Something more "on topic" can't be that bad.
- jrockway 3y ago
  I think people like the hallucinations because they seem like the right answer. Imagine I make an AI assistant. You ask it, "how do I catch the bus into town?" If it replies, "run around in your backyard in a circle, then clap 3 times", (i.e. just use a random number generator) you'll think that's junk and will hate the system. But if it says "go to the corner of Main and Pine at 6:53 AM", then you'll probably do that. No matter that there is no bus route on Main or Pine. It sounds right, so you'll want to follow up. I imagine the same applies to searching for novel molecules. "This looks kind of right, I never would have thought of that", so you get a grant to synthesize it, do some clinical trials, and find that it doesn't work at all. But hey, you had something to do for a couple years, and that outcome happened in the non-AI process all the time too. By the time there is enough data that points strongly at the AI suggestions being as good as random chance, the AI gold rush will be over and cashed out. Or, the hallucinations will turn out to be really good and we'll cure cancer or whatever. What matters is that whoever made the model makes a lot of money either way.
- jug 3y ago
  I mean, statistically it's maybe bound to happen but if it's drowned by bullshit it's probably not a very useful method anyway due to running costs.
deleted 3y ago
[deleted]

I really didn't think anyone was applauding the "hallucinations" but maybe I just hang around people who aren't visionaries.

by glasss 3y ago

I really didn't think anyone was applauding the "hallucinations" but maybe I just hang around people who aren't visionaries.

ethanbond 3y ago

I've seen more and more of it over the past few weeks. I work in clinical trials and just recently saw the claim that hallucinations can help researchers "expand the search space for molecules they didn't consider." I'm doubtful...

0xfae 3y ago
Interesting. If the computer can simulate things that a human wouldn't have thought up, great, but if it's considering things that any sane person knows wouldn't work and passing them off as correct, framing it that way is laughable. That's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature. But then again, many discoveries have been made because someone trying something that shouldn't work.
- mrbungie 3y ago
  Just a thought. Hypothesis making may be interpreted as interpolation/extrapolation in a hypothesis space plus some heuristics to reduce that search space based on previous knowledge/valid hypothesis, how much weight you give to said knowledge and evidence, and some soft and hard logic rules. That is in part what allows (some, not counting Dunning–Kruger here) humans how certain to be about what they're arguing/talking about. Maybe if the LLM is refeed with how likely (i.e. how many samples/tokens support it's response) is the output in its datasets, it may reevaluate its confidence and rephrase its answer. In the end, the real problem of hallucinations in LLMs is about its confidence in the correctness/plausability of its own output. But that is something 1) humans can also be guilty of; and 2) that is no purely negative, as it can be exploited to generate new knowledge when applying robust hypothesis validation and testing to said ideas. As you say in your last paragraph, people who've made discoveries in some areas have been treated as insane when tackling problems from a new perspective or when disregarding previous knowledge. If they weren't so strongheaded about their ideas, maybe we wouldn't even be posting in this forum right now. PS: Still, I agree LLMs commit laughable mistakes sometimes ;)
- nomel 3y ago
  > knows wouldn't work and passing them off as correct, framing it that way is laughable. I don't think this completely true. Considering something that wouldn't work often still leads to ideas, because it kicks your brain of a rut it may be in. This is why it can be incredibly useful to brainstorm with someone that doesn't have expertise in a topic. They'll say zany things that can be inspiring! Random word sequences are a pretty common way to get inspiration. Something more "on topic" can't be that bad.
jrockway 3y ago
I think people like the hallucinations because they seem like the right answer. Imagine I make an AI assistant. You ask it, "how do I catch the bus into town?" If it replies, "run around in your backyard in a circle, then clap 3 times", (i.e. just use a random number generator) you'll think that's junk and will hate the system. But if it says "go to the corner of Main and Pine at 6:53 AM", then you'll probably do that. No matter that there is no bus route on Main or Pine. It sounds right, so you'll want to follow up. I imagine the same applies to searching for novel molecules. "This looks kind of right, I never would have thought of that", so you get a grant to synthesize it, do some clinical trials, and find that it doesn't work at all. But hey, you had something to do for a couple years, and that outcome happened in the non-AI process all the time too. By the time there is enough data that points strongly at the AI suggestions being as good as random chance, the AI gold rush will be over and cashed out. Or, the hallucinations will turn out to be really good and we'll cure cancer or whatever. What matters is that whoever made the model makes a lot of money either way.
jug 3y ago
I mean, statistically it's maybe bound to happen but if it's drowned by bullshit it's probably not a very useful method anyway due to running costs.

deleted 3y ago

[deleted]