4 ms·

> It sounds as if it made perfect sense, except to know that it actually completely doesn't you have to spend hours reading the spec first. This has been my ex

> It sounds as if it made perfect sense, except to know that it actually completely doesn't you have to spend hours reading the spec first. This has been my experience as well. As I work on some projects I’ve been asking it basic questions for things I’ve already learned or know quite well. Once you deviate past the basics (content you’d find in common tutorials) the hallucination rate is out of control. The surreal part is that it all sounds so confident and plausible. The way it puts words together satisfies my “that sounds about right” reflex, but the actual content is incorrect or illogical quite frequently. If I continue prodding I can get it to come up the right answer many times, but I have to ask it a lot of leading questions to get there.

WA 3y ago
So basically, the typical disinformation asymmetry gets even worse. It’s now cheaper than ever to produce fake information and even harder to check facts. And most is probably not even done on purpose.
Animats 3y ago
> This has been my experience as well. As I work on some projects I’ve been asking it basic questions for things I’ve already learned or know quite well. Once you deviate past the basics (content you’d find in common tutorials) the hallucination rate is out of control. That's inherent in the approach. Large language models reflect the majority opinion of the training set. If there's not enough source material in an area that the same thing hasn't been covered in a few different ways, the thing gets lost. This is well known. What's being done about it?
- StrangeDoctor 3y ago
  But it seems like they aren’t just storing the majority opinion, or that opinions couldn’t be influenced from other related areas. I am assuming it’s a harder problem to only select for the majority at the exclusion of else vs sampling it all. With prompting maybe we could tease out the minority views, but I could also see it just confidently making shit up to a higher degree.

WA 3y ago

So basically, the typical disinformation asymmetry gets even worse. It’s now cheaper than ever to produce fake information and even harder to check facts. And most is probably not even done on purpose.

Animats 3y ago

> This has been my experience as well. As I work on some projects I’ve been asking it basic questions for things I’ve already learned or know quite well. Once you deviate past the basics (content you’d find in common tutorials) the hallucination rate is out of control. That's inherent in the approach. Large language models reflect the majority opinion of the training set. If there's not enough source material in an area that the same thing hasn't been covered in a few different ways, the thing gets lost. This is well known. What's being done about it?

StrangeDoctor 3y ago
But it seems like they aren’t just storing the majority opinion, or that opinions couldn’t be influenced from other related areas. I am assuming it’s a harder problem to only select for the majority at the exclusion of else vs sampling it all. With prompting maybe we could tease out the minority views, but I could also see it just confidently making shit up to a higher degree.