3 ms·

It seems that we are in agreement. > The piece that would make them distinguish "true" from "strongly correlated with the corpus" is not yet discovered, AFAIK

by otikik 2y ago

It seems that we are in agreement. > The piece that would make them distinguish "true" from "strongly correlated with the corpus" is not yet discovered, AFAIK That piece is what would categorize us into not "hallucinating". I don't think it has been discovered, and I don't know wether it exists. It could be just that that our corpus is huge.

mewpmewp2 2y ago
But people also constantly make mistakes, produce invalid output, misremember things, state things incorrectly. Depending on the environment, they do this with varying frequencies. It is always a balance of truth vs action, where if you always safely say "I'm not sure, I'd rather not answer." you will never be able to do anything in life, vs if you always try to answer you will frequently be wrong. LLMs are biased to try to answer in their training sets, so they will try to make an answer up, because otherwise they wouldn't be helpful enough as assistants. You could train LLMs to bias more towards only answering when they are confident, e.g. "Paris is the capital of France", but they wouldn't be as useful then. I would also think it's not really "hallucinating", because "hallucinating" is error in processing the input causing it to be malformed. It implies that the input text you put in as a prompt was somehow invalidly processed to something else. When an LLM is wrong I would rather think it's trying to find an answer from a compressed dataset "hoping" it will be correct, because that's how it's finetuned to do.
- otikik 2y ago
  > But people also constantly make mistakes I absolutely agree. As far as I know, the judge is on the table about whether we do anything different from them. > "hallucinating" is error in processing the input causing it to be malformed For now, "Erroneous" is a concept exclusive to us humans. To a model, a response is not "erroneous". It's just the result of processing an input, and it is indistinguishable from any other. How do we bridge that gap? No idea. Perhaps it will happen eventually, by sheer quantity. Perhaps the algorithms need to be implemented in a different way. Perhaps we need different kinds of models interacting with each other in order to mimic the kind of response a human would give (including the "I don't know"s). All I know is that we are not there yet.

by otikik 2y ago

mewpmewp2 2y ago

But people also constantly make mistakes, produce invalid output, misremember things, state things incorrectly. Depending on the environment, they do this with varying frequencies. It is always a balance of truth vs action, where if you always safely say "I'm not sure, I'd rather not answer." you will never be able to do anything in life, vs if you always try to answer you will frequently be wrong. LLMs are biased to try to answer in their training sets, so they will try to make an answer up, because otherwise they wouldn't be helpful enough as assistants. You could train LLMs to bias more towards only answering when they are confident, e.g. "Paris is the capital of France", but they wouldn't be as useful then. I would also think it's not really "hallucinating", because "hallucinating" is error in processing the input causing it to be malformed. It implies that the input text you put in as a prompt was somehow invalidly processed to something else. When an LLM is wrong I would rather think it's trying to find an answer from a compressed dataset "hoping" it will be correct, because that's how it's finetuned to do.

otikik 2y ago
> But people also constantly make mistakes I absolutely agree. As far as I know, the judge is on the table about whether we do anything different from them. > "hallucinating" is error in processing the input causing it to be malformed For now, "Erroneous" is a concept exclusive to us humans. To a model, a response is not "erroneous". It's just the result of processing an input, and it is indistinguishable from any other. How do we bridge that gap? No idea. Perhaps it will happen eventually, by sheer quantity. Perhaps the algorithms need to be implemented in a different way. Perhaps we need different kinds of models interacting with each other in order to mimic the kind of response a human would give (including the "I don't know"s). All I know is that we are not there yet.