7 ms·

All they do is hallucinating. Their results are statistic extrapolations from a big corpus of data. For them a "true" fact "looks the same" as a "false, but st

by otikik 2y ago

All they do is hallucinating. Their results are statistic extrapolations from a big corpus of data. For them a "true" fact "looks the same" as a "false, but strongly correlated with the corpus" fact. The piece that would make them distinguish "true" from "strongly correlated with the corpus" is not yet discovered, AFAIK. It would probably need to be a different, context-specific tool. For example, if you are asking a model to write a program for you, the "verification step" would be a test bench for said program.

mewpmewp2 2y ago
What is the differing thing in humans that presumably could categorize us into not "hallucinating"? As far as I understand, every time we are querying our memories we are also slightly changing them, so there's no guarantee when we are trying to remember something that it is a true representation of reality of what happened. See also reliability of witness testimonies. Ultimately all knowledge and storage graph in humans and LLMs has to be compressed heavily and be statistical in its nature with compromises made about the accuracy vs how much you are able to store. We can't store 8k footage of everything we see in our brains and so we use a lot of compression techniques, likely statistical graphs in nature.
- otikik 2y ago
  It seems that we are in agreement. > The piece that would make them distinguish "true" from "strongly correlated with the corpus" is not yet discovered, AFAIK That piece is what would categorize us into not "hallucinating". I don't think it has been discovered, and I don't know wether it exists. It could be just that that our corpus is huge.
  - mewpmewp2 2y ago
    But people also constantly make mistakes, produce invalid output, misremember things, state things incorrectly. Depending on the environment, they do this with varying frequencies. It is always a balance of truth vs action, where if you always safely say "I'm not sure, I'd rather not answer." you will never be able to do anything in life, vs if you always try to answer you will frequently be wrong. LLMs are biased to try to answer in their training sets, so they will try to make an answer up, because otherwise they wouldn't be helpful enough as assistants. You could train LLMs to bias more towards only answering when they are confident, e.g. "Paris is the capital of France", but they wouldn't be as useful then. I would also think it's not really "hallucinating", because "hallucinating" is error in processing the input causing it to be malformed. It implies that the input text you put in as a prompt was somehow invalidly processed to something else. When an LLM is wrong I would rather think it's trying to find an answer from a compressed dataset "hoping" it will be correct, because that's how it's finetuned to do.
    - otikik 2y ago
      > But people also constantly make mistakes I absolutely agree. As far as I know, the judge is on the table about whether we do anything different from them. > "hallucinating" is error in processing the input causing it to be malformed For now, "Erroneous" is a concept exclusive to us humans. To a model, a response is not "erroneous". It's just the result of processing an input, and it is indistinguishable from any other. How do we bridge that gap? No idea. Perhaps it will happen eventually, by sheer quantity. Perhaps the algorithms need to be implemented in a different way. Perhaps we need different kinds of models interacting with each other in order to mimic the kind of response a human would give (including the "I don't know"s). All I know is that we are not there yet.
- AndrewDucker 2y ago
  We make correlations based on experience of reality. They make correlations based on experience of text. Text is only meaningful when it is correlated with reality. An LLM will see that "Dog" and "Woof" occur near each other in text a lot, but it doesn't know what a dog is or what a woof is. You do know that, because you have experienced an actual dog and heard it go woof.
  - mewpmewp2 2y ago
    But there's also plenty of things I haven't seen in real life and have only read about in books or text. Does it mean that I wouldn't know what they are as well? In addition you can feed LLMs image, audio input as well, and in theory you could feed any sensory data that we get. In fact it can describe what is on an image, to quite a lot of detail. Is knowing what a dog is being able to recognize it on an image, and is knowing what a woof is being able to recognize it in an audio? Ultimately, after all, all of it can be represented as input tokens, binary data or otherwise. We also receive constant input from the World, which could be represented in any various different forms and processing layers can transform it into different helpful symbolic things to problem solve based on that. LLMs in order to be able to logically consider what a dog might do given certain situations, would have needed to build some form of neural representation of the idea to be able to produce the output it can, that goes beyond parroting the training data in my view. I can give it an input in some interesting configurations and permutations and it's able to logically use the concepts to figure out what might happen.
    - AndrewDucker 2y ago
      "Does it mean that I wouldn't know what they are as well?" Are they explained in terms of things that you have seen in real life? Or in ways that build on that? Then you can certainly gain that by building on those models. And yes, if you can feed an attention-based neural network (not an LLM, those are specifically Language-based) sense data (and preferably action data too, for how it interacts with the world) then that would give it a picture of the world. And once it can recognise a dog then you can say "And we call this a dog!". There's research in to this already - see this piece https://www.economist.com/science-and-technology/2024/06/05/robots-are-suddenly-getting-cleverer-whats-changed https://www.economist.com/science-and-technology/2024/06/05/... (you may need to use archive.is to read the archive of that page). ==== These “vision-language-action models” (VLAMs) take in text and images, plus data relating to the robot’s presence in the physical world, including the readings on internal sensors, the degree of rotation of different joints and the positions of actuators (such as grippers, or the fingers of a robot’s hands). The resulting models can then answer questions about a scene, such as “can you see an apple?” But they can also predict how a robot arm needs to move to pick that apple up, as well as how this will affect what the world looks like." Grounding the model’s perception in the real world in this way greatly reduces hallucinations (the tendency for AI models to make things up and get things wrong). ====

by otikik 2y ago

mewpmewp2 2y ago

What is the differing thing in humans that presumably could categorize us into not "hallucinating"? As far as I understand, every time we are querying our memories we are also slightly changing them, so there's no guarantee when we are trying to remember something that it is a true representation of reality of what happened. See also reliability of witness testimonies. Ultimately all knowledge and storage graph in humans and LLMs has to be compressed heavily and be statistical in its nature with compromises made about the accuracy vs how much you are able to store. We can't store 8k footage of everything we see in our brains and so we use a lot of compression techniques, likely statistical graphs in nature.

otikik 2y ago
It seems that we are in agreement. > The piece that would make them distinguish "true" from "strongly correlated with the corpus" is not yet discovered, AFAIK That piece is what would categorize us into not "hallucinating". I don't think it has been discovered, and I don't know wether it exists. It could be just that that our corpus is huge.
- mewpmewp2 2y ago
  But people also constantly make mistakes, produce invalid output, misremember things, state things incorrectly. Depending on the environment, they do this with varying frequencies. It is always a balance of truth vs action, where if you always safely say "I'm not sure, I'd rather not answer." you will never be able to do anything in life, vs if you always try to answer you will frequently be wrong. LLMs are biased to try to answer in their training sets, so they will try to make an answer up, because otherwise they wouldn't be helpful enough as assistants. You could train LLMs to bias more towards only answering when they are confident, e.g. "Paris is the capital of France", but they wouldn't be as useful then. I would also think it's not really "hallucinating", because "hallucinating" is error in processing the input causing it to be malformed. It implies that the input text you put in as a prompt was somehow invalidly processed to something else. When an LLM is wrong I would rather think it's trying to find an answer from a compressed dataset "hoping" it will be correct, because that's how it's finetuned to do.
  - otikik 2y ago
    > But people also constantly make mistakes I absolutely agree. As far as I know, the judge is on the table about whether we do anything different from them. > "hallucinating" is error in processing the input causing it to be malformed For now, "Erroneous" is a concept exclusive to us humans. To a model, a response is not "erroneous". It's just the result of processing an input, and it is indistinguishable from any other. How do we bridge that gap? No idea. Perhaps it will happen eventually, by sheer quantity. Perhaps the algorithms need to be implemented in a different way. Perhaps we need different kinds of models interacting with each other in order to mimic the kind of response a human would give (including the "I don't know"s). All I know is that we are not there yet.
AndrewDucker 2y ago
We make correlations based on experience of reality. They make correlations based on experience of text. Text is only meaningful when it is correlated with reality. An LLM will see that "Dog" and "Woof" occur near each other in text a lot, but it doesn't know what a dog is or what a woof is. You do know that, because you have experienced an actual dog and heard it go woof.
- mewpmewp2 2y ago
  But there's also plenty of things I haven't seen in real life and have only read about in books or text. Does it mean that I wouldn't know what they are as well? In addition you can feed LLMs image, audio input as well, and in theory you could feed any sensory data that we get. In fact it can describe what is on an image, to quite a lot of detail. Is knowing what a dog is being able to recognize it on an image, and is knowing what a woof is being able to recognize it in an audio? Ultimately, after all, all of it can be represented as input tokens, binary data or otherwise. We also receive constant input from the World, which could be represented in any various different forms and processing layers can transform it into different helpful symbolic things to problem solve based on that. LLMs in order to be able to logically consider what a dog might do given certain situations, would have needed to build some form of neural representation of the idea to be able to produce the output it can, that goes beyond parroting the training data in my view. I can give it an input in some interesting configurations and permutations and it's able to logically use the concepts to figure out what might happen.
  - AndrewDucker 2y ago
    "Does it mean that I wouldn't know what they are as well?" Are they explained in terms of things that you have seen in real life? Or in ways that build on that? Then you can certainly gain that by building on those models. And yes, if you can feed an attention-based neural network (not an LLM, those are specifically Language-based) sense data (and preferably action data too, for how it interacts with the world) then that would give it a picture of the world. And once it can recognise a dog then you can say "And we call this a dog!". There's research in to this already - see this piece https://www.economist.com/science-and-technology/2024/06/05/robots-are-suddenly-getting-cleverer-whats-changed https://www.economist.com/science-and-technology/2024/06/05/... (you may need to use archive.is to read the archive of that page). ==== These “vision-language-action models” (VLAMs) take in text and images, plus data relating to the robot’s presence in the physical world, including the readings on internal sensors, the degree of rotation of different joints and the positions of actuators (such as grippers, or the fingers of a robot’s hands). The resulting models can then answer questions about a scene, such as “can you see an apple?” But they can also predict how a robot arm needs to move to pick that apple up, as well as how this will affect what the world looks like." Grounding the model’s perception in the real world in this way greatly reduces hallucinations (the tendency for AI models to make things up and get things wrong). ====