3 ms·

Incomplete training data is kind of a pointless thing to measure. Isn’t incomplete data the whole point of learning in general? The reason why we have machine

Incomplete training data is kind of a pointless thing to measure. Isn’t incomplete data the whole point of learning in general? The reason why we have machine learning is because data was incomplete. If we had complete data we don’t need ml. We just build a function that maps the input to output based off the complete data. Machine learning is about filling in the gaps based off of a prediction. In fact this is what learning in general is doing. It means this whole thing about incomplete data applies to human intelligence and learning as well. Everything this theory is going after basically has application learning and intelligence in general. So sure you can say that LLMs will always hallucinate. But humans will also always hallucinate. The real problem that needs to be solved is: how do we get LLMs to hallucinate in the same way humans hallucinate?

skzv 2y ago
Yes, but it also makes a huge difference whether we are asking the model to interpolate or extrapolate. Generally speaking, models perform much better on the former task, and have big problems with the latter.
- _heimdall 2y ago
  Without any ability to reason about the known facts, are we better off with LLMs trying to interpolate at all rather than acting as a huge search space that returns only references? If an LLM has the exact answer needed it could simply be returned without needing to be rephrased or predicted at all. If the exact answer is not found, or if the LLM attempts to paraphrase the answer through prediction, isn't it already extrapolating? That doesn't even get to the point where it is attempting to combine multiple pieces of training data or fill in blanks that it hasn't seen.
abernard1 2y ago
> Machine learning is about filling in the gaps based off of a prediction. I think this is a generous interpretation of network-based ML. ML was designed to solve problems. We had lots of data, and we knew large amounts of data could derive functions (networks) as opposed to deliberate construction of algorithms with GOFAI. But "intelligence" with ML as it stands now is not how humans think. Humans do not need millions of examples of cats to know what a cat is. They might need two or three, and they can permanently identify them later. Moreover, they don't need to see all sorts of "representative" cats. A human could see a single instance of black cat and identify all other types of house cats as cats correctly. (And they do: just observe children). Intelligence is the ability to come up with a solution without previous knowledge. The more intelligent an entity is, the less data it needs. As we approach more intelligent systems, they will need less data to be effective, not more.
- imoverclocked 2y ago
  > Humans do not need millions of examples of cats to know what a cat is. We have evolved over time to recognize things in our environment. We also don’t need to be told that snakes are dangerous as many humans have an innate understanding of that. Our training data is partially inherited.
  - abernard1 2y ago
    DNA is around ~725 megabytes. There is no "snakes are dangerous" encoded in there. Our training data is instinctual behaviors we recognize from our sensory inputs. The idea that we're "pre-trained" the way an LLM is (with hundreds of lifetimes of actual sensory experience) is incorrect.
    - imoverclocked 2y ago
      > "The results show that the brain has special neural circuits to detect snakes, and this suggests that the neural circuits to detect snakes have been genetically encoded," Nishijo said. > The monkeys tested in the experiment were reared in a walled colony and neither had previously encountered a real snake. [1] https://www.ucdavis.edu/news/snakes-brain-are-primates-hard-wired-see-snakes#:~:text=%22The%20results%20show%20that%20the,able%20to%20collaborate%20with%20neuroscientists https://www.ucdavis.edu/news/snakes-brain-are-primates-hard-....
      - abernard1 2y ago
        All this is saying is our neural architecture has specific places that respond to danger and the instinct of fear. Anyone who has seen an MRI knows this is the case. It does not mean actual knowledge of snakes is encoded in our DNA. Our brains are "trained" on the data they receive during early development. To the degree that evolutionary pressures stored "data," it stored data about how to make our brains (compute) or physiology more effective. Modern ML tries to shortcut this by making the architecture dumb and the data numerous. If the creators of ML were in charge of a forced evolution, they'd be arguing we need to make DNA 100s of gigabytes and that we needed to store all the memories of our ancestors in it.

skzv 2y ago

Yes, but it also makes a huge difference whether we are asking the model to interpolate or extrapolate. Generally speaking, models perform much better on the former task, and have big problems with the latter.

_heimdall 2y ago
Without any ability to reason about the known facts, are we better off with LLMs trying to interpolate at all rather than acting as a huge search space that returns only references? If an LLM has the exact answer needed it could simply be returned without needing to be rephrased or predicted at all. If the exact answer is not found, or if the LLM attempts to paraphrase the answer through prediction, isn't it already extrapolating? That doesn't even get to the point where it is attempting to combine multiple pieces of training data or fill in blanks that it hasn't seen.

abernard1 2y ago

> Machine learning is about filling in the gaps based off of a prediction. I think this is a generous interpretation of network-based ML. ML was designed to solve problems. We had lots of data, and we knew large amounts of data could derive functions (networks) as opposed to deliberate construction of algorithms with GOFAI. But "intelligence" with ML as it stands now is not how humans think. Humans do not need millions of examples of cats to know what a cat is. They might need two or three, and they can permanently identify them later. Moreover, they don't need to see all sorts of "representative" cats. A human could see a single instance of black cat and identify all other types of house cats as cats correctly. (And they do: just observe children). Intelligence is the ability to come up with a solution without previous knowledge. The more intelligent an entity is, the less data it needs. As we approach more intelligent systems, they will need less data to be effective, not more.

imoverclocked 2y ago
> Humans do not need millions of examples of cats to know what a cat is. We have evolved over time to recognize things in our environment. We also don’t need to be told that snakes are dangerous as many humans have an innate understanding of that. Our training data is partially inherited.
- abernard1 2y ago
  DNA is around ~725 megabytes. There is no "snakes are dangerous" encoded in there. Our training data is instinctual behaviors we recognize from our sensory inputs. The idea that we're "pre-trained" the way an LLM is (with hundreds of lifetimes of actual sensory experience) is incorrect.
  - imoverclocked 2y ago
    > "The results show that the brain has special neural circuits to detect snakes, and this suggests that the neural circuits to detect snakes have been genetically encoded," Nishijo said. > The monkeys tested in the experiment were reared in a walled colony and neither had previously encountered a real snake. [1] https://www.ucdavis.edu/news/snakes-brain-are-primates-hard-wired-see-snakes#:~:text=%22The%20results%20show%20that%20the,able%20to%20collaborate%20with%20neuroscientists https://www.ucdavis.edu/news/snakes-brain-are-primates-hard-....
    - abernard1 2y ago
      All this is saying is our neural architecture has specific places that respond to danger and the instinct of fear. Anyone who has seen an MRI knows this is the case. It does not mean actual knowledge of snakes is encoded in our DNA. Our brains are "trained" on the data they receive during early development. To the degree that evolutionary pressures stored "data," it stored data about how to make our brains (compute) or physiology more effective. Modern ML tries to shortcut this by making the architecture dumb and the data numerous. If the creators of ML were in charge of a forced evolution, they'd be arguing we need to make DNA 100s of gigabytes and that we needed to store all the memories of our ancestors in it.