5 ms·

Humans have a huge base model trained over billions of years of evolution. It’s impressive how quickly we learn, but it’s arguably comparable to fine tuning an

Humans have a huge base model trained over billions of years of evolution. It’s impressive how quickly we learn, but it’s arguably comparable to fine tuning an existing model rather than starting from scratch. Conversely, I find it insanely impressive how human-like LLMs have become by doing little more than feeding a vast quantity of reading material into a very basic neural architecture. I wouldn’t have ever believed how successful that approach would be.

chongli 3y ago
The human model isn’t just trained in language though, it’s trained on the world. There’s such a huge amount of tacit knowledge we have about how the world works and how other people work. We have models for recognizing faces and reading body language, emotions, and intent. We can tell when a person is looking at us from across the room and tell that apart from someone looking just over our shoulder at someone else. I think most of what we ascribe to intelligence in LLMs is a kind of textual pareidolia. We infer a mind behind the text because our brains are built to recognize minds. That’s how our theory of mind works. But these LLMs know nothing about the world. They might capture some features of the world as patterns that help them better predict what text to generate for us. But that is not the same as knowledge of the world. They have no sensory contact with the world so they are unable to synthesize true knowledge about it. This is why they are so prone to hallucination.
- jamiek88 3y ago
  > I think most of what we ascribe to intelligence in LLMs is a kind of textual pareidolia This is a great way of expressing a thought I’ve been unable to fully articulate.
- kordlessagain 3y ago
  So we are model ensembles, probably capable of real-time training of short term memory. LLMs are maybe a type of mind. They may carry and operate on basic thoughts, ideas, books, pages, sentences, words, letters…. They can probably problem solve and write code that runs sometimes. What they know about the world was told to them through stories, so whatever way we see it, they will too, maybe. I don’t have to imagine anything in mind to process thoughts. All I need is an attention loop.
  - chongli 3y ago
    What they know about the world was told to them through stories, so whatever way we see it, they will too, maybe. Knowledge is justified true belief. If you park your car on 5th avenue, you know it is parked there because you witnessed yourself parking it there. You have strong justification for your belief that you parked it on 5th avenue. But if you then go to work and your car is out of sight, your justification fades. If you look out the window and see your car parked there, you reaffirm your belief. However, if you look and your car is not there, you realize you don’t know where your car is. For a period of time you were mistaken in your belief about the whereabouts of your car. During the time when your car was out of sight, you went from knowing to not knowing where your car was, but nothing changed inside your mind. To update your belief, you needed new information about the world. An LLM has no justification for any of the information in its training set. It has no experiences with which to connect words and facts to real world objects and events. Therefore it cannot be said to have knowledge of any of it.
    - Davidzheng 3y ago
      I think this is an incredibly strict definition of knowledge that would preclude many experts from many fields--and very different from how it is normally used.
      - chongli 3y ago
        It's not my opinion, it's the standard philosophical account of knowledge [1]. Do you have any examples of "experts" whose basic knowledge would be precluded by this definition? [1] https://plato.stanford.edu/entries/knowledge-analysis/#KnowJustTrueBeli https://plato.stanford.edu/entries/knowledge-analysis/#KnowJ...
- persnickety 3y ago
  Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. But I won't do it. Instead, I'll just replace a few words in your last paragraph to reveal that it doesn't discredit LLMs as much as it seems. * * * But these historians know nothing about the distant past. They might capture some features of the past as patterns that help them better predict where to start an archaeological dig. But that is not the same as knowledge of the past. They have no sensory contact with the past so they are unable to sunthesize true knoledge about it. This is why they are prone to hallucination.
  - chongli 3y ago
    Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. Two things: Justification for belief isn’t a binary state, it’s a spectrum. When we have a small amount of evidence supporting a belief, we have a small justification. When we have a large amount of evidence, we have a large justification. We describe this in terms of our confidence in our beliefs. Without getting too far off into the weeds, this extends into all areas of science. We can’t see, touch, hear, smell, or taste electrons (9V battery tests notwithstanding), but we can gain confidence by replicating experiments with them. Working with a cathode ray tube, for example, we can manipulate the voltage on the plates and directly observe the deflection of the point of light on the screen. We can use the equations from a physics textbook, combined with voltage measurements on a multimeter, to make predictions about the electron beam’s deflection and then test them directly on the device. Secondly (and crucially), the text consumed by LLMs isn’t about the world. It’s text of unknown provenance. Some of it may be about the world, some of it may be complete fiction. The LLM has no basis whatsoever for fact-checking the text. All it can do is perform a statistical test on the commonality of a text and then infer from that. Unfortunately that does not get you any closer to reality. A falsehood repeated many times does not become the truth. As for historians? They have way more than text to go on. They have physical evidence (artifacts), they have radioisotope testing, they have DNA testing. They have geographic locations where texts were found. None of this real world sense data is available to an LLM.
- famouswaffles 3y ago
  >The human model isn’t just trained in language though, it’s trained on the world. It's not trained on "the world". It's trained on a small slice of it, a few senses that are themselves slimmed down and then fabricated at parts. To the bird that can feel and sense electromagnetic waves intuitively to guide travels, you're not trained on "the world" >This is why they are so prone to hallucination. No it's not. LLMs can distinguish between truth and hallucination. They just don't care to communicate that. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 https://arxiv.org/abs/2310.06824
SamFold 3y ago
I think it's somewhat of a stretch to say our "base model" had billions of years of evolution. Billions of years ago, mammals didn't exist and the only things around were more like plankton or algae and had nothing like a "base model" we could say we somehow inherited. The earliest ape-like creatures appeared around 10M years ago. The first mammals appeared around 225M years ago so you could potentially argue that our "base model" first started evolving around then but I still think it's something of a stretch to compare this kind "training" to the ways we are training modern neural networks. The "base model" at this time was simply survival, eat, reproduce, survive, and enough circuitry to manage your base biological functions. We are essentially running the entire volume of human knowledge through a neural network through billions of iterations and the model itself has 175 billion plus parameters. Humans nor any of our evolutionary ancestors never received this kind of "training", it's simply not comparable at all. Our mammalian ancestors were exposed to "basic" natural environments, they were not "pushed" into artifical situations to learn tool usage or language. If we look at when apes first came about (10M years) ago and let's say since then the average ape or humanoid lived to 30-40 years, and estimate the average generation length for apes at 20 years (which is roughly accurate according to the latest research). This means that since the first recorded apes there have been about 500'000 generations of apes and humans. (12'000 generations for humans only). So now if you compare how we are training our models, GPT-3 at 175B parameters and billions of iterations of training, GPT-4 we don't know. And again, extremely focused and specific training, feeding the entire human generated corpus of language, mathemetics, logic, etc etc into it, and we get something that does pretty well at human language. Humans have a "base model" as you put it which really hasn't been trained for many generations and has been mostly exposed at random to external stimuli in an ad-hoc, unfocused way, and no single individual has ever been exposed to even a fraction of a fraction of a percent as much stimuli as a GPT model. So there is something different going on with our brain and neural networks and I think it can't really be compared at all: the mechanisms, numbers, and crucially, the results, do not match up in the slightest.
- cameronh90 3y ago
  I was being slightly facetious in parts. The point I was driving at is that a human mind has had a very long time to do something perhaps akin to hyperparameter tuning, and we know that even imperceptibly minor changes in brain architecture can be the difference between struggling to put your socks on and being a genius that’ll be remembered through the ages. So those 500M years since the first neuron can’t just be written off. Ultimately I agree with your final conclusion. You can’t really compare a LLM and evolved human directly. Even just a neuron in an ANN is nothing remotely comparable to a biological neuron. Of course it isn’t surprising that humans and LLMs are different given that they are built to do completely different things on fundamentally different hardware. It just seems like many people are keen to write off the significance of GPT just because it’s not yet quite as good at everything compared to the world’s most marvellous example of engineering we all have in our skull. We didn’t even have transistors 75 years ago, but now we have a pretty believable facsimile (until you really interrogate it) of human intelligence that’s improving million times faster than evolution was ever capable of. But now the criticism is that it learns in a fundamentally different way to humans and doesn’t generalise fast enough. It’s true, but.. really?

chongli 3y ago

The human model isn’t just trained in language though, it’s trained on the world. There’s such a huge amount of tacit knowledge we have about how the world works and how other people work. We have models for recognizing faces and reading body language, emotions, and intent. We can tell when a person is looking at us from across the room and tell that apart from someone looking just over our shoulder at someone else. I think most of what we ascribe to intelligence in LLMs is a kind of textual pareidolia. We infer a mind behind the text because our brains are built to recognize minds. That’s how our theory of mind works. But these LLMs know nothing about the world. They might capture some features of the world as patterns that help them better predict what text to generate for us. But that is not the same as knowledge of the world. They have no sensory contact with the world so they are unable to synthesize true knowledge about it. This is why they are so prone to hallucination.

jamiek88 3y ago
> I think most of what we ascribe to intelligence in LLMs is a kind of textual pareidolia This is a great way of expressing a thought I’ve been unable to fully articulate.
kordlessagain 3y ago
So we are model ensembles, probably capable of real-time training of short term memory. LLMs are maybe a type of mind. They may carry and operate on basic thoughts, ideas, books, pages, sentences, words, letters…. They can probably problem solve and write code that runs sometimes. What they know about the world was told to them through stories, so whatever way we see it, they will too, maybe. I don’t have to imagine anything in mind to process thoughts. All I need is an attention loop.
- chongli 3y ago
  What they know about the world was told to them through stories, so whatever way we see it, they will too, maybe. Knowledge is justified true belief. If you park your car on 5th avenue, you know it is parked there because you witnessed yourself parking it there. You have strong justification for your belief that you parked it on 5th avenue. But if you then go to work and your car is out of sight, your justification fades. If you look out the window and see your car parked there, you reaffirm your belief. However, if you look and your car is not there, you realize you don’t know where your car is. For a period of time you were mistaken in your belief about the whereabouts of your car. During the time when your car was out of sight, you went from knowing to not knowing where your car was, but nothing changed inside your mind. To update your belief, you needed new information about the world. An LLM has no justification for any of the information in its training set. It has no experiences with which to connect words and facts to real world objects and events. Therefore it cannot be said to have knowledge of any of it.
  - Davidzheng 3y ago
    I think this is an incredibly strict definition of knowledge that would preclude many experts from many fields--and very different from how it is normally used.
    - chongli 3y ago
      It's not my opinion, it's the standard philosophical account of knowledge [1]. Do you have any examples of "experts" whose basic knowledge would be precluded by this definition? [1] https://plato.stanford.edu/entries/knowledge-analysis/#KnowJustTrueBeli https://plato.stanford.edu/entries/knowledge-analysis/#KnowJ...
persnickety 3y ago
Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. But I won't do it. Instead, I'll just replace a few words in your last paragraph to reveal that it doesn't discredit LLMs as much as it seems. * * * But these historians know nothing about the distant past. They might capture some features of the past as patterns that help them better predict where to start an archaeological dig. But that is not the same as knowledge of the past. They have no sensory contact with the past so they are unable to sunthesize true knoledge about it. This is why they are prone to hallucination.
- chongli 3y ago
  Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. Two things: Justification for belief isn’t a binary state, it’s a spectrum. When we have a small amount of evidence supporting a belief, we have a small justification. When we have a large amount of evidence, we have a large justification. We describe this in terms of our confidence in our beliefs. Without getting too far off into the weeds, this extends into all areas of science. We can’t see, touch, hear, smell, or taste electrons (9V battery tests notwithstanding), but we can gain confidence by replicating experiments with them. Working with a cathode ray tube, for example, we can manipulate the voltage on the plates and directly observe the deflection of the point of light on the screen. We can use the equations from a physics textbook, combined with voltage measurements on a multimeter, to make predictions about the electron beam’s deflection and then test them directly on the device. Secondly (and crucially), the text consumed by LLMs isn’t about the world. It’s text of unknown provenance. Some of it may be about the world, some of it may be complete fiction. The LLM has no basis whatsoever for fact-checking the text. All it can do is perform a statistical test on the commonality of a text and then infer from that. Unfortunately that does not get you any closer to reality. A falsehood repeated many times does not become the truth. As for historians? They have way more than text to go on. They have physical evidence (artifacts), they have radioisotope testing, they have DNA testing. They have geographic locations where texts were found. None of this real world sense data is available to an LLM.
famouswaffles 3y ago
>The human model isn’t just trained in language though, it’s trained on the world. It's not trained on "the world". It's trained on a small slice of it, a few senses that are themselves slimmed down and then fabricated at parts. To the bird that can feel and sense electromagnetic waves intuitively to guide travels, you're not trained on "the world" >This is why they are so prone to hallucination. No it's not. LLMs can distinguish between truth and hallucination. They just don't care to communicate that. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 https://arxiv.org/abs/2310.06824

SamFold 3y ago

I think it's somewhat of a stretch to say our "base model" had billions of years of evolution. Billions of years ago, mammals didn't exist and the only things around were more like plankton or algae and had nothing like a "base model" we could say we somehow inherited. The earliest ape-like creatures appeared around 10M years ago. The first mammals appeared around 225M years ago so you could potentially argue that our "base model" first started evolving around then but I still think it's something of a stretch to compare this kind "training" to the ways we are training modern neural networks. The "base model" at this time was simply survival, eat, reproduce, survive, and enough circuitry to manage your base biological functions. We are essentially running the entire volume of human knowledge through a neural network through billions of iterations and the model itself has 175 billion plus parameters. Humans nor any of our evolutionary ancestors never received this kind of "training", it's simply not comparable at all. Our mammalian ancestors were exposed to "basic" natural environments, they were not "pushed" into artifical situations to learn tool usage or language. If we look at when apes first came about (10M years) ago and let's say since then the average ape or humanoid lived to 30-40 years, and estimate the average generation length for apes at 20 years (which is roughly accurate according to the latest research). This means that since the first recorded apes there have been about 500'000 generations of apes and humans. (12'000 generations for humans only). So now if you compare how we are training our models, GPT-3 at 175B parameters and billions of iterations of training, GPT-4 we don't know. And again, extremely focused and specific training, feeding the entire human generated corpus of language, mathemetics, logic, etc etc into it, and we get something that does pretty well at human language. Humans have a "base model" as you put it which really hasn't been trained for many generations and has been mostly exposed at random to external stimuli in an ad-hoc, unfocused way, and no single individual has ever been exposed to even a fraction of a fraction of a percent as much stimuli as a GPT model. So there is something different going on with our brain and neural networks and I think it can't really be compared at all: the mechanisms, numbers, and crucially, the results, do not match up in the slightest.

cameronh90 3y ago
I was being slightly facetious in parts. The point I was driving at is that a human mind has had a very long time to do something perhaps akin to hyperparameter tuning, and we know that even imperceptibly minor changes in brain architecture can be the difference between struggling to put your socks on and being a genius that’ll be remembered through the ages. So those 500M years since the first neuron can’t just be written off. Ultimately I agree with your final conclusion. You can’t really compare a LLM and evolved human directly. Even just a neuron in an ANN is nothing remotely comparable to a biological neuron. Of course it isn’t surprising that humans and LLMs are different given that they are built to do completely different things on fundamentally different hardware. It just seems like many people are keen to write off the significance of GPT just because it’s not yet quite as good at everything compared to the world’s most marvellous example of engineering we all have in our skull. We didn’t even have transistors 75 years ago, but now we have a pretty believable facsimile (until you really interrogate it) of human intelligence that’s improving million times faster than evolution was ever capable of. But now the criticism is that it learns in a fundamentally different way to humans and doesn’t generalise fast enough. It’s true, but.. really?