4 ms·

That's a great way of describing it, and I think a very necessary and important thing to communicate at this time. A lot of people in this yhread are saying tha

That's a great way of describing it, and I think a very necessary and important thing to communicate at this time. A lot of people in this yhread are saying that it's all "just" statistics, but "mere" statistics can give enough info to support inferences to a stable underlying world, and the reasoning about the world shows up in sophisticated associations made by the models.

sirsinsalot 4y ago
I mean, my brain, and physics is all just statistics and approximate side effects (and models thereof)
- blindhippo 4y ago
  Hah I was going to say - isn't quantum physics in many ways the intersection of statistics/probabilities and reality?
simonh 4y ago
It’s clear they do seem to construct models from which to derive responses. The problem is once you stray away from purely textual content, those models often get completely batshit. For example if you ask it what latitude and longitude are, and what makes a town further north than another, it will tell you. But if you ask it if this town is further north than this other town, it will give you latitudes that are sometimes correct, sometimes made up, and will randomly get which one is further north wrong, even based on the latitudes it gave. That’s because it doesn’t have an actual understanding of the geography of the globe, because the training texts werent sufficient to give it that. It can explain latitude, but doesn’t actually know how to reason about it, even though it can explain how to reason about it. That’s because explaining something and doing it are completely different kinds of tasks. If it does this with the globe and simple stuff like latitudes, what are the chances it will mess up basic relationships between organs, symptoms, treatments, etc for the human body? Im not going to trust medical advice from these things without an awful lot of very strong evidence.
- kaibee 4y ago
  I don't think this is a particular interesting criticism. The fact of the matter is that this just solved by chain-of-though reasoning. If you need the model to be "correct", you can make it get there by first writing out the two different latitudes, and then it will get it right. This is basically the same way that people can/will guesstimate at something vs doing the actual math. For a medical AI, you'll definitely need it to chain-of-thought every inference and step/conclusion on the path but...
  - simonh 4y ago
    >you can make it get there by first writing out the two different latitudes, and then it will get it right As I said in my comment, even if the model 'knows' and tells you that town A is at 64' North latitude and town B is at 53', it will sometimes tell you town B is the furthest north. That's because it's training set includes texts where people talk about one town being further north that the other, and their latitudes, but the neural net wasn't able to infer the significance of the numbers in the latitude values. There wasn't enough correlation in the text for it to infer their significance, or generate a model for accurately doing calculations on them. Meanwhile the training text must have contained many explanations of what latitude and longitude are and how to do calculations on them. As a result the model can splurge out texts explaining latitude and longitude. That only helps it splurge out that kind of text though. It doesn't do anything towards actually teaching it what these concepts are, how they relate to a spherical geographic model, or to actually do the calculations. It's the same way GPT-3 could reliably generate texts explaining mathematics and how to do arithmetic in lots of very accurate detail, because it was trained on many texts that gave such explanations, but couldn't actually do maths. It is possible to overcome these issues with a huge amount of domain relevant training text to help the LLM build a model of the specific problem domain. So these problems can be overcome. But the point stands that just because a model can explain in detail how to do something, that doesn't mean it can actually do it itself at all. They're completely different things that require radically different training approaches.
    - MVissers 4y ago
xp84 4y ago
^ Agree. I'm convinced my 2-year-old doesn't operate on a dramatically different strategy than a LLM -- she's learned that when you are negotiating something (continued access to browse pictures on parent's phone, getting to watch TV, staying longer at a place she likes, etc), you can add on "2 minutes?" to your request and sometimes the opposing negotiator will give you some more time. She doesn't know what exactly a minute is or that specific number, but she's observed that it's correlated with getting what you want more than say, a whine. This is simple statistics and probability, in a biological neural network. I think it's really cute how defensive and dismissive humans get (including those who profess zero supernatural beliefs) when they're trying so valiantly to write off all AI as a cheap parlor trick.
- gerad 4y ago
  All that said, the fact that AI is catching up to 2 year olds is pretty impressive. Human's brains surpass dog's at about that age. It shows we're getting close to the realm of "human."
  - chromanoid 4y ago
    I think finding an analogy with two year olds tells more about those who spout it than about where we are getting close to...
  - taneq 4y ago
    Given how many university-level tests GPT4 places better than 50th percentile at, I don't know if "catching up to 2 year olds" is a fair description. For that kind of text based task it seems well ahead of the general adult human population.
    - Sharlin 4y ago
      To be fair, such tests are designed with the human mind in, well, mind, and assume that various hard-to-quantify variables – ones that the tester is actually interested in – correlate with test performance. But LLMs are alien minds with very different correlations. It’s clear, of course, that ChatGPT’s language skills vastly exceed those of an average 2-year-old, and indeed surpass the skills of a considerable fraction of general adult population, but the generality of its intelligence is probably not above a human toddler.

sirsinsalot 4y ago

I mean, my brain, and physics is all just statistics and approximate side effects (and models thereof)

blindhippo 4y ago
Hah I was going to say - isn't quantum physics in many ways the intersection of statistics/probabilities and reality?

simonh 4y ago

It’s clear they do seem to construct models from which to derive responses. The problem is once you stray away from purely textual content, those models often get completely batshit. For example if you ask it what latitude and longitude are, and what makes a town further north than another, it will tell you. But if you ask it if this town is further north than this other town, it will give you latitudes that are sometimes correct, sometimes made up, and will randomly get which one is further north wrong, even based on the latitudes it gave. That’s because it doesn’t have an actual understanding of the geography of the globe, because the training texts werent sufficient to give it that. It can explain latitude, but doesn’t actually know how to reason about it, even though it can explain how to reason about it. That’s because explaining something and doing it are completely different kinds of tasks. If it does this with the globe and simple stuff like latitudes, what are the chances it will mess up basic relationships between organs, symptoms, treatments, etc for the human body? Im not going to trust medical advice from these things without an awful lot of very strong evidence.

kaibee 4y ago
I don't think this is a particular interesting criticism. The fact of the matter is that this just solved by chain-of-though reasoning. If you need the model to be "correct", you can make it get there by first writing out the two different latitudes, and then it will get it right. This is basically the same way that people can/will guesstimate at something vs doing the actual math. For a medical AI, you'll definitely need it to chain-of-thought every inference and step/conclusion on the path but...
- simonh 4y ago
  >you can make it get there by first writing out the two different latitudes, and then it will get it right As I said in my comment, even if the model 'knows' and tells you that town A is at 64' North latitude and town B is at 53', it will sometimes tell you town B is the furthest north. That's because it's training set includes texts where people talk about one town being further north that the other, and their latitudes, but the neural net wasn't able to infer the significance of the numbers in the latitude values. There wasn't enough correlation in the text for it to infer their significance, or generate a model for accurately doing calculations on them. Meanwhile the training text must have contained many explanations of what latitude and longitude are and how to do calculations on them. As a result the model can splurge out texts explaining latitude and longitude. That only helps it splurge out that kind of text though. It doesn't do anything towards actually teaching it what these concepts are, how they relate to a spherical geographic model, or to actually do the calculations. It's the same way GPT-3 could reliably generate texts explaining mathematics and how to do arithmetic in lots of very accurate detail, because it was trained on many texts that gave such explanations, but couldn't actually do maths. It is possible to overcome these issues with a huge amount of domain relevant training text to help the LLM build a model of the specific problem domain. So these problems can be overcome. But the point stands that just because a model can explain in detail how to do something, that doesn't mean it can actually do it itself at all. They're completely different things that require radically different training approaches.
  - MVissers 4y ago

xp84 4y ago

^ Agree. I'm convinced my 2-year-old doesn't operate on a dramatically different strategy than a LLM -- she's learned that when you are negotiating something (continued access to browse pictures on parent's phone, getting to watch TV, staying longer at a place she likes, etc), you can add on "2 minutes?" to your request and sometimes the opposing negotiator will give you some more time. She doesn't know what exactly a minute is or that specific number, but she's observed that it's correlated with getting what you want more than say, a whine. This is simple statistics and probability, in a biological neural network. I think it's really cute how defensive and dismissive humans get (including those who profess zero supernatural beliefs) when they're trying so valiantly to write off all AI as a cheap parlor trick.

gerad 4y ago
All that said, the fact that AI is catching up to 2 year olds is pretty impressive. Human's brains surpass dog's at about that age. It shows we're getting close to the realm of "human."
- chromanoid 4y ago
  I think finding an analogy with two year olds tells more about those who spout it than about where we are getting close to...
- taneq 4y ago
  Given how many university-level tests GPT4 places better than 50th percentile at, I don't know if "catching up to 2 year olds" is a fair description. For that kind of text based task it seems well ahead of the general adult human population.
  - Sharlin 4y ago
    To be fair, such tests are designed with the human mind in, well, mind, and assume that various hard-to-quantify variables – ones that the tester is actually interested in – correlate with test performance. But LLMs are alien minds with very different correlations. It’s clear, of course, that ChatGPT’s language skills vastly exceed those of an average 2-year-old, and indeed surpass the skills of a considerable fraction of general adult population, but the generality of its intelligence is probably not above a human toddler.