3 ms·

Am I wrong to think that most modern machine learning models are simply about sophisticated pattern recognition and statistical inference? If so, is human inte

by bko 6y ago

Am I wrong to think that most modern machine learning models are simply about sophisticated pattern recognition and statistical inference? If so, is human intelligence doing something similar? Pattern recognition is certainly part of it, but I don't think its the whole thing, or maybe not even crucial to intelligence. The remarkable thing about gpt-3 is its size, 175 billion parameters. At that point theres a lot of room to store a lot of memorized patterns. Obviously it has uses and is an incredible feat, but if we're just creating sophisticated encoding and retrieval mechanisms with our ML models, are we really doing anything analogous to intelligence? Wouldn't this have a limit on functionality? Or is this in a crude sense what's going on in our brains as well?

termy 6y ago
I don't think that you're wrong in thinking that. That is how I view these systems. Artificial neurons and networks of them pale when compared to the complexity of any biological neural network. At a basic level, ANNs are a simplification of biological neurons but I think that we lose something crucial when we perform this simplification. So I think that there is deffinetly more to intelligence than what ML/Deep Learning is doing.
- didericis 6y ago
  Freeman Dyson had an interesting lecture hypothesizing that at least part of why we haven’t been able to make anything like the human mind with our current computers is that our brains are analog. See here: https://m.youtube.com/watch?v=JLT6omWrvIw https://m.youtube.com/watch?v=JLT6omWrvIw
WillDaSilva 6y ago
Our brains have lots of highly specific bits of hardware that corresponds to different parts of the body, or different tasks such as recognizing faces. Most of what our brains do isn't necessary to replicate to produce something which is, at least in some way, generally intelligent. Artificial neural networks are only trying to replicate a small subset of our brain's hardware, and relatively basic (albeit large) architectures like GPT-3 are doing this by becoming skilled at recognizing patterns, and extending them. Pattern recognition is an enormous part of our higher level (generalizable) thinking. It also means that things like our brains and GPT-3 are extremely efficient but lossy compression algorithms.
klmadfejno 6y ago
It's a decent proxy for how our brains process language, but processing language is just one of many processes involved in having a conversation. The goal isn't intelligence. The best possible model would be able to identify the relationships between all words in all contexts. Which is cool, but isn't the whole picture.
- TheOtherHobbes 6y ago
  The best possible model would also be able to identify the relationships between all the people involved in all contexts. The philosophical question is whether there's more to semantics than a certain set of standard paths through a large set of textual relationships. The answer is "Almost certainly", but not necessarily in all contexts - which means that in fact mechanical representations are already good enough for some contexts, although still with a relatively poor hit rate.
X6S1x6Okd1st 6y ago
> Am I wrong to think that most modern machine learning models are simply about sophisticated pattern recognition and statistical inference? You aren't wrong. Modern Machine learning is entirely statistical inference, mostly through minimizing an objective function. > Wouldn't this have a limit on functionality? A neural net can approximate any continuous function (from one euclidean space to another) https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- screye 6y ago
  > neural net can approximate any continuous function Technically a lot of Neural networks are universal function approximators to the limit of infinite data and infinite iterations. It's like saying that with infinite time and motivation, I would be the smartest man on earth. Technically true, but an entirely useless statement.
- Barrin92 6y ago
  >A neural net can approximate any continuous function (from one euclidean space to another) Which doesn't answer the question by the way because Neural Nets suffer the same limitations of Turing Machines, that is to say even a neural net can't solve the halting problem. The fact that something is a universal function approximator should not be mistaken as equivalent to "isn't limited in functionality".
  - X6S1x6Okd1st 6y ago
    > that is to say even a neural net can't solve the halting problem. Can anything?
    - Barrin92 6y ago
      well humans can certainly deal with a lot of halting problems, because humans can make decisions that aren't obviously computational or algorithmic in nature. Which is actually the relevant question that people seem to have skipped over completely, at least from living organisms it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.
5986043handy 6y ago
The Chinese Room (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) would agree it's the same
calebkaiser 6y ago
This is a well-reasoned question, and something I think many people grapple with when they first dig deeper into machine learning. There are a couple things here: 1. Yes, most machine learning approaches are explicitly about sophisticated pattern recognition. To be clear, people working on machine learning have never been unclear about this. On this note, I'd recommend checking out some of the work done by Arthur Samuel, the man who coined the term "machine learning." https://en.wikipedia.org/wiki/Arthur_Samuel https://en.wikipedia.org/wiki/Arthur_Samuel 2. Related to the above, there is a common conflation of the goals of machine learning research and our more sci-fi, AGI ambitions. The goal of a given model, typically, is to make predictions that score well on a given task. Generating passably human text, classifying images, whatever. I don't know of any recently released models whose stated purpose is to recreate complete human intelligence. 3. If we are going to judge machine learning efforts by their ability to recreate the exact mechanisms of human intelligence, we first need a much clearer image of what those mechanisms are.

by bko 6y ago

termy 6y ago

I don't think that you're wrong in thinking that. That is how I view these systems. Artificial neurons and networks of them pale when compared to the complexity of any biological neural network. At a basic level, ANNs are a simplification of biological neurons but I think that we lose something crucial when we perform this simplification. So I think that there is deffinetly more to intelligence than what ML/Deep Learning is doing.

didericis 6y ago
Freeman Dyson had an interesting lecture hypothesizing that at least part of why we haven’t been able to make anything like the human mind with our current computers is that our brains are analog. See here: https://m.youtube.com/watch?v=JLT6omWrvIw https://m.youtube.com/watch?v=JLT6omWrvIw

WillDaSilva 6y ago

Our brains have lots of highly specific bits of hardware that corresponds to different parts of the body, or different tasks such as recognizing faces. Most of what our brains do isn't necessary to replicate to produce something which is, at least in some way, generally intelligent. Artificial neural networks are only trying to replicate a small subset of our brain's hardware, and relatively basic (albeit large) architectures like GPT-3 are doing this by becoming skilled at recognizing patterns, and extending them. Pattern recognition is an enormous part of our higher level (generalizable) thinking. It also means that things like our brains and GPT-3 are extremely efficient but lossy compression algorithms.

klmadfejno 6y ago

It's a decent proxy for how our brains process language, but processing language is just one of many processes involved in having a conversation. The goal isn't intelligence. The best possible model would be able to identify the relationships between all words in all contexts. Which is cool, but isn't the whole picture.

TheOtherHobbes 6y ago
The best possible model would also be able to identify the relationships between all the people involved in all contexts. The philosophical question is whether there's more to semantics than a certain set of standard paths through a large set of textual relationships. The answer is "Almost certainly", but not necessarily in all contexts - which means that in fact mechanical representations are already good enough for some contexts, although still with a relatively poor hit rate.

X6S1x6Okd1st 6y ago

> Am I wrong to think that most modern machine learning models are simply about sophisticated pattern recognition and statistical inference? You aren't wrong. Modern Machine learning is entirely statistical inference, mostly through minimizing an objective function. > Wouldn't this have a limit on functionality? A neural net can approximate any continuous function (from one euclidean space to another) https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...

screye 6y ago
> neural net can approximate any continuous function Technically a lot of Neural networks are universal function approximators to the limit of infinite data and infinite iterations. It's like saying that with infinite time and motivation, I would be the smartest man on earth. Technically true, but an entirely useless statement.
Barrin92 6y ago
>A neural net can approximate any continuous function (from one euclidean space to another) Which doesn't answer the question by the way because Neural Nets suffer the same limitations of Turing Machines, that is to say even a neural net can't solve the halting problem. The fact that something is a universal function approximator should not be mistaken as equivalent to "isn't limited in functionality".
- X6S1x6Okd1st 6y ago
  > that is to say even a neural net can't solve the halting problem. Can anything?
  - Barrin92 6y ago
    well humans can certainly deal with a lot of halting problems, because humans can make decisions that aren't obviously computational or algorithmic in nature. Which is actually the relevant question that people seem to have skipped over completely, at least from living organisms it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.

5986043handy 6y ago

The Chinese Room (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) would agree it's the same

calebkaiser 6y ago

This is a well-reasoned question, and something I think many people grapple with when they first dig deeper into machine learning. There are a couple things here: 1. Yes, most machine learning approaches are explicitly about sophisticated pattern recognition. To be clear, people working on machine learning have never been unclear about this. On this note, I'd recommend checking out some of the work done by Arthur Samuel, the man who coined the term "machine learning." https://en.wikipedia.org/wiki/Arthur_Samuel https://en.wikipedia.org/wiki/Arthur_Samuel 2. Related to the above, there is a common conflation of the goals of machine learning research and our more sci-fi, AGI ambitions. The goal of a given model, typically, is to make predictions that score well on a given task. Generating passably human text, classifying images, whatever. I don't know of any recently released models whose stated purpose is to recreate complete human intelligence. 3. If we are going to judge machine learning efforts by their ability to recreate the exact mechanisms of human intelligence, we first need a much clearer image of what those mechanisms are.