4 ms·

Scary possibility: what if there is no good formal theory to explain how it works? What if intelligence, both animal and machine, is purely random trial and err

by mabbo 9y ago

Scary possibility: what if there is no good formal theory to explain how it works? What if intelligence, both animal and machine, is purely random trial and error and "this thing seems to work"? I don't believe that's true necessarily, but it will sure hamper the authors hopes.

azag0 9y ago
In the same way that there is no formal theory for the exact shapes of proteins? I think it’s possible. But as with proteins, there are probably some general aspects of the problem that can be explained in simpler terms.
canes123456 9y ago
> what if there is no good formal theory to explain how it works? We don't need a theory that is perfect. Each theory was partially wrong but still lets you make useful predictions about the world. We need useful models that let you reason about the world. All models are wrong, some are useful. > What if intelligence, both animal and machine, is purely random trial and error and "this thing seems to work"? Evolution could be just considered random trail and error. However until we reach the singularity, we need people to speed up the evolution process by adapting and remixing pieces that worked before. We need models for what each level does so have ideas of what to try for a new application.
- jfoutz 9y ago
  I think the gp is referring to something a little more general. Let's pretend for a moment that our minds can be modeled by a formal system. Every thought has a chain of axioms grounding it. So, the scary part is, Godel showed us there are true things that can't be represented by a formal system. Maybe the useful models exist, but we can't comprehend them, because they're true outside of the set of rules we happened to get built into our minds? generally though, i'm on board with you. all models are wrong, some models are useful.
  - eli_gottlieb 9y ago
    >Let's pretend for a moment that our minds can be modeled by a formal system. Since our minds are more like perception-action variational inference systems, that's a hell of a lot of pretending ;-).
  - canes123456 9y ago
    I am saying there is no connection between the two. You don't need a single complete formal system for a useful model of what is occurring. Godel was saying there can't be one true complete set of axioms for mathematics. This does not mean all mathematical models, theories are suddenly impossible.
  - rocqua 9y ago
    > Godel showed us there are true things that can't be represented by a formal system. This is stretching things a bit. Specifically, it defines truth as 'does not lead to a contradiction in (some formal system that extends) Peano arithmetic'. Then, as there are statements that are 'true' in this sense in such a system A but not probable by that system, there are 'unproveable truths'. But is that satisfactory as a definition of truth? It used to be because we had hope for a complete and consistent formal system, which feel very truthy. When Godel proved that cannot exist, perhaps the conclusion is that formal systems aren't the 'base' for truth.
wirrbel 9y ago
This might be indeed true for deep learning in its current shape. However, in the long run I think we will see more models that (1) are more composable and (2) better approaches to engineering such models, however, these, engineered models will look a bit different than what your research scientist throws over the fence today. At the moment we are in a phase were, to stick to the optics metaphor, we stack up lenses until we see the object on the screen. This means we end up with models that are sprawling, instead of having models that were engineered. Another trend that seems to start in deep learning is that layers become more constrained. I expect, that in 20 years, we will see much more constrained models and much generative models.
- m_ke 9y ago
  1. deep learning models are extremely composable 2. hopefully with time we'll have better approaches to engineer all things that are engineered No, at the moment we go for the biggest and shiniest lens that we can get our hands on and hope that it's capable enough to tackle our problem. If it is we can waste time designing a smaller, more constrained, lens to ship to consumers.
  - taeric 9y ago
    I'm curious if you two are going to be talking past each other with the first point. Any chance I could get you both to explore what you mean by composable?
    - m_ke 9y ago
      These days any method that uses gradient descent to optimize a computational graph gets branded as deep learning. It's a very general paradigm that allows for almost any composition of functions as long as it's differentiable. If that's not composable then I don't know what is. There's a reason why Lecun wanted to rebrand deep learning as differentiable programming. https://www.facebook.com/yann.lecun/posts/10155003011462143 https://www.facebook.com/yann.lecun/posts/10155003011462143 I'm not sure what wirrbel meant.
romaniv 9y ago
Then maybe we should put more resources to study other approaches and higher-level concepts, like statistical learning research done by Vladimir Vapnik.
Maken 9y ago
>What if intelligence, both animal and machine, is purely random trial and error and "this thing seems to work"? Then you apply statistics. Which are the foundations of machine learning.
marzell 9y ago
Extrapolating this concept in various and biased directions... you could say that intelligence, behavior, and evolution (defined as physical iterative behavior across successive reproductive generations) are products of random changes/mutations, reinforced by gradient descent in the form of Survival of the Fittest.
CamperBob2 9y ago
Scary possibility: what if there is no good formal theory to explain how it works? It's scary, but to my thinking, inevitable. It was all well and good for the early atomic scientists to say that "Math is unreasonably effective at explaining Nature," [1] but our level of understanding of both mathematics and natural law is still superficial in several important areas. The universe doesn't owe us a formal theory of anything, much less everything. It seems likely that we will soon start building -- and relying upon -- software that no human actually understands. The math community is already having to confront this dilemma to some extent, when an outlying figure like Mochizuki releases a massive work that takes months for anyone else to understand, much less prove or refute. At some point we will have to give up, and let the machines maintain our models for us. [1] https://www.dartmouth.edu/~matc/MathDrama/reading/Wigner.html https://www.dartmouth.edu/~matc/MathDrama/reading/Wigner.htm...
deleted 9y ago
[deleted]

by mabbo 9y ago

azag0 9y ago

In the same way that there is no formal theory for the exact shapes of proteins? I think it’s possible. But as with proteins, there are probably some general aspects of the problem that can be explained in simpler terms.

canes123456 9y ago

> what if there is no good formal theory to explain how it works? We don't need a theory that is perfect. Each theory was partially wrong but still lets you make useful predictions about the world. We need useful models that let you reason about the world. All models are wrong, some are useful. > What if intelligence, both animal and machine, is purely random trial and error and "this thing seems to work"? Evolution could be just considered random trail and error. However until we reach the singularity, we need people to speed up the evolution process by adapting and remixing pieces that worked before. We need models for what each level does so have ideas of what to try for a new application.

jfoutz 9y ago
I think the gp is referring to something a little more general. Let's pretend for a moment that our minds can be modeled by a formal system. Every thought has a chain of axioms grounding it. So, the scary part is, Godel showed us there are true things that can't be represented by a formal system. Maybe the useful models exist, but we can't comprehend them, because they're true outside of the set of rules we happened to get built into our minds? generally though, i'm on board with you. all models are wrong, some models are useful.
- eli_gottlieb 9y ago
  >Let's pretend for a moment that our minds can be modeled by a formal system. Since our minds are more like perception-action variational inference systems, that's a hell of a lot of pretending ;-).
- canes123456 9y ago
  I am saying there is no connection between the two. You don't need a single complete formal system for a useful model of what is occurring. Godel was saying there can't be one true complete set of axioms for mathematics. This does not mean all mathematical models, theories are suddenly impossible.
- rocqua 9y ago
  > Godel showed us there are true things that can't be represented by a formal system. This is stretching things a bit. Specifically, it defines truth as 'does not lead to a contradiction in (some formal system that extends) Peano arithmetic'. Then, as there are statements that are 'true' in this sense in such a system A but not probable by that system, there are 'unproveable truths'. But is that satisfactory as a definition of truth? It used to be because we had hope for a complete and consistent formal system, which feel very truthy. When Godel proved that cannot exist, perhaps the conclusion is that formal systems aren't the 'base' for truth.

wirrbel 9y ago

This might be indeed true for deep learning in its current shape. However, in the long run I think we will see more models that (1) are more composable and (2) better approaches to engineering such models, however, these, engineered models will look a bit different than what your research scientist throws over the fence today. At the moment we are in a phase were, to stick to the optics metaphor, we stack up lenses until we see the object on the screen. This means we end up with models that are sprawling, instead of having models that were engineered. Another trend that seems to start in deep learning is that layers become more constrained. I expect, that in 20 years, we will see much more constrained models and much generative models.

m_ke 9y ago
1. deep learning models are extremely composable 2. hopefully with time we'll have better approaches to engineer all things that are engineered No, at the moment we go for the biggest and shiniest lens that we can get our hands on and hope that it's capable enough to tackle our problem. If it is we can waste time designing a smaller, more constrained, lens to ship to consumers.
- taeric 9y ago
  I'm curious if you two are going to be talking past each other with the first point. Any chance I could get you both to explore what you mean by composable?
  - m_ke 9y ago
    These days any method that uses gradient descent to optimize a computational graph gets branded as deep learning. It's a very general paradigm that allows for almost any composition of functions as long as it's differentiable. If that's not composable then I don't know what is. There's a reason why Lecun wanted to rebrand deep learning as differentiable programming. https://www.facebook.com/yann.lecun/posts/10155003011462143 https://www.facebook.com/yann.lecun/posts/10155003011462143 I'm not sure what wirrbel meant.

romaniv 9y ago

Then maybe we should put more resources to study other approaches and higher-level concepts, like statistical learning research done by Vladimir Vapnik.

Maken 9y ago

>What if intelligence, both animal and machine, is purely random trial and error and "this thing seems to work"? Then you apply statistics. Which are the foundations of machine learning.

marzell 9y ago

Extrapolating this concept in various and biased directions... you could say that intelligence, behavior, and evolution (defined as physical iterative behavior across successive reproductive generations) are products of random changes/mutations, reinforced by gradient descent in the form of Survival of the Fittest.

CamperBob2 9y ago

Scary possibility: what if there is no good formal theory to explain how it works? It's scary, but to my thinking, inevitable. It was all well and good for the early atomic scientists to say that "Math is unreasonably effective at explaining Nature," [1] but our level of understanding of both mathematics and natural law is still superficial in several important areas. The universe doesn't owe us a formal theory of anything, much less everything. It seems likely that we will soon start building -- and relying upon -- software that no human actually understands. The math community is already having to confront this dilemma to some extent, when an outlying figure like Mochizuki releases a massive work that takes months for anyone else to understand, much less prove or refute. At some point we will have to give up, and let the machines maintain our models for us. [1] https://www.dartmouth.edu/~matc/MathDrama/reading/Wigner.html https://www.dartmouth.edu/~matc/MathDrama/reading/Wigner.htm...

deleted 9y ago

[deleted]