3 ms·

> Am I wrong to think that most modern machine learning models are simply about sophisticated pattern recognition and statistical inference? You aren't wrong.

> Am I wrong to think that most modern machine learning models are simply about sophisticated pattern recognition and statistical inference? You aren't wrong. Modern Machine learning is entirely statistical inference, mostly through minimizing an objective function. > Wouldn't this have a limit on functionality? A neural net can approximate any continuous function (from one euclidean space to another) https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...

screye 6y ago
> neural net can approximate any continuous function Technically a lot of Neural networks are universal function approximators to the limit of infinite data and infinite iterations. It's like saying that with infinite time and motivation, I would be the smartest man on earth. Technically true, but an entirely useless statement.
Barrin92 6y ago
>A neural net can approximate any continuous function (from one euclidean space to another) Which doesn't answer the question by the way because Neural Nets suffer the same limitations of Turing Machines, that is to say even a neural net can't solve the halting problem. The fact that something is a universal function approximator should not be mistaken as equivalent to "isn't limited in functionality".
- X6S1x6Okd1st 6y ago
  > that is to say even a neural net can't solve the halting problem. Can anything?
  - Barrin92 6y ago
    well humans can certainly deal with a lot of halting problems, because humans can make decisions that aren't obviously computational or algorithmic in nature. Which is actually the relevant question that people seem to have skipped over completely, at least from living organisms it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.
    - X6S1x6Okd1st 6y ago
      > well humans can certainly deal with a lot of halting problems Could you give an example of a halting problem that humans can solve? > it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are. Do you have supporting evidence?
baron_harkonnen 6y ago
> Modern Machine learning is entirely statistical inference In virtually no cases that I can think of does modern machine learning approach statistical inference. Statistical inference is always concerned with estimating confidence in beliefs and typically more concerned with the confidence in the model parameters rather than the model predictions. Machine learning is almost exclusively concern with model predictions and the performance of these predictions. You can easily have a model that is very useful for statistical inference but would be awful for machine learning, and most machine learning models (especially neural networks) as useless for statistical inference but great at prediction. This is not a critique of machine learning or vice versa, its just the case that these two approaches to modeling are quite different and used to solve very different problems.
amirkdv 6y ago
> A neural net can approximate any continuous function. Can't a RF or boosted regression tree theoretically do the same and then some (approximate discontinuous functions)?

screye 6y ago

> neural net can approximate any continuous function Technically a lot of Neural networks are universal function approximators to the limit of infinite data and infinite iterations. It's like saying that with infinite time and motivation, I would be the smartest man on earth. Technically true, but an entirely useless statement.

Barrin92 6y ago

>A neural net can approximate any continuous function (from one euclidean space to another) Which doesn't answer the question by the way because Neural Nets suffer the same limitations of Turing Machines, that is to say even a neural net can't solve the halting problem. The fact that something is a universal function approximator should not be mistaken as equivalent to "isn't limited in functionality".

X6S1x6Okd1st 6y ago
> that is to say even a neural net can't solve the halting problem. Can anything?
- Barrin92 6y ago
  well humans can certainly deal with a lot of halting problems, because humans can make decisions that aren't obviously computational or algorithmic in nature. Which is actually the relevant question that people seem to have skipped over completely, at least from living organisms it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.
  - X6S1x6Okd1st 6y ago
    > well humans can certainly deal with a lot of halting problems Could you give an example of a halting problem that humans can solve? > it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are. Do you have supporting evidence?

baron_harkonnen 6y ago

> Modern Machine learning is entirely statistical inference In virtually no cases that I can think of does modern machine learning approach statistical inference. Statistical inference is always concerned with estimating confidence in beliefs and typically more concerned with the confidence in the model parameters rather than the model predictions. Machine learning is almost exclusively concern with model predictions and the performance of these predictions. You can easily have a model that is very useful for statistical inference but would be awful for machine learning, and most machine learning models (especially neural networks) as useless for statistical inference but great at prediction. This is not a critique of machine learning or vice versa, its just the case that these two approaches to modeling are quite different and used to solve very different problems.

amirkdv 6y ago

> A neural net can approximate any continuous function. Can't a RF or boosted regression tree theoretically do the same and then some (approximate discontinuous functions)?