7 ms·

There are many caveats to this, esp. that this "fact" has nothing to do with whether training a neural network on a dataset will be useful. There is often no f

There are many caveats to this, esp. that this "fact" has nothing to do with whether training a neural network on a dataset will be useful. There is often no function to find in solving a problem, ie., there is no mapping from ImageSpace -> DogCatSpace. Ie., most things are genuine ambiguitites --- a stick in a water appears bent, indistinguishably from an actually bent stick in some other transparent fluid. Animals solve the problem of the "ambiguity of inference" by being in the world and being able to experiment. Ie., taking the stick out of the water. A neural network, in this sense, cannot "take the stick out the water" -- it cannot resolve ambiguities. So that it can "approximate functions" is neither necessary nor sufficient for a useful learning system. More significantly, a NN is a very very bad approximator of many functions. Consider approximating a trajectory so that one can then find an acceleration, ie., here we need f(x) to find d2f/dx2 -- NN approximations are typically "OK" at the f(x) level and really really crappy at the df/dx level, because the non-linear functions NNs just glue together are only trainable if theyre very rough. For these, and lots of other reasons, this theoretical approach to learning is largely marketing none-sense. If you go out and actually study the only known systems which learn effectively (ie., animals), one does not find the need for universal approximator theorems in explaining their capacities. These theorems only show that NNs are, like many computational statistical techniques, sufficient for a wide class of "mere approximation" problems that are only narrowly useful within the whole field of learning-as-such.

benrbray 5y ago
What techniques are used to estimate trajectories? Would something like a Gaussian Process perform better?
- mjburgess 5y ago
  Well NNs like GPs are "basically" non-parametric methods, in the sense that one does not start with a known parameterised statistical distribution that comes from domain expertise. These are worst-case techniques when we dont have the option to start with "the right answer", eg., in the case of large datasets where we have no idea how some pixels distribute over cat/dog images. In the case of a trajectory we would likey already know the answer, in the form of just doing some physics. The role of computational stats here then is to start with the known form of the solution and find the specific parameters to fit it. Since we have physics, we can find the "perfect answer" to the trajectory question with very few data points -- and take as many derivatives as we like. Brute-force ML is often used when we dont have theories, making it all the more dangerous; and alas, all the more useful. We can get a 5% improvement on click-thru rate without having any theory of human behavioural psychology --- god knows then, what we are doing to human behaviour when we implement this system. `
- aqme28 5y ago
  If you’re asking how to solve difficult differential equations in general, we use numerical methods like finite elements or finite differences
  - adgjlsfhk1 5y ago
    that said, there has been some promising research on using NNs for solving nonlinear pdes.
curiousgal 5y ago
> There is often no function to find in solving a problem In finance NN can be used to calibrate models, i.e. generate parameters to a model (function) that replicates existing data (obsered prices/volatilities).
- mjburgess 5y ago
  Well i dont think prices are functions of economic variables. Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. So what you're modelling is something like `E_hope[y] = f(x)` where `E_hope` is "a hopeful expectation" that the mean of the underlying ambiguous y1,..yn does "reliably" mean to a unique `x`. This "hopeful expectation" is certainly more common than there being any actual function connecting `y` to `x`, but i think its often quite false too. Ie., even the expectation of prices is genuinely ambiguous. To handle this we might ensemble models `E_ensemble[E1_hope[y], ...En_hope[y]]`, but to repeat a famous idiom in finance, this is very much "building sandcastles in the sky". The idea that you can just "expect" (/statstics) your way out of the need for experimentation is a dangerous superstition which is at the heart of ML. It is impossible to simply "model data", measurement produces genuine ambiguities which can only be resolved by changing the world and seeing-what-happens. There is no function to find.
  - thfuran 5y ago
    >Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. y1,...,yn is a perfectly reasonable function output. Functions don't have to produce scalars.
    - mjburgess 5y ago
      Functions have to resolve to one point in the output domain, even if that point is multi-dim. Here, consider `y_houseprice = price(house data, economic data, etc.)`. There isnt a unique house price in terms of those variables. The real world observes many such prices for the same value of those variables. An overly mathematical view of the world has obscured the scientific method from our thinking here. Generally, there arent actually functions from X to Y, and there arent actually stable XY distributions over time. The world, as measured, is basically always ambigouous and discontinuous. Data, as measurement, isnt the foundation of our theory-building. We build theories by changing the world; data comes in as a guiding light to our theory building, not as the basis. ..wwhich is our direct causal interactions with our environment, ie., its the actual stuff of our bodies and the stuff of the world as we change it
shawnz 5y ago
> If you go out and actually study the only known systems which learn effectively (ie., animals), one does not find the need for universal approximator theorems in explaining their capacities. What's the explanation for their capacities, then?
- mjburgess 5y ago
  direct, theory-laden, causal contact and pro-active engagement with their environments... eg., taking the stick out of the water. "data" (even from an animal's pov) is basically, by nature, ambiguous. Measurement is an event in the world (eg., light hitting the eye) which isn't somehow unambiguously informative. To over come this problem, basically, animals move stuff. The adaptable motor cortex of the most intelligent animals, therefore, isnt something to be tacked-on to "intelligence", it's the precondition for it. Glibly: tools before thoughts. Reason needs content to operate on, the content of our thoughts is built via our (sensory-)motor systems. The idea that we need ever more theoretically powerful models of reasoning here is a misdirection -- it misses that the heart of everything we know arrives via some form of repeated experiment. Every more automated "pure reasoning" either via statistics or symbolically, is always just a means of juicing the data we provide the system. Useful as a technology, but not as a genuine learning system. It will never have the means of resolving the many ambiguities within data itself. In the case, for example, of NLP -- the structure of 1 trillion documents will never enable a machine to answer the question "what do you like about what i'm wearing?" -- because (1) the machine isnt here with me; (2) i am asking for its personal judgement, not a summary of a trillion documents; and (3) that summary of those trillion documents has to be unique, but the question has no "right answer". Whilst computer science has the driving seat over what "intelligence" is, we will forever be stuck with this incredibly diminished view of our own capacities and the size of the technological challenge. The goal isnt to sift through everything we have already done and "take a mean", the goal is to produce a system which could have done "everything we have done" without us.
  - gowld 5y ago
    Experiments are data collection. It's the iinput to the thinking process.
version_five 5y ago
> There is often no function to find in solving a problem, ie., there is no mapping from ImageSpace -> DogCatSpace. Ie., most things are genuine ambiguitites --- a stick in a water appears bent, indistinguishably from an actually bent stick in some other transparent fluid. That's tangential imo. There is a function that maps from image space to dog/cat/don't know space. A sentient being that get the "don't know" can get more info to resolve the ambiguity (or rephrase the question). A universal function approximator can still make itself useful even if all it can do it say it doesn't know. This is a question of problem setup. NNs are bad at extrapolating, including e.g. trivially to periodic functions. This is a limitation if you thought they could do that, but again a question of understanding what they do. Hype, as you say, leads some people to believe NNs are magic, leading to mismatched expectation. A universal interpolate-only function approximator is still pretty useful though, just maybe disappointing if you understood that to imply sentience
- mjburgess 5y ago
  Sure, we could see `animal = f(image)` as a partial function and lift it into some total function space, `maybe_animal = f(image)`. Is our reasoning here actually this total function though? We really do always need to keep in mind that animal reasoning will terminate, animals will act "guided by other reasoning", and resume the original reasoning process. Action really messes up this neat "totalising option" for partial functions. If i'm not sure if "fluffy" is a dog or a cat, i might wait longer for it to move; I might throw something at it; I might ask its owner. This isnt as simple as "don't know", since reasoning is kinda time-bound and time-parameterised, my very measuring process is sensitive to my own confidence in what-something-is. Is this really "f(image) = dont-know"? I dont think so. I think it's more like, "judgement = body-brain-state(t, reasoning-goal, {action policies}, ...large-number-of-other-things)". This gets to my issue. In my view what animals are doing is better described by a dynamical equation of state (like a wavefunction), as the whole system is operating under its own dynamical evolution (including, eg., what fluffy does in response to your puzzlement). I dont see Dog|Cat|DontKnow as the answer here. I dont think it's the actual total function which corresponds to our judgement, though it probably is total -- we just end up with "DontKnow" in all the cases actual intellkigence is requireds.... the very think we're aiming to model.
Der_Einzige 5y ago
Interpolation in GANs seems a lot like "being able to experiment. Ie., taking the stick out of the water"...
- tsimionescu 5y ago
  They are not taking the stick out of the water, because they don't have hands and are not looking at a real stick. They are being trained on a static data set, and a picture of a stick isn't a stick. They can try to extrapolate all they want, but they are fundamentally not going to be able to get more information out of the data than there exists. And in a static photo of a bent stick, or a fluffy critter, there simply isn't any information to tell whether this is a bent stick or a stick in water; or whether it's a cat or a dog. The intelligent response is not "I don't know", and it's not "60% it's a cat, 40% it's a dog". It's "here is the set of actions that need to be taken to create more data to be able to settle the question". And creating that set of actions is completely different from current approaches. No state of the art GAN can say "you need to view the subject from a steeper angle, check for a reflection to see if it's in water" or "poke it with a stick, see if it meows or barks", because they don't have enough information about the world in their train g sets to be able to even know that these are possibilities.
  - YeGoblynQueenne 5y ago
    They simultaneously have not enough information, and too much information. A human child can learn what a chicken is by seeing a chicken once, but "seeing a chicken" is not the same as the representation of a chicken as a set of pixels in an image. It's ... something else, that we don't quite have the words to describe. Clearly, when we look at things in the real world we take in much more information than when a CNN takes in an image. Machine learning datasets have too much data, but it's the wrong kind of data and they're the wrong kind of learning system, to learn what humans can learn from "single examples". And humans can also learn about radically new categories of things that they have never experienced just by looking at an image, even a low-quality image. And that's humans which are like the apex of intelligence on the planet (I know, right?). Neural nets can't even built the representation of the world that a simpler animal, like a cockroach, must have to be able to exist autonomously in its environment. We don't have anything like a machine that can get close to the ability of a cockroach for autonomy. It's mad. But people keep thinking, "hey, we'll just use more data and more layers" and get to the moon by jumping higher with a larger pogo stick.
slibhb 5y ago
> Animals solve the problem of the "ambiguity of inference" by being in the world and being able to experiment. Ie., taking the stick out of the water. A neural network, in this sense, cannot "take the stick out the water" -- it cannot resolve ambiguities. Couldn't training data include a video of someone taking a stick out of the water?
- ddingus 5y ago
  Then we have no ambiguity. The data is inclusive. And the cost is a very seriously larger problem space to classify.
- YeGoblynQueenne 5y ago
  In that case, a neural net will learn to predict the next frame in a video stream, but it still won't learn what a stick is or why it looks bent when it's in the water.
sdenton4 5y ago
You've got a kind of narrow view of the matter... The need for interactive understanding is addressed in different ways by reinforcement learning, GANs, and autoregressive recurrent neutral networks. In the latter cases, the generative output of the network is a kind of experiment, and back prop from the loss function provides a route to improvement. I think it's an unfortunate historical accident that the field of machine learning is so transfixed with classifiers. But they're really not the only game in town.
fxtentacle 5y ago
The article also conveniently glosses over the fact that all AI calculations are limited in their maximum complexity by the depth of the AI. For example, for x! the best a regular DL AI can do is to memorize some values and interpolate between them.
tiborsaas 5y ago
Did you really compare a single neural network with a real physical agent (dog) with 5 senses, trillions of cells, a complex inter-connected brain operating in the real world? This is close to what I'd call unfair, or straw-man argument. This is like calling calculus useless, because it doesn't help you pick one kind of milk versus another in a supermarket. Universal function approximation is a great tool if used correctly. From what I understand, it comes in handy when our puny human brain can't come up with an algorithm to produce such function. Let's stick to image recognition/processing. Can you write a function that recogizes a cat or a dog on a 64x64 greyscale image? Or write a function to remove the background from a picture of a person. Can you write a function to generate a 3D depth map for a 2D picture? > These theorems only show that NNs are, like many computational statistical techniques, sufficient for a wide class of "mere approximation" problems that are only narrowly useful within the whole field of learning-as-such. That's exactly what people are looking for in most cases, the mere approximation can be good enough to consider the problem solved.
- jazzyjackson 5y ago
  > Did you really compare... I think the guilty party is whoever chose the label "neural" to describe high dimensional regression. I don't see a rebuttal in your reply. NNs can make for good classifiers but they certainly get overused trying to make predictions in spaces that are not predictable.
  - tiborsaas 5y ago
    It's just so annoying to read a condescending tone of a cool mathematical and computational tool to the degree of almost ridiculing it. It happened before and it caused the first AI winter, which was unfortunate. Maybe they are annoyed too that it gets overhyped and misrepresented.
- deleted 5y ago
  [deleted]
- mjburgess 5y ago
  I've spoken with no-end of people who think "universal fn approximation" is some magic token to be played in the AI debate -- people getting PhDs in ML no less. These are people really, in my experience, with no science background -- c-sci programmers who dont have any conceptual foundations in applied mathematics or science (outside of the discrete math taught in csci) -- and who take these properties as genuinely quite magical. "Intelligence" is, to them, just some function and if NNs can approximate any fn, then presumably they'll be intelligent. They arent aware that in a sense, everything is a dynamical function of space and time (say, stuff(x, t)) and to instantite it, one requires the entire universe. In otherwords, csci people are not used to thinking about applied mathematics in the sense of science (, implementation of functions, dynamical functions, and so on). I think it is important for this audience to demystify these properties. Being a universal fn approximator, in my view, is neither a necessary nor sufficient property of any system implementing intelligence. It's really a misdirection.
emmelaich 5y ago
Can you just show the moving stick to the NN? Are you just denying information to the NN that is available to the animal?

benrbray 5y ago

What techniques are used to estimate trajectories? Would something like a Gaussian Process perform better?

mjburgess 5y ago
Well NNs like GPs are "basically" non-parametric methods, in the sense that one does not start with a known parameterised statistical distribution that comes from domain expertise. These are worst-case techniques when we dont have the option to start with "the right answer", eg., in the case of large datasets where we have no idea how some pixels distribute over cat/dog images. In the case of a trajectory we would likey already know the answer, in the form of just doing some physics. The role of computational stats here then is to start with the known form of the solution and find the specific parameters to fit it. Since we have physics, we can find the "perfect answer" to the trajectory question with very few data points -- and take as many derivatives as we like. Brute-force ML is often used when we dont have theories, making it all the more dangerous; and alas, all the more useful. We can get a 5% improvement on click-thru rate without having any theory of human behavioural psychology --- god knows then, what we are doing to human behaviour when we implement this system. `
aqme28 5y ago
If you’re asking how to solve difficult differential equations in general, we use numerical methods like finite elements or finite differences
- adgjlsfhk1 5y ago
  that said, there has been some promising research on using NNs for solving nonlinear pdes.

curiousgal 5y ago

> There is often no function to find in solving a problem In finance NN can be used to calibrate models, i.e. generate parameters to a model (function) that replicates existing data (obsered prices/volatilities).

mjburgess 5y ago
Well i dont think prices are functions of economic variables. Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. So what you're modelling is something like `E_hope[y] = f(x)` where `E_hope` is "a hopeful expectation" that the mean of the underlying ambiguous y1,..yn does "reliably" mean to a unique `x`. This "hopeful expectation" is certainly more common than there being any actual function connecting `y` to `x`, but i think its often quite false too. Ie., even the expectation of prices is genuinely ambiguous. To handle this we might ensemble models `E_ensemble[E1_hope[y], ...En_hope[y]]`, but to repeat a famous idiom in finance, this is very much "building sandcastles in the sky". The idea that you can just "expect" (/statstics) your way out of the need for experimentation is a dangerous superstition which is at the heart of ML. It is impossible to simply "model data", measurement produces genuine ambiguities which can only be resolved by changing the world and seeing-what-happens. There is no function to find.
- thfuran 5y ago
  >Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. y1,...,yn is a perfectly reasonable function output. Functions don't have to produce scalars.
  - mjburgess 5y ago
    Functions have to resolve to one point in the output domain, even if that point is multi-dim. Here, consider `y_houseprice = price(house data, economic data, etc.)`. There isnt a unique house price in terms of those variables. The real world observes many such prices for the same value of those variables. An overly mathematical view of the world has obscured the scientific method from our thinking here. Generally, there arent actually functions from X to Y, and there arent actually stable XY distributions over time. The world, as measured, is basically always ambigouous and discontinuous. Data, as measurement, isnt the foundation of our theory-building. We build theories by changing the world; data comes in as a guiding light to our theory building, not as the basis. ..wwhich is our direct causal interactions with our environment, ie., its the actual stuff of our bodies and the stuff of the world as we change it

shawnz 5y ago

> If you go out and actually study the only known systems which learn effectively (ie., animals), one does not find the need for universal approximator theorems in explaining their capacities. What's the explanation for their capacities, then?

mjburgess 5y ago
direct, theory-laden, causal contact and pro-active engagement with their environments... eg., taking the stick out of the water. "data" (even from an animal's pov) is basically, by nature, ambiguous. Measurement is an event in the world (eg., light hitting the eye) which isn't somehow unambiguously informative. To over come this problem, basically, animals move stuff. The adaptable motor cortex of the most intelligent animals, therefore, isnt something to be tacked-on to "intelligence", it's the precondition for it. Glibly: tools before thoughts. Reason needs content to operate on, the content of our thoughts is built via our (sensory-)motor systems. The idea that we need ever more theoretically powerful models of reasoning here is a misdirection -- it misses that the heart of everything we know arrives via some form of repeated experiment. Every more automated "pure reasoning" either via statistics or symbolically, is always just a means of juicing the data we provide the system. Useful as a technology, but not as a genuine learning system. It will never have the means of resolving the many ambiguities within data itself. In the case, for example, of NLP -- the structure of 1 trillion documents will never enable a machine to answer the question "what do you like about what i'm wearing?" -- because (1) the machine isnt here with me; (2) i am asking for its personal judgement, not a summary of a trillion documents; and (3) that summary of those trillion documents has to be unique, but the question has no "right answer". Whilst computer science has the driving seat over what "intelligence" is, we will forever be stuck with this incredibly diminished view of our own capacities and the size of the technological challenge. The goal isnt to sift through everything we have already done and "take a mean", the goal is to produce a system which could have done "everything we have done" without us.
- gowld 5y ago
  Experiments are data collection. It's the iinput to the thinking process.

version_five 5y ago

> There is often no function to find in solving a problem, ie., there is no mapping from ImageSpace -> DogCatSpace. Ie., most things are genuine ambiguitites --- a stick in a water appears bent, indistinguishably from an actually bent stick in some other transparent fluid. That's tangential imo. There is a function that maps from image space to dog/cat/don't know space. A sentient being that get the "don't know" can get more info to resolve the ambiguity (or rephrase the question). A universal function approximator can still make itself useful even if all it can do it say it doesn't know. This is a question of problem setup. NNs are bad at extrapolating, including e.g. trivially to periodic functions. This is a limitation if you thought they could do that, but again a question of understanding what they do. Hype, as you say, leads some people to believe NNs are magic, leading to mismatched expectation. A universal interpolate-only function approximator is still pretty useful though, just maybe disappointing if you understood that to imply sentience

mjburgess 5y ago
Sure, we could see `animal = f(image)` as a partial function and lift it into some total function space, `maybe_animal = f(image)`. Is our reasoning here actually this total function though? We really do always need to keep in mind that animal reasoning will terminate, animals will act "guided by other reasoning", and resume the original reasoning process. Action really messes up this neat "totalising option" for partial functions. If i'm not sure if "fluffy" is a dog or a cat, i might wait longer for it to move; I might throw something at it; I might ask its owner. This isnt as simple as "don't know", since reasoning is kinda time-bound and time-parameterised, my very measuring process is sensitive to my own confidence in what-something-is. Is this really "f(image) = dont-know"? I dont think so. I think it's more like, "judgement = body-brain-state(t, reasoning-goal, {action policies}, ...large-number-of-other-things)". This gets to my issue. In my view what animals are doing is better described by a dynamical equation of state (like a wavefunction), as the whole system is operating under its own dynamical evolution (including, eg., what fluffy does in response to your puzzlement). I dont see Dog|Cat|DontKnow as the answer here. I dont think it's the actual total function which corresponds to our judgement, though it probably is total -- we just end up with "DontKnow" in all the cases actual intellkigence is requireds.... the very think we're aiming to model.

Der_Einzige 5y ago

Interpolation in GANs seems a lot like "being able to experiment. Ie., taking the stick out of the water"...

tsimionescu 5y ago
They are not taking the stick out of the water, because they don't have hands and are not looking at a real stick. They are being trained on a static data set, and a picture of a stick isn't a stick. They can try to extrapolate all they want, but they are fundamentally not going to be able to get more information out of the data than there exists. And in a static photo of a bent stick, or a fluffy critter, there simply isn't any information to tell whether this is a bent stick or a stick in water; or whether it's a cat or a dog. The intelligent response is not "I don't know", and it's not "60% it's a cat, 40% it's a dog". It's "here is the set of actions that need to be taken to create more data to be able to settle the question". And creating that set of actions is completely different from current approaches. No state of the art GAN can say "you need to view the subject from a steeper angle, check for a reflection to see if it's in water" or "poke it with a stick, see if it meows or barks", because they don't have enough information about the world in their train g sets to be able to even know that these are possibilities.
- YeGoblynQueenne 5y ago
  They simultaneously have not enough information, and too much information. A human child can learn what a chicken is by seeing a chicken once, but "seeing a chicken" is not the same as the representation of a chicken as a set of pixels in an image. It's ... something else, that we don't quite have the words to describe. Clearly, when we look at things in the real world we take in much more information than when a CNN takes in an image. Machine learning datasets have too much data, but it's the wrong kind of data and they're the wrong kind of learning system, to learn what humans can learn from "single examples". And humans can also learn about radically new categories of things that they have never experienced just by looking at an image, even a low-quality image. And that's humans which are like the apex of intelligence on the planet (I know, right?). Neural nets can't even built the representation of the world that a simpler animal, like a cockroach, must have to be able to exist autonomously in its environment. We don't have anything like a machine that can get close to the ability of a cockroach for autonomy. It's mad. But people keep thinking, "hey, we'll just use more data and more layers" and get to the moon by jumping higher with a larger pogo stick.

slibhb 5y ago

> Animals solve the problem of the "ambiguity of inference" by being in the world and being able to experiment. Ie., taking the stick out of the water. A neural network, in this sense, cannot "take the stick out the water" -- it cannot resolve ambiguities. Couldn't training data include a video of someone taking a stick out of the water?

ddingus 5y ago
Then we have no ambiguity. The data is inclusive. And the cost is a very seriously larger problem space to classify.
YeGoblynQueenne 5y ago
In that case, a neural net will learn to predict the next frame in a video stream, but it still won't learn what a stick is or why it looks bent when it's in the water.

sdenton4 5y ago

You've got a kind of narrow view of the matter... The need for interactive understanding is addressed in different ways by reinforcement learning, GANs, and autoregressive recurrent neutral networks. In the latter cases, the generative output of the network is a kind of experiment, and back prop from the loss function provides a route to improvement. I think it's an unfortunate historical accident that the field of machine learning is so transfixed with classifiers. But they're really not the only game in town.

fxtentacle 5y ago

The article also conveniently glosses over the fact that all AI calculations are limited in their maximum complexity by the depth of the AI. For example, for x! the best a regular DL AI can do is to memorize some values and interpolate between them.

tiborsaas 5y ago

Did you really compare a single neural network with a real physical agent (dog) with 5 senses, trillions of cells, a complex inter-connected brain operating in the real world? This is close to what I'd call unfair, or straw-man argument. This is like calling calculus useless, because it doesn't help you pick one kind of milk versus another in a supermarket. Universal function approximation is a great tool if used correctly. From what I understand, it comes in handy when our puny human brain can't come up with an algorithm to produce such function. Let's stick to image recognition/processing. Can you write a function that recogizes a cat or a dog on a 64x64 greyscale image? Or write a function to remove the background from a picture of a person. Can you write a function to generate a 3D depth map for a 2D picture? > These theorems only show that NNs are, like many computational statistical techniques, sufficient for a wide class of "mere approximation" problems that are only narrowly useful within the whole field of learning-as-such. That's exactly what people are looking for in most cases, the mere approximation can be good enough to consider the problem solved.

jazzyjackson 5y ago
> Did you really compare... I think the guilty party is whoever chose the label "neural" to describe high dimensional regression. I don't see a rebuttal in your reply. NNs can make for good classifiers but they certainly get overused trying to make predictions in spaces that are not predictable.
- tiborsaas 5y ago
  It's just so annoying to read a condescending tone of a cool mathematical and computational tool to the degree of almost ridiculing it. It happened before and it caused the first AI winter, which was unfortunate. Maybe they are annoyed too that it gets overhyped and misrepresented.
deleted 5y ago
[deleted]
mjburgess 5y ago
I've spoken with no-end of people who think "universal fn approximation" is some magic token to be played in the AI debate -- people getting PhDs in ML no less. These are people really, in my experience, with no science background -- c-sci programmers who dont have any conceptual foundations in applied mathematics or science (outside of the discrete math taught in csci) -- and who take these properties as genuinely quite magical. "Intelligence" is, to them, just some function and if NNs can approximate any fn, then presumably they'll be intelligent. They arent aware that in a sense, everything is a dynamical function of space and time (say, stuff(x, t)) and to instantite it, one requires the entire universe. In otherwords, csci people are not used to thinking about applied mathematics in the sense of science (, implementation of functions, dynamical functions, and so on). I think it is important for this audience to demystify these properties. Being a universal fn approximator, in my view, is neither a necessary nor sufficient property of any system implementing intelligence. It's really a misdirection.

emmelaich 5y ago

Can you just show the moving stick to the NN? Are you just denying information to the NN that is available to the animal?