4 ms·

Did you really compare a single neural network with a real physical agent (dog) with 5 senses, trillions of cells, a complex inter-connected brain operating in

Did you really compare a single neural network with a real physical agent (dog) with 5 senses, trillions of cells, a complex inter-connected brain operating in the real world? This is close to what I'd call unfair, or straw-man argument. This is like calling calculus useless, because it doesn't help you pick one kind of milk versus another in a supermarket. Universal function approximation is a great tool if used correctly. From what I understand, it comes in handy when our puny human brain can't come up with an algorithm to produce such function. Let's stick to image recognition/processing. Can you write a function that recogizes a cat or a dog on a 64x64 greyscale image? Or write a function to remove the background from a picture of a person. Can you write a function to generate a 3D depth map for a 2D picture? > These theorems only show that NNs are, like many computational statistical techniques, sufficient for a wide class of "mere approximation" problems that are only narrowly useful within the whole field of learning-as-such. That's exactly what people are looking for in most cases, the mere approximation can be good enough to consider the problem solved.

jazzyjackson 5y ago
> Did you really compare... I think the guilty party is whoever chose the label "neural" to describe high dimensional regression. I don't see a rebuttal in your reply. NNs can make for good classifiers but they certainly get overused trying to make predictions in spaces that are not predictable.
- tiborsaas 5y ago
  It's just so annoying to read a condescending tone of a cool mathematical and computational tool to the degree of almost ridiculing it. It happened before and it caused the first AI winter, which was unfortunate. Maybe they are annoyed too that it gets overhyped and misrepresented.
deleted 5y ago
[deleted]
mjburgess 5y ago
I've spoken with no-end of people who think "universal fn approximation" is some magic token to be played in the AI debate -- people getting PhDs in ML no less. These are people really, in my experience, with no science background -- c-sci programmers who dont have any conceptual foundations in applied mathematics or science (outside of the discrete math taught in csci) -- and who take these properties as genuinely quite magical. "Intelligence" is, to them, just some function and if NNs can approximate any fn, then presumably they'll be intelligent. They arent aware that in a sense, everything is a dynamical function of space and time (say, stuff(x, t)) and to instantite it, one requires the entire universe. In otherwords, csci people are not used to thinking about applied mathematics in the sense of science (, implementation of functions, dynamical functions, and so on). I think it is important for this audience to demystify these properties. Being a universal fn approximator, in my view, is neither a necessary nor sufficient property of any system implementing intelligence. It's really a misdirection.
- tiborsaas 5y ago
  I'm one of these persons with no formal training in any scientific fields. I don't think any of what you assume people like me think in your stereotype. I know it's not magic, it's brute force problem solving. Once you are capable of training systems like this, you get to solutions which are really powerful. All you see is numbers and functions, and all I see is problems being solved by these AI/ML methods and often in quantifiably better ways than intelligent humans can. NN-s are the building blocks. Neurons in your brain are dumb as well, it's the quality of the network that matters. So instead of moving goalposts, what do you think is required to implement intelligence?
  - mjburgess 5y ago
    Well, I'm not making a sterotype -- I'm offering an explanation of the people i've met. It's very hard to converse with people whose academic background is discrete mathematics, in an essentially empirical domain (that of modelling intelligence). There's a lot which is odd (and dubious) in AI/ML that traces is origins to this peculilar situation of a discipline (csci) in the "early phases" of modelling an empirical phenomenon, without yet, an intra-disciplinary theoretical support for it -- csci people take geometry and classical physics to make video games. They dont yet take the equivalent to make intelligent systems (which would include stuff on learning in animals, humans; and in my view, more applied math). In any case, to answer your question about implementation, see https://news.ycombinator.com/threads?id=mjburgess#30579711 https://news.ycombinator.com/threads?id=mjburgess#30579711

jazzyjackson 5y ago

> Did you really compare... I think the guilty party is whoever chose the label "neural" to describe high dimensional regression. I don't see a rebuttal in your reply. NNs can make for good classifiers but they certainly get overused trying to make predictions in spaces that are not predictable.

tiborsaas 5y ago
It's just so annoying to read a condescending tone of a cool mathematical and computational tool to the degree of almost ridiculing it. It happened before and it caused the first AI winter, which was unfortunate. Maybe they are annoyed too that it gets overhyped and misrepresented.

deleted 5y ago

[deleted]

mjburgess 5y ago

I've spoken with no-end of people who think "universal fn approximation" is some magic token to be played in the AI debate -- people getting PhDs in ML no less. These are people really, in my experience, with no science background -- c-sci programmers who dont have any conceptual foundations in applied mathematics or science (outside of the discrete math taught in csci) -- and who take these properties as genuinely quite magical. "Intelligence" is, to them, just some function and if NNs can approximate any fn, then presumably they'll be intelligent. They arent aware that in a sense, everything is a dynamical function of space and time (say, stuff(x, t)) and to instantite it, one requires the entire universe. In otherwords, csci people are not used to thinking about applied mathematics in the sense of science (, implementation of functions, dynamical functions, and so on). I think it is important for this audience to demystify these properties. Being a universal fn approximator, in my view, is neither a necessary nor sufficient property of any system implementing intelligence. It's really a misdirection.

tiborsaas 5y ago
I'm one of these persons with no formal training in any scientific fields. I don't think any of what you assume people like me think in your stereotype. I know it's not magic, it's brute force problem solving. Once you are capable of training systems like this, you get to solutions which are really powerful. All you see is numbers and functions, and all I see is problems being solved by these AI/ML methods and often in quantifiably better ways than intelligent humans can. NN-s are the building blocks. Neurons in your brain are dumb as well, it's the quality of the network that matters. So instead of moving goalposts, what do you think is required to implement intelligence?
- mjburgess 5y ago
  Well, I'm not making a sterotype -- I'm offering an explanation of the people i've met. It's very hard to converse with people whose academic background is discrete mathematics, in an essentially empirical domain (that of modelling intelligence). There's a lot which is odd (and dubious) in AI/ML that traces is origins to this peculilar situation of a discipline (csci) in the "early phases" of modelling an empirical phenomenon, without yet, an intra-disciplinary theoretical support for it -- csci people take geometry and classical physics to make video games. They dont yet take the equivalent to make intelligent systems (which would include stuff on learning in animals, humans; and in my view, more applied math). In any case, to answer your question about implementation, see https://news.ycombinator.com/threads?id=mjburgess#30579711 https://news.ycombinator.com/threads?id=mjburgess#30579711