5 ms·

I disagree. In engineering we've shown great advancements not by exactly reproducing biology from cases like transportation (wheel vs legs) or flight (engine an

by emcq 11y ago

I disagree. In engineering we've shown great advancements not by exactly reproducing biology from cases like transportation (wheel vs legs) or flight (engine and wing vs flapping feathers). We can't even mass produce synthetic muscles and instead use a geared motor. The growth in building increasingly sophisticated AI is faster than our efforts to reverse engineer biology. I could see that changing with improved observational techniques like optogenetics or bacteria and viruses we can "program" to explore. Researchers are already focusing on concrete insights from cognitive science, neuroscience, etc such as one shot learning or memory that we haven't yet figured out in a cohesive machine learning framework. For the time being I'd bet on more advaces coming without significant changes in biological understanding.

sabertoothed 11y ago
I feel that your analogy does not disprove OP's claim. You would first have to prove that the analogy is actually applicable. Theoretically, one could argue that motion is substantially different from intelligence such that the analogy does not hold. But I am with you in that engineering / CS / ... should not wait for neuroscience to make further discoveries but continue the journey.
- emcq 11y ago
  Interesting point, but which do you think is a more reasonable prior: that intelligence is a deterministic process similar other's we've encountered or that there is something uniquely different about intelligence than other physical phenomena? I think the latter requires more assumptions so I would argue philosophically it is the one that requires the proof or more evidence :) Regardless, my feeling is that there is a healthy dose of human hubris around intelligence. If I train a dog to go fetch me a beer from the fridge, that seems pretty smart. It learned how to understand a request, execute a complex sequence of actions for motion and planning, reason around occluded objects, understand depth and 3 dimensional space, differentiate between objects, and more without me writing a sequence of rules to follow. I'd be happy to have a robot that intelligent. Plants dont have brains or neurons yet react to sensory stimulus such as light touch or sound, communicate, and even have memory and learn. It's not at a scale to do something interesting within one plant but communities of plants are arguably the most successful organisms on the planet. Andrew Ng likes to point to a Ferret experiment [0] where experimental neuroscientists rewired the visual inputs to the auditory cortex and the auditory cortex learned to "see" the visual signals! This suggests that there may be some amount of unified "learning" rules to the brain. Biology is never so clean but if humans have whatever this intelligence thing is that lesser organisms do not, there is another angle to look at things. We have a lot of neurons which suggests less per neuron specialization than say a C. elegans; basically large populations of neurons perform tasks that in lesser creatures single or few neurons may perform. While the trees are complex and important to understand for biology and medicine, the forest may have some high level rules. Looking at something that appeared intelligent 50-100 years ago but seems mechanical now, we have text to speech. NETtalk was a simplified computational neuroscience model from the 80s that could synthesize human speech. Today we have far better quality techniques that came out of R&D focused on things like large high quality labeled datasets for training, better soundcards, more processing power, and algorithmic improvements. Researchers didn't continue trying to model the brain and instead threw an HMM and a couple other tricks at it. Now we're going full circle back to neural networks but they aren't using any advances from biology and certainly arent produced by computational neuroscientists like Terry Sejnowski. It's funny because at the time of NETtalk they thought that learning to read would be an extremely hard problem because it incorporates so many components of the human brain [1]. While it certainly wasn't a trivial problem, state of the art OCR and object recognition came from similar artificial neural networks a decade later with LeNet and MNIST * . And no, ANNs != biological neuronal networks. The models of computational neuroscientists are different; for example look at [2, 3] for high level models or [4] for a tool. Now I'm even more convinced than before that understanding the brain is great for humanity but wont be necessary for building intelligent systems that can perform tasks similar to biological ones. [0] http://www.nature.com/nature/journal/v404/n6780/full/404871a0.html http://www.nature.com/nature/journal/v404/n6780/full/404871a... [1] https://en.wikipedia.org/wiki/NETtalk_(artificial_neural_network) https://en.wikipedia.org/wiki/NETtalk_(artificial_neural_net... [2] http://science.sciencemag.org/content/338/6111/1202 http://science.sciencemag.org/content/338/6111/1202 [3] http://ganguli-gang.stanford.edu/pdf/InvModelTheory.pdf http://ganguli-gang.stanford.edu/pdf/InvModelTheory.pdf [4] http://neuralensemble.org/docs/PyNN/index.html http://neuralensemble.org/docs/PyNN/index.html * Perhaps you could argue that convolutions are loosely inspired by neuron structure, but that sort of knowledge had existed for quite some time, with inspiration arguably within Camillo Golgi's amazing neuronal physiology diagrams from the 1870s let alone the 1960-80s. It's telling that papers on CNNs have little to no neuroscience and a lot of applied math :)
  - sabertoothed 11y ago
    I did not mean to have implied that I think there is anything magical about intelligence. Of course, it is based on physical phenomena. I am doing my PhD right now and I try to incorporate as much AI/ML into it as possible. What I meant to say is that our ANNs are so ridiculously simplified versions of real neural networks that there might still be something to be learnt from the real brain. This shall not imply that to achieve intelligence, the solution necessarily has to mimic a biological brain. (Thank you for your detailed response. I love to read about this stuff!) edit: missing word
simonh 11y ago
I think you're right, and AlphaGo is an example. The big advancements in flight give us supersonic jets far faster than birds, but we've only recently developed microdrones that can land on a tree branch. We have wheeled vehicles far faster than horses, but haven't got a single vehicle that can complete an Eventing course. So in AI we have computers far better at Chess than any human, far better at indexing and searching databases, and now (or certainly very soon) far better at playing Go. But we are still nowhere near developing a computer that can learn to play Monopoly, Risk or Axis & Allies just from reading the rule book and looking at the components. If you aim your machine at a very narrow subset of a problem, you can optimise it to amazing degrees, far exceeding humans or nature. But developing a machine that has the whole package is staggeringly hard in comparison. But you know what? That's fine. Special purpose, single domain tools are fantastically useful and are easier to make highly reliable, with well understood limitations.

by emcq 11y ago

sabertoothed 11y ago

I feel that your analogy does not disprove OP's claim. You would first have to prove that the analogy is actually applicable. Theoretically, one could argue that motion is substantially different from intelligence such that the analogy does not hold. But I am with you in that engineering / CS / ... should not wait for neuroscience to make further discoveries but continue the journey.

emcq 11y ago
Interesting point, but which do you think is a more reasonable prior: that intelligence is a deterministic process similar other's we've encountered or that there is something uniquely different about intelligence than other physical phenomena? I think the latter requires more assumptions so I would argue philosophically it is the one that requires the proof or more evidence :) Regardless, my feeling is that there is a healthy dose of human hubris around intelligence. If I train a dog to go fetch me a beer from the fridge, that seems pretty smart. It learned how to understand a request, execute a complex sequence of actions for motion and planning, reason around occluded objects, understand depth and 3 dimensional space, differentiate between objects, and more without me writing a sequence of rules to follow. I'd be happy to have a robot that intelligent. Plants dont have brains or neurons yet react to sensory stimulus such as light touch or sound, communicate, and even have memory and learn. It's not at a scale to do something interesting within one plant but communities of plants are arguably the most successful organisms on the planet. Andrew Ng likes to point to a Ferret experiment [0] where experimental neuroscientists rewired the visual inputs to the auditory cortex and the auditory cortex learned to "see" the visual signals! This suggests that there may be some amount of unified "learning" rules to the brain. Biology is never so clean but if humans have whatever this intelligence thing is that lesser organisms do not, there is another angle to look at things. We have a lot of neurons which suggests less per neuron specialization than say a C. elegans; basically large populations of neurons perform tasks that in lesser creatures single or few neurons may perform. While the trees are complex and important to understand for biology and medicine, the forest may have some high level rules. Looking at something that appeared intelligent 50-100 years ago but seems mechanical now, we have text to speech. NETtalk was a simplified computational neuroscience model from the 80s that could synthesize human speech. Today we have far better quality techniques that came out of R&D focused on things like large high quality labeled datasets for training, better soundcards, more processing power, and algorithmic improvements. Researchers didn't continue trying to model the brain and instead threw an HMM and a couple other tricks at it. Now we're going full circle back to neural networks but they aren't using any advances from biology and certainly arent produced by computational neuroscientists like Terry Sejnowski. It's funny because at the time of NETtalk they thought that learning to read would be an extremely hard problem because it incorporates so many components of the human brain [1]. While it certainly wasn't a trivial problem, state of the art OCR and object recognition came from similar artificial neural networks a decade later with LeNet and MNIST * . And no, ANNs != biological neuronal networks. The models of computational neuroscientists are different; for example look at [2, 3] for high level models or [4] for a tool. Now I'm even more convinced than before that understanding the brain is great for humanity but wont be necessary for building intelligent systems that can perform tasks similar to biological ones. [0] http://www.nature.com/nature/journal/v404/n6780/full/404871a0.html http://www.nature.com/nature/journal/v404/n6780/full/404871a... [1] https://en.wikipedia.org/wiki/NETtalk_(artificial_neural_network) https://en.wikipedia.org/wiki/NETtalk_(artificial_neural_net... [2] http://science.sciencemag.org/content/338/6111/1202 http://science.sciencemag.org/content/338/6111/1202 [3] http://ganguli-gang.stanford.edu/pdf/InvModelTheory.pdf http://ganguli-gang.stanford.edu/pdf/InvModelTheory.pdf [4] http://neuralensemble.org/docs/PyNN/index.html http://neuralensemble.org/docs/PyNN/index.html * Perhaps you could argue that convolutions are loosely inspired by neuron structure, but that sort of knowledge had existed for quite some time, with inspiration arguably within Camillo Golgi's amazing neuronal physiology diagrams from the 1870s let alone the 1960-80s. It's telling that papers on CNNs have little to no neuroscience and a lot of applied math :)
- sabertoothed 11y ago
  I did not mean to have implied that I think there is anything magical about intelligence. Of course, it is based on physical phenomena. I am doing my PhD right now and I try to incorporate as much AI/ML into it as possible. What I meant to say is that our ANNs are so ridiculously simplified versions of real neural networks that there might still be something to be learnt from the real brain. This shall not imply that to achieve intelligence, the solution necessarily has to mimic a biological brain. (Thank you for your detailed response. I love to read about this stuff!) edit: missing word

simonh 11y ago

I think you're right, and AlphaGo is an example. The big advancements in flight give us supersonic jets far faster than birds, but we've only recently developed microdrones that can land on a tree branch. We have wheeled vehicles far faster than horses, but haven't got a single vehicle that can complete an Eventing course. So in AI we have computers far better at Chess than any human, far better at indexing and searching databases, and now (or certainly very soon) far better at playing Go. But we are still nowhere near developing a computer that can learn to play Monopoly, Risk or Axis & Allies just from reading the rule book and looking at the components. If you aim your machine at a very narrow subset of a problem, you can optimise it to amazing degrees, far exceeding humans or nature. But developing a machine that has the whole package is staggeringly hard in comparison. But you know what? That's fine. Special purpose, single domain tools are fantastically useful and are easier to make highly reliable, with well understood limitations.