7 ms·

I think we need more advances in neuroscience and, I know this will be controversial, psychology before we really know what the cake even is. Edit: I actually

• jacquesm 11y ago
  > I know this will be controversial, psychology Why would that be controversial? It seems to make extremely good sense and even though it may be doubted in some circles I think that most people involved in AI research are painfully aware of our limited understanding of our own psyche.
  • argonaut 11y ago
    I disagree with hacknat (and the reason I suspect it's controversial) because psychology is generally far too high-level (e.g. focused on things like traits and high-level behavior and vague "perceptions") and has far too little empirical rigor for engineers to be able to build any sort of actual system from insights gleaned from psychology. The recent replication crisis in psychology does little to help this reputation. Neuroscience does concern itself a great deal with low level biological mechanisms and architectures, and is more amenable to cross-pollination with machine learning. Though I would like to point out that thus far deep learning has taken few ideas from neuroscience.
    • jacquesm 11y ago
      Neural networks are the big driver and arguably were taken from neuroscience. The reason why psychology is 'too high-level' is exactly what is meant with 'we don't understand it', we're approaching the psyche at the macro level of observable traits, but there is a very large gap between the 'wiring' and the 'traits', some of that gap belongs to neuroscience but quite possibly the larger parts belongs to psychology. The two will meet somewhere in the middle. A similar thing happens in biology with dna and genetics on the one side and embryology on the other.
      • argonaut 11y ago
        They were taken from neuroscience... 50 years ago. Since then, very few ideas have been explicitly taken from neuroscience.
        • quonn 11y ago
          Exactly, and even then, half of the details or how real neurons work were ignored or discarded. NN are now often taught by sticking closely to the math and avoiding the term and any reference to biology completely.
          • jacquesm 11y ago
            Does it matter? As long as the original source for the idea is remembered I don't particularly care which field originated it. There are so many instances and examples of cross pollination between IT and other sciences that in the end whether or not we stick to the original in a literal fashion or not should not matter (and if we did we'd probably lose out on a good bit of progress).
        • jacquesm 11y ago
          That's like saying there has been no development in automobiles since 1890. Sure, at first glance the cars from then are still like the cars from now. ICE (or electrical) power source, maybe a gearbox, some seats, a steering wheel and something to keep you out of the weather. But that's at the same time ignoring many years of work on those concepts in order to further refine them. The neural networks that were taken 'explicitly from neuroscience' have gone through a vast transformation and that + a whole lot of work on training and other stuff besides is what powers the current crop of AI software. All the way to computers that learn about games, that label images and that drive cars with impressive accuracy to date. The problem is - and I think that was what the original question was about - that neuroscience is rather very low level. We need something at the intermediate level, a 'useful building block' approach if you will, something that is not quite a fully formed intelligence but also not so basic as plumbing and wiring.
          • 11y ago
    • hacknat 11y ago
      If anything the replication crisis in Psychology is proof of just how little we really know about intelligence. I think Psychology is necessary, because I think it needs to come up with an adequate definition of what intelligence is first, then neuroscience can say, "this is how that definition actually works", then engineers can build it.
    • paganel 11y ago
      > and has far too little empirical rigor for engineers I think this is one of the key points of the problem: we, as engineers, are expecting this problem (imitating the human psyche/getting to true AI) to be able to be defined rigorously. What if it can't? I'm not a religious person and I don't generally believe in something inside us that it's "undefinable", but looking at our cultural and intellectual history of about 2,500 years I can see that there are lots of things that we we haven't been able to define rigorously, but which are quintessential to human psyche: poetry, telling jokes, word puns, the sentiment of nostalgia and I could go on and on.
    • ewjordan 11y ago
      Even now, deep learning is basically 1980s-era backdrop with a few tweaks and better hardware - it turns out that despite decades of backlash against it, plain old backdrop pretty much is the shit, as long as you have a lot of compute power and a few tricks up your sleeve!
  • quonn 11y ago
    Well, it just seems unlikely that psychology will produce significant insights here, because it mostly just looks at how the brain behaves. While this can be insightful, I doubt that it will explain intelligence in the end, because this happens a layer below. That's precisely what neuroscience covers. The other approach (just thinking about the problem and trying to to build a AI from first principles) is CS. So I would expect strong AI from CS, possibly with a paradigm shift caused by an advance in neuroscience.
    • hacknat 11y ago
      I think someone needs to come up with a good theory of what intelligence even is, then we can try to discover its mechanism(s).
      • quonn 11y ago
        AI has some good definitions as far as intelligence is concerned. Perhaps you are worried about consciousness or something, but this is not needed for a definition of what intelligence means.
      • ktRolster 11y ago
        Here are three potential definitions: 1) Acting like a human (this is the Turing test approach) 2) Thinking like a human (this is cognitive science, and for now is focused on figuring out how humans think) 3) Thinking rationally, ie, following formal logic (the difficulty here is encoding all the information in the world as formal logic) 4) Acting rationally. That is, entities that react rationally to their goals (this one is notable because it allows fairly stupid entities). These are all explained in more detail in Artificial Intelligence, a Modern Approach by Russell and Norvig
        • hacknat 11y ago
          Those aren't broad definitions at all! Quick question though, what is "acting", what is "thinking", what is "rational"? Your definition of intelligence practically includes the term in its definition(s).
• drcode 11y ago
  Just like ornithologists invented the first airplane, and horse vets built the first automobile, etc.
  • tim333 11y ago
    Though airplane designers probably looked at bird wings for inspiration. Similar to how Deep Mind are looking at the brain.
    • jacquesm 11y ago
      Early airplane inventors tried very hard to imitate flapping bird wings for propulsion. https://en.wikipedia.org/wiki/Ornithopter https://en.wikipedia.org/wiki/Ornithopter Some even worked.
  • djcb 11y ago
    While they weren't card-carrying ornithologists, the Wright brothers studied bird flight extensively and explicitly used it for their designs. But they concluded that the flapping motion wasn't essential for propulsion, and could more easily be achieved by a propeller.
• emcq 11y ago
  I disagree. In engineering we've shown great advancements not by exactly reproducing biology from cases like transportation (wheel vs legs) or flight (engine and wing vs flapping feathers). We can't even mass produce synthetic muscles and instead use a geared motor. The growth in building increasingly sophisticated AI is faster than our efforts to reverse engineer biology. I could see that changing with improved observational techniques like optogenetics or bacteria and viruses we can "program" to explore. Researchers are already focusing on concrete insights from cognitive science, neuroscience, etc such as one shot learning or memory that we haven't yet figured out in a cohesive machine learning framework. For the time being I'd bet on more advaces coming without significant changes in biological understanding.
  • sabertoothed 11y ago
    I feel that your analogy does not disprove OP's claim. You would first have to prove that the analogy is actually applicable. Theoretically, one could argue that motion is substantially different from intelligence such that the analogy does not hold. But I am with you in that engineering / CS / ... should not wait for neuroscience to make further discoveries but continue the journey.
    • emcq 11y ago
      Interesting point, but which do you think is a more reasonable prior: that intelligence is a deterministic process similar other's we've encountered or that there is something uniquely different about intelligence than other physical phenomena? I think the latter requires more assumptions so I would argue philosophically it is the one that requires the proof or more evidence :) Regardless, my feeling is that there is a healthy dose of human hubris around intelligence. If I train a dog to go fetch me a beer from the fridge, that seems pretty smart. It learned how to understand a request, execute a complex sequence of actions for motion and planning, reason around occluded objects, understand depth and 3 dimensional space, differentiate between objects, and more without me writing a sequence of rules to follow. I'd be happy to have a robot that intelligent. Plants dont have brains or neurons yet react to sensory stimulus such as light touch or sound, communicate, and even have memory and learn. It's not at a scale to do something interesting within one plant but communities of plants are arguably the most successful organisms on the planet. Andrew Ng likes to point to a Ferret experiment [0] where experimental neuroscientists rewired the visual inputs to the auditory cortex and the auditory cortex learned to "see" the visual signals! This suggests that there may be some amount of unified "learning" rules to the brain. Biology is never so clean but if humans have whatever this intelligence thing is that lesser organisms do not, there is another angle to look at things. We have a lot of neurons which suggests less per neuron specialization than say a C. elegans; basically large populations of neurons perform tasks that in lesser creatures single or few neurons may perform. While the trees are complex and important to understand for biology and medicine, the forest may have some high level rules. Looking at something that appeared intelligent 50-100 years ago but seems mechanical now, we have text to speech. NETtalk was a simplified computational neuroscience model from the 80s that could synthesize human speech. Today we have far better quality techniques that came out of R&D focused on things like large high quality labeled datasets for training, better soundcards, more processing power, and algorithmic improvements. Researchers didn't continue trying to model the brain and instead threw an HMM and a couple other tricks at it. Now we're going full circle back to neural networks but they aren't using any advances from biology and certainly arent produced by computational neuroscientists like Terry Sejnowski. It's funny because at the time of NETtalk they thought that learning to read would be an extremely hard problem because it incorporates so many components of the human brain [1]. While it certainly wasn't a trivial problem, state of the art OCR and object recognition came from similar artificial neural networks a decade later with LeNet and MNIST * . And no, ANNs != biological neuronal networks. The models of computational neuroscientists are different; for example look at [2, 3] for high level models or [4] for a tool. Now I'm even more convinced than before that understanding the brain is great for humanity but wont be necessary for building intelligent systems that can perform tasks similar to biological ones. [0] http://www.nature.com/nature/journal/v404/n6780/full/404871a0.html http://www.nature.com/nature/journal/v404/n6780/full/404871a... [1] https://en.wikipedia.org/wiki/NETtalk_(artificial_neural_network) https://en.wikipedia.org/wiki/NETtalk_(artificial_neural_net... [2] http://science.sciencemag.org/content/338/6111/1202 http://science.sciencemag.org/content/338/6111/1202 [3] http://ganguli-gang.stanford.edu/pdf/InvModelTheory.pdf http://ganguli-gang.stanford.edu/pdf/InvModelTheory.pdf [4] http://neuralensemble.org/docs/PyNN/index.html http://neuralensemble.org/docs/PyNN/index.html * Perhaps you could argue that convolutions are loosely inspired by neuron structure, but that sort of knowledge had existed for quite some time, with inspiration arguably within Camillo Golgi's amazing neuronal physiology diagrams from the 1870s let alone the 1960-80s. It's telling that papers on CNNs have little to no neuroscience and a lot of applied math :)
• typon 11y ago
  I disagree. I think we need advances in philosophy to truly answer these questions in depth.
  • bloaf 11y ago
    I'm not so sure about this. If I had to guess, I suspect things will go like this: 1. Scientists get a really good understanding of how learning works. 2. One third of the philosophers claim that their philosophical ideas are vindicated, another third claim that their models aren't really contradicted by this new scientific model, and the final third claim that the scientists have somehow missed the point, and that special philosophical learning is still unexplained.
  • outlace 11y ago
    Read David Deutsch's book eh?
• deleted 11y ago
  [deleted]
• cLeEOGPw 11y ago
  That's an equivalent of saying that advancements in airplanes will come from biology rather than engineering fields. Biology at best can give hints to improve aerodynamics, and it still can be solved mathematically better. Same will be with neuroscience. I believe it's more likely that engineering of AI will bring new ideas to neuroscience instead, just like after building helicopters we gained some intuition and understanding on why certain features of dragonflies exist.
  • yellowstuff 11y ago
    The Wright brothers studied bird flight extensively, and drew important ideas from birds. An aeronautical engineer today has the luxury of a more mature field, and can probably afford to put less thought into bird flight. Despite the history and significant progress in AI we still don't know that much about what approach will result in the first strong AI, or even if it's possible to make one. In an important sense AI is more like aeronautical engineering in 1902 than aeronautical engineering today, so it's possible that better understanding of biology will result in an important innovation.