5 ms·

Deep learning is great and useful and a big step forward, but it's still basically just brute-forced pattern matching. Wonderful for some kinds of data analysis

Deep learning is great and useful and a big step forward, but it's still basically just brute-forced pattern matching. Wonderful for some kinds of data analysis but, without additional architectural breakthroughs, useless for developing the kinds of AI everyone dreams about -- systems that can take creative action in a world (whether the real world or some virtually constructed one) and evaluate the effects and utility (for some interesting utility function) of said actions.

frutiger 11y ago
> but it's still basically just brute-forced pattern matching Do you have any reason to believe the human brain is any different? We just have more neurons.
- shock-value 11y ago
  We have more neurons and, in particular, they are arranged in architecturally interesting ways, rather than just feedforward networks as in most current strategies for deep learning. There is of course work being done on recurrent networks, but it's basically in its infancy. And what is being done so far is still nowhere near as complex as the human brain, which clearly has many distinct areas with distinct but of course interrelated functions.
  - maroonblazer 11y ago
    I'm not an AI researcher so this is probably a naive question: Do we necessariliy need to mimic the architecture of the human brain to achieve AGI? As someone (Chalmers?) once said about the problem of consciousness: we didn't need to replicate the flapping of wings or the locomotion of sea creatures before taking to the air or underwater. Might it not also be the case with AI that there's some fundamental principle we've yet to discover that just so happens to have expression in the substrate of 1200cc's of fatty tissue, but could possess the same fidelity (and greater) in silicon and looks nothing like the architecture of the human brain?
    - shock-value 11y ago
      With respect to needing to mimic biology: not necessarily, though it is certainly true that e.g. a purely feedforward model could NOT achieve AGI, since it can't make predictions over time. Feedforward just means that the outputs of the network don't "feed" back into its inputs -- and as a result the same input will always cause the same output regardless of previous inputs. Humans/animals certainly take time into account when acting -- e.g. something as simple as coordinating muscles for simple movements requires varied output over time, despite most inputs (in the form of tactile senses) being mostly unchanged (at least for some kinds of movement, e.g. waving your hand). Recurrent models (where some outputs are connected back to the inputs) are one possible way to account for time, but the work is still really early on these methods, and it's not clear what architecture (e.g. which and how many inputs and outputs should connect) would be efficacious.
    - rdtsc 11y ago
      > of wings or the locomotion of sea creatures before taking to the air or underwater. It is not all or nothing. There are analogies at different abstraction level. Yes we probably shouldn't be plugging in continuous differential equations to mimic chemistry of neuroreceptors, cell sodium channels etc. to replicate it at that level. So in that respect we agree, airplanes are not like birds. Far from it. No flapping. Not composed of cells. Not biological in nature. On the other hand, there is another way to look at systems -- look at higher functional components and how they are connected. So maybe there is a language processing area connecting to memory. And so on. This is called the connectome of the brian as well. Which identifies what parts are connected to what. In this regards airplanes are similar to birds. They both have wings. Fuselage. A tail. They are built with similar structural material contraints -- light and durable. Aluminum, titanium for aircraft, and porous bones for birds. Another way to look at it is in so many decades of AI, we haven't yet come up with another model. So while having to wait for enlightment to hit us one day why not learn from an already existent example.
rl3 11y ago
In Superintelligence, Bostrom does not presume to know the path via which we will arrive at AGI. One hypothetical path discussed is that of tool AI. That is, robust search processes - things we are already quite adept at (genetic algorithms, deep learning, etc) - purposed towards AGI-related goals. It's not hard to imagine these existing methods being used in the pursuit of a recursively self-optimizing agent (seed AI) that then snowballs into AGI. Such an approach may not require any fundamental knowledge concerning the nature or architecture of AGI. It would simply be an application of brute computational force using existing tools and knowledge.
- shock-value 11y ago
  I do agree that current methods like deep learning's neural nets will end up being a part of some future general AI -- e.g. the convolutional deep nets used for image recognition are already reputedly similar to some aspects of human/animal biological vision systems. Yet even if all we need is this sort of "seed AI", we still need new architectural insights to be able to create it in the first place. Otherwise someone would have surely demonstrated it by now? If nothing else, such a system would need to evaluate effects of its outputs on its inputs over time (e.g. if I shoot a basketball, it takes seconds before it either goes in or doesn't; if I plant a seed in the ground, months will pass before it sprouts -- or not, depending on conditions). Research into recurrent networks, one possible avenue for doing this, is still pretty primitive. And I'm not sure I'm convinced that this core "seed AI" is sufficient to emulate human cognition. Such a system might effectively integrate audio and visual senses, for example (in order to combine both for prediction tasks), but could such a system ever emulate the sort of continuous verbal inner monologue we all have which narrates our experience? That we have this inner monologue which seems to run alongside our other senses but yet makes use of them (along with stored memories) suggests, at least to me, that some more complicated pathways are involved which link together these various "component systems" (senses, stored memories, emotional states, linguistic synthesis, etc.) beyond just the simple prediction/reward circuitry which I presume the "seed AI" would encapsulate.
  - rl3 11y ago
    >Yet even if all we need is this sort of "seed AI", we still need new architectural insights to be able to create it in the first place. Not necessarily. That was my entire point, that robust search processes using existing tools and knowledge may yield a seed AI. >Otherwise someone would have surely demonstrated it by now? Again, not necessarily. AGI may not yet exist primarily due to dumb luck. The computational requirements, especially when you consider most computing capacity on the planet is networked, may already be adequate or even far exceed adequate. >And I'm not sure I'm convinced that this core "seed AI" is sufficient to emulate human cognition. It probably won't be. It will most likely be completely alien when compared to human cognition. At the same time, that doesn't preclude it from being vastly more powerful.

frutiger 11y ago

> but it's still basically just brute-forced pattern matching Do you have any reason to believe the human brain is any different? We just have more neurons.

shock-value 11y ago
We have more neurons and, in particular, they are arranged in architecturally interesting ways, rather than just feedforward networks as in most current strategies for deep learning. There is of course work being done on recurrent networks, but it's basically in its infancy. And what is being done so far is still nowhere near as complex as the human brain, which clearly has many distinct areas with distinct but of course interrelated functions.
- maroonblazer 11y ago
  I'm not an AI researcher so this is probably a naive question: Do we necessariliy need to mimic the architecture of the human brain to achieve AGI? As someone (Chalmers?) once said about the problem of consciousness: we didn't need to replicate the flapping of wings or the locomotion of sea creatures before taking to the air or underwater. Might it not also be the case with AI that there's some fundamental principle we've yet to discover that just so happens to have expression in the substrate of 1200cc's of fatty tissue, but could possess the same fidelity (and greater) in silicon and looks nothing like the architecture of the human brain?
  - shock-value 11y ago
    With respect to needing to mimic biology: not necessarily, though it is certainly true that e.g. a purely feedforward model could NOT achieve AGI, since it can't make predictions over time. Feedforward just means that the outputs of the network don't "feed" back into its inputs -- and as a result the same input will always cause the same output regardless of previous inputs. Humans/animals certainly take time into account when acting -- e.g. something as simple as coordinating muscles for simple movements requires varied output over time, despite most inputs (in the form of tactile senses) being mostly unchanged (at least for some kinds of movement, e.g. waving your hand). Recurrent models (where some outputs are connected back to the inputs) are one possible way to account for time, but the work is still really early on these methods, and it's not clear what architecture (e.g. which and how many inputs and outputs should connect) would be efficacious.
  - rdtsc 11y ago
    > of wings or the locomotion of sea creatures before taking to the air or underwater. It is not all or nothing. There are analogies at different abstraction level. Yes we probably shouldn't be plugging in continuous differential equations to mimic chemistry of neuroreceptors, cell sodium channels etc. to replicate it at that level. So in that respect we agree, airplanes are not like birds. Far from it. No flapping. Not composed of cells. Not biological in nature. On the other hand, there is another way to look at systems -- look at higher functional components and how they are connected. So maybe there is a language processing area connecting to memory. And so on. This is called the connectome of the brian as well. Which identifies what parts are connected to what. In this regards airplanes are similar to birds. They both have wings. Fuselage. A tail. They are built with similar structural material contraints -- light and durable. Aluminum, titanium for aircraft, and porous bones for birds. Another way to look at it is in so many decades of AI, we haven't yet come up with another model. So while having to wait for enlightment to hit us one day why not learn from an already existent example.

rl3 11y ago

In Superintelligence, Bostrom does not presume to know the path via which we will arrive at AGI. One hypothetical path discussed is that of tool AI. That is, robust search processes - things we are already quite adept at (genetic algorithms, deep learning, etc) - purposed towards AGI-related goals. It's not hard to imagine these existing methods being used in the pursuit of a recursively self-optimizing agent (seed AI) that then snowballs into AGI. Such an approach may not require any fundamental knowledge concerning the nature or architecture of AGI. It would simply be an application of brute computational force using existing tools and knowledge.

shock-value 11y ago
I do agree that current methods like deep learning's neural nets will end up being a part of some future general AI -- e.g. the convolutional deep nets used for image recognition are already reputedly similar to some aspects of human/animal biological vision systems. Yet even if all we need is this sort of "seed AI", we still need new architectural insights to be able to create it in the first place. Otherwise someone would have surely demonstrated it by now? If nothing else, such a system would need to evaluate effects of its outputs on its inputs over time (e.g. if I shoot a basketball, it takes seconds before it either goes in or doesn't; if I plant a seed in the ground, months will pass before it sprouts -- or not, depending on conditions). Research into recurrent networks, one possible avenue for doing this, is still pretty primitive. And I'm not sure I'm convinced that this core "seed AI" is sufficient to emulate human cognition. Such a system might effectively integrate audio and visual senses, for example (in order to combine both for prediction tasks), but could such a system ever emulate the sort of continuous verbal inner monologue we all have which narrates our experience? That we have this inner monologue which seems to run alongside our other senses but yet makes use of them (along with stored memories) suggests, at least to me, that some more complicated pathways are involved which link together these various "component systems" (senses, stored memories, emotional states, linguistic synthesis, etc.) beyond just the simple prediction/reward circuitry which I presume the "seed AI" would encapsulate.
- rl3 11y ago
  >Yet even if all we need is this sort of "seed AI", we still need new architectural insights to be able to create it in the first place. Not necessarily. That was my entire point, that robust search processes using existing tools and knowledge may yield a seed AI. >Otherwise someone would have surely demonstrated it by now? Again, not necessarily. AGI may not yet exist primarily due to dumb luck. The computational requirements, especially when you consider most computing capacity on the planet is networked, may already be adequate or even far exceed adequate. >And I'm not sure I'm convinced that this core "seed AI" is sufficient to emulate human cognition. It probably won't be. It will most likely be completely alien when compared to human cognition. At the same time, that doesn't preclude it from being vastly more powerful.