4 ms·

Jeff Hawkins talk on modeling neocortex and its impact on machine intelligence

53 points by psawaya 16y ago

nswanberg 16y ago
Hawkins has also given some great talks on starting a company. He has an interesting take on entrepreneurship in that he views it as a tool of last resort to be used when you can't get something made within an existing organization: http://ecorner.stanford.edu/author/jeff_hawkins http://ecorner.stanford.edu/author/jeff_hawkins
TheEzEzz 16y ago
I watched the whole thing and like the philosophy of the approach. Unfortunately there isn't a single demonstration of the software, nor a direct comparison between the software and other machine learning algorithms. At one point Hawkins even says he's not going to waste time by talking about specific data sets they've applied the software to because there are too many! Then pick one! Pick the best!
- Parabola 16y ago
  Like all of machine learning, hierarchical temporal memories are overhyped and underdeliver. The field has a long way to go.
- hadronzoo 16y ago
  Vitamin D's software is built atop Numenta's platform: http://vitamindinc.com/demo.php http://vitamindinc.com/demo.php
  - ogrisel 16y ago
    AFAIK this is not based on the new architecture introduced in this talk.
- thijsterlouw 16y ago
  In the presentation it was mentioned they will release some software in 2011. He also mentioned there are papers + pseudo-code on the website for research purposes to play with. So for people really interested in it (and it does sound very interesting and promising), it should be possible to make those kind of demos/comparisons. Though I have to agree with you that one demo would have been nice.
icandoitbetter 16y ago
Jeff Hawkins began a few decades ago by criticizing all those neural-net researchers who were all promises and no results and (surprise!) he's become one of them now.
- sleight42 16y ago
  Apparently false. See the above comment: http://news.ycombinator.com/item?id=1945885 http://news.ycombinator.com/item?id=1945885
  - icandoitbetter 16y ago
    Oh-come-on. With the same logic, neural-nets were applied in a thousand and one domains. The problem of object tracking has been solved a thousand times in the past. Even if Numenta's algorithm is better at it than every other solution, you still can't call this a major result! Even Hawkins himself has admitted that his hierarchical network design hasn't caused the huge bang that he expected it to cause.
weeksie 16y ago
This was really fun to watch. I read On Intelligence, by Jeff Hawkins several years ago and wondered when or if Numenta was ever going to get off the ground. It looks like they're getting close to their goal. Yeah, there were no real demonstrations, but this looks a lot more promising than previous AI/Neural Network stuff.
brendano 16y ago
Have they compared their algorithm against standard well-known algorithms for the tasks they claim to solve? Last time I checked they still hadn't done that, or at least hadn't reported any such results. Without that, they're not worth anyone's time.
sleight42 16y ago
Or a YouTube video of a presentation from earlier this year: http://www.youtube.com/watch?v=TDzr0_fbnVk# http://www.youtube.com/watch?v=TDzr0_fbnVk#. Flash video is sadly unfriendly to my iPad (where I am now)
- SiVal 16y ago
  So, "Flash video is sadly unfriendly" to your welcoming little iPad? And the Mac needed its own proprietary browser, because others were so unfriendly. Talk about a reality distortion field....
cgs 16y ago
Coincidentally, I just finished On Intelligence, and I found it pretty mind-blowing. If you're interested in learning and the brain, read it! You can also read about the HTM algorithm they are working on here: http://www.numenta.com/htm-overview/education.php http://www.numenta.com/htm-overview/education.php According to the first paper, there's enough detail there for you to implement the algorithm yourself. Cool!
- possibilistic 16y ago
  Don't be fooled by approaches like this. HTM is an oversimplification that doesn't bring us any closer to real machine intelligence. I read the book a few years back when I was taking courses in machine learning and metaheuristics, and I recall being impressed. After picking up a molecular biology background, however, I've become skeptical of any claims to model "algorithms" after the brain or neuron or neocortex. Regardless of the level of abstraction chosen by the investigator, it isn't enough. To put it simply, I strongly feel that achieving any kind of biologically-inspired intelligent agent will require a systems biologic approach where we model every minute molecular detail in silico. This isn't an undertaking that we even have the technology for at present. We don't have the raw speed, level of parallelism, or even the molecular/cell physiologic details necessary to model even parts of the brain. (Even of Drosophila!) The Blue Brain project is nice and is worth following for nothing other than learning best engineering practices for developing the architecture behind something of this scale and complexity--but every simplification we make introduces error. (Imagine patching it! Imagine the "oops" moment, when some molecular mechanism doesn't work as we expected--and that's a regularly occurring event.) I'm not even sure how much simplification we can make before the emergent properties of the brain no longer function. Some of my colleagues say membrane potentials and the cytoskeletal system have key quantum interactions that encode state information--something we don't even understand yet. (I can't comment much on that, since I haven't studied quantum physics.) I'm actually learning to develop algorithms that will focus on the interplay of the genomic machinery (promoters/enhancers, tx, translation, chaperones/folding, etc), biochemical pathways and kinetics, concentration levels, receptors, etc. in the hopes that one day we will be able to model systems like the brain. But from my limited knowledge, a project on the brain scale will only succeed after we solve the "much less complicated" problems: cancer, alzheimers, and aging, all of which are all cell-level problems. That's where we have to focus at present--and you can see how much more remains to be done.
evolvingstuff 16y ago
Very interesting video, these concepts seemed to have progressed a lot since last I checked in on them. That being said, I'm finding it very difficult to find any objective comparisons of these algorithms to other, more mainstream machine learning techniques. In the talk, he gave the impression that there was a tremendous amount of data to back up these claims, and that he just didn't have time to present it all. I went through many of the white papers available on the Numenta website. Many were just overall outlines of the approach. A few of them demonstrate tasks for which some form of learning is occurring, however, it was hard for me to know, in the absence of objective comparisons to other techniques, just how good the results really are. So far, the only objective comparison I could find involved handwritten character recognition, and that was against what appeared to be a standard feed forward neural network with only a single hidden layer. Not exactly state of the art.. why not compare to SVMs, convolutional NNs, deep belief nets, etc..? So I am at this point hopeful, but fairly skeptical. If nothing else these are some inspiring ideas.

53 points by psawaya 16y ago

Hawkins has also given some great talks on starting a company. He has an interesting take on entrepreneurship in that he views it as a tool of last resort to be used when you can't get something made within an existing organization: http://ecorner.stanford.edu/author/jeff_hawkins http://ecorner.stanford.edu/author/jeff_hawkins

TheEzEzz 16y ago

I watched the whole thing and like the philosophy of the approach. Unfortunately there isn't a single demonstration of the software, nor a direct comparison between the software and other machine learning algorithms. At one point Hawkins even says he's not going to waste time by talking about specific data sets they've applied the software to because there are too many! Then pick one! Pick the best!

Parabola 16y ago
Like all of machine learning, hierarchical temporal memories are overhyped and underdeliver. The field has a long way to go.
hadronzoo 16y ago
Vitamin D's software is built atop Numenta's platform: http://vitamindinc.com/demo.php http://vitamindinc.com/demo.php
- ogrisel 16y ago
  AFAIK this is not based on the new architecture introduced in this talk.
thijsterlouw 16y ago
In the presentation it was mentioned they will release some software in 2011. He also mentioned there are papers + pseudo-code on the website for research purposes to play with. So for people really interested in it (and it does sound very interesting and promising), it should be possible to make those kind of demos/comparisons. Though I have to agree with you that one demo would have been nice.

icandoitbetter 16y ago

Jeff Hawkins began a few decades ago by criticizing all those neural-net researchers who were all promises and no results and (surprise!) he's become one of them now.

sleight42 16y ago
Apparently false. See the above comment: http://news.ycombinator.com/item?id=1945885 http://news.ycombinator.com/item?id=1945885
- icandoitbetter 16y ago
  Oh-come-on. With the same logic, neural-nets were applied in a thousand and one domains. The problem of object tracking has been solved a thousand times in the past. Even if Numenta's algorithm is better at it than every other solution, you still can't call this a major result! Even Hawkins himself has admitted that his hierarchical network design hasn't caused the huge bang that he expected it to cause.

weeksie 16y ago

This was really fun to watch. I read On Intelligence, by Jeff Hawkins several years ago and wondered when or if Numenta was ever going to get off the ground. It looks like they're getting close to their goal. Yeah, there were no real demonstrations, but this looks a lot more promising than previous AI/Neural Network stuff.

brendano 16y ago

Have they compared their algorithm against standard well-known algorithms for the tasks they claim to solve? Last time I checked they still hadn't done that, or at least hadn't reported any such results. Without that, they're not worth anyone's time.

sleight42 16y ago

Or a YouTube video of a presentation from earlier this year: http://www.youtube.com/watch?v=TDzr0_fbnVk# http://www.youtube.com/watch?v=TDzr0_fbnVk#. Flash video is sadly unfriendly to my iPad (where I am now)

SiVal 16y ago
So, "Flash video is sadly unfriendly" to your welcoming little iPad? And the Mac needed its own proprietary browser, because others were so unfriendly. Talk about a reality distortion field....

cgs 16y ago

Coincidentally, I just finished On Intelligence, and I found it pretty mind-blowing. If you're interested in learning and the brain, read it! You can also read about the HTM algorithm they are working on here: http://www.numenta.com/htm-overview/education.php http://www.numenta.com/htm-overview/education.php According to the first paper, there's enough detail there for you to implement the algorithm yourself. Cool!

possibilistic 16y ago
Don't be fooled by approaches like this. HTM is an oversimplification that doesn't bring us any closer to real machine intelligence. I read the book a few years back when I was taking courses in machine learning and metaheuristics, and I recall being impressed. After picking up a molecular biology background, however, I've become skeptical of any claims to model "algorithms" after the brain or neuron or neocortex. Regardless of the level of abstraction chosen by the investigator, it isn't enough. To put it simply, I strongly feel that achieving any kind of biologically-inspired intelligent agent will require a systems biologic approach where we model every minute molecular detail in silico. This isn't an undertaking that we even have the technology for at present. We don't have the raw speed, level of parallelism, or even the molecular/cell physiologic details necessary to model even parts of the brain. (Even of Drosophila!) The Blue Brain project is nice and is worth following for nothing other than learning best engineering practices for developing the architecture behind something of this scale and complexity--but every simplification we make introduces error. (Imagine patching it! Imagine the "oops" moment, when some molecular mechanism doesn't work as we expected--and that's a regularly occurring event.) I'm not even sure how much simplification we can make before the emergent properties of the brain no longer function. Some of my colleagues say membrane potentials and the cytoskeletal system have key quantum interactions that encode state information--something we don't even understand yet. (I can't comment much on that, since I haven't studied quantum physics.) I'm actually learning to develop algorithms that will focus on the interplay of the genomic machinery (promoters/enhancers, tx, translation, chaperones/folding, etc), biochemical pathways and kinetics, concentration levels, receptors, etc. in the hopes that one day we will be able to model systems like the brain. But from my limited knowledge, a project on the brain scale will only succeed after we solve the "much less complicated" problems: cancer, alzheimers, and aging, all of which are all cell-level problems. That's where we have to focus at present--and you can see how much more remains to be done.

evolvingstuff 16y ago

Very interesting video, these concepts seemed to have progressed a lot since last I checked in on them. That being said, I'm finding it very difficult to find any objective comparisons of these algorithms to other, more mainstream machine learning techniques. In the talk, he gave the impression that there was a tremendous amount of data to back up these claims, and that he just didn't have time to present it all. I went through many of the white papers available on the Numenta website. Many were just overall outlines of the approach. A few of them demonstrate tasks for which some form of learning is occurring, however, it was hard for me to know, in the absence of objective comparisons to other techniques, just how good the results really are. So far, the only objective comparison I could find involved handwritten character recognition, and that was against what appeared to be a standard feed forward neural network with only a single hidden layer. Not exactly state of the art.. why not compare to SVMs, convolutional NNs, deep belief nets, etc..? So I am at this point hopeful, but fairly skeptical. If nothing else these are some inspiring ideas.