4 ms·

Coincidentally, I just finished On Intelligence, and I found it pretty mind-blowing. If you're interested in learning and the brain, read it! You can also read

by cgs 16y ago

Coincidentally, I just finished On Intelligence, and I found it pretty mind-blowing. If you're interested in learning and the brain, read it! You can also read about the HTM algorithm they are working on here: http://www.numenta.com/htm-overview/education.php http://www.numenta.com/htm-overview/education.php According to the first paper, there's enough detail there for you to implement the algorithm yourself. Cool!

possibilistic 16y ago
Don't be fooled by approaches like this. HTM is an oversimplification that doesn't bring us any closer to real machine intelligence. I read the book a few years back when I was taking courses in machine learning and metaheuristics, and I recall being impressed. After picking up a molecular biology background, however, I've become skeptical of any claims to model "algorithms" after the brain or neuron or neocortex. Regardless of the level of abstraction chosen by the investigator, it isn't enough. To put it simply, I strongly feel that achieving any kind of biologically-inspired intelligent agent will require a systems biologic approach where we model every minute molecular detail in silico. This isn't an undertaking that we even have the technology for at present. We don't have the raw speed, level of parallelism, or even the molecular/cell physiologic details necessary to model even parts of the brain. (Even of Drosophila!) The Blue Brain project is nice and is worth following for nothing other than learning best engineering practices for developing the architecture behind something of this scale and complexity--but every simplification we make introduces error. (Imagine patching it! Imagine the "oops" moment, when some molecular mechanism doesn't work as we expected--and that's a regularly occurring event.) I'm not even sure how much simplification we can make before the emergent properties of the brain no longer function. Some of my colleagues say membrane potentials and the cytoskeletal system have key quantum interactions that encode state information--something we don't even understand yet. (I can't comment much on that, since I haven't studied quantum physics.) I'm actually learning to develop algorithms that will focus on the interplay of the genomic machinery (promoters/enhancers, tx, translation, chaperones/folding, etc), biochemical pathways and kinetics, concentration levels, receptors, etc. in the hopes that one day we will be able to model systems like the brain. But from my limited knowledge, a project on the brain scale will only succeed after we solve the "much less complicated" problems: cancer, alzheimers, and aging, all of which are all cell-level problems. That's where we have to focus at present--and you can see how much more remains to be done.
- StavrosK 16y ago
  I completely agree with you. I asked one of my professors at my Machine Learning masters course about this, and he said "if I've never heard of it, how good can it be?" Algorithms that replicate biological processes are popular because people can easily grasp them. Anyone can understand why evolutionary systems work, or why neural networks work (because it's in nature, dummy!), but they don't work as well as other, purely mathematical methods. In the end, these sorts of things tend to be toys or marginally useful, where other, more mathematically sound algorithms dominate the landscape. I got very excited about HTMs too, when I didn't know as much about ML as I do now, but I've realised that it hasn't made even a dent in academic circles (you know, the ones with the thousands of people who study these things for a living).
  - Dn_Ab 16y ago
    I also agree but while neural networks are a sort of black box they are also entirely mathematical. Also some of the most impressive current research is in the area of deep learners, an example of which is a type of neural network: RBMs and Deep Belief Nets. In particular Deep Belief Nets have the added advantaged of being able to display their interal abstractions and state and so are not so boxed up.
    - StavrosK 16y ago
      I will concede that, but they're a very roundabout way to building a classifier. An SVM, for example, is really much simpler and works much, much better than a neural network. I'm not familiar with RBMs and DBNs, so I can't comment on that, sadly...
      - tgflynn 16y ago
        On very large real world data sets my experience has been the opposite.
      - Dn_Ab 16y ago
        I would remove one maybe both muches. I agree that in general SVMs are better but they are also every bit as black box. Deep belief networks though, are on another level. literally, svms are depth 2, dbn's are depth unbounded, i think. http://www.youtube.com/watch?v=VdIURAu1-aU http://www.youtube.com/watch?v=VdIURAu1-aU

by cgs 16y ago

possibilistic 16y ago

Don't be fooled by approaches like this. HTM is an oversimplification that doesn't bring us any closer to real machine intelligence. I read the book a few years back when I was taking courses in machine learning and metaheuristics, and I recall being impressed. After picking up a molecular biology background, however, I've become skeptical of any claims to model "algorithms" after the brain or neuron or neocortex. Regardless of the level of abstraction chosen by the investigator, it isn't enough. To put it simply, I strongly feel that achieving any kind of biologically-inspired intelligent agent will require a systems biologic approach where we model every minute molecular detail in silico. This isn't an undertaking that we even have the technology for at present. We don't have the raw speed, level of parallelism, or even the molecular/cell physiologic details necessary to model even parts of the brain. (Even of Drosophila!) The Blue Brain project is nice and is worth following for nothing other than learning best engineering practices for developing the architecture behind something of this scale and complexity--but every simplification we make introduces error. (Imagine patching it! Imagine the "oops" moment, when some molecular mechanism doesn't work as we expected--and that's a regularly occurring event.) I'm not even sure how much simplification we can make before the emergent properties of the brain no longer function. Some of my colleagues say membrane potentials and the cytoskeletal system have key quantum interactions that encode state information--something we don't even understand yet. (I can't comment much on that, since I haven't studied quantum physics.) I'm actually learning to develop algorithms that will focus on the interplay of the genomic machinery (promoters/enhancers, tx, translation, chaperones/folding, etc), biochemical pathways and kinetics, concentration levels, receptors, etc. in the hopes that one day we will be able to model systems like the brain. But from my limited knowledge, a project on the brain scale will only succeed after we solve the "much less complicated" problems: cancer, alzheimers, and aging, all of which are all cell-level problems. That's where we have to focus at present--and you can see how much more remains to be done.

StavrosK 16y ago
I completely agree with you. I asked one of my professors at my Machine Learning masters course about this, and he said "if I've never heard of it, how good can it be?" Algorithms that replicate biological processes are popular because people can easily grasp them. Anyone can understand why evolutionary systems work, or why neural networks work (because it's in nature, dummy!), but they don't work as well as other, purely mathematical methods. In the end, these sorts of things tend to be toys or marginally useful, where other, more mathematically sound algorithms dominate the landscape. I got very excited about HTMs too, when I didn't know as much about ML as I do now, but I've realised that it hasn't made even a dent in academic circles (you know, the ones with the thousands of people who study these things for a living).
- Dn_Ab 16y ago
  I also agree but while neural networks are a sort of black box they are also entirely mathematical. Also some of the most impressive current research is in the area of deep learners, an example of which is a type of neural network: RBMs and Deep Belief Nets. In particular Deep Belief Nets have the added advantaged of being able to display their interal abstractions and state and so are not so boxed up.
  - StavrosK 16y ago
    I will concede that, but they're a very roundabout way to building a classifier. An SVM, for example, is really much simpler and works much, much better than a neural network. I'm not familiar with RBMs and DBNs, so I can't comment on that, sadly...
    - tgflynn 16y ago
      On very large real world data sets my experience has been the opposite.
    - Dn_Ab 16y ago
      I would remove one maybe both muches. I agree that in general SVMs are better but they are also every bit as black box. Deep belief networks though, are on another level. literally, svms are depth 2, dbn's are depth unbounded, i think. http://www.youtube.com/watch?v=VdIURAu1-aU http://www.youtube.com/watch?v=VdIURAu1-aU