3 ms·

For those interested, Jeff Hawkins wrote a book called On Intelligence and he is largely the guy behind: https://en.wikipedia.org/wiki/Hierarchical_temporal_mem

For those interested, Jeff Hawkins wrote a book called On Intelligence and he is largely the guy behind: https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://en.wikipedia.org/wiki/Hierarchical_temporal_memory Deep learning proponents focus on solving specific problems using mathematical models inspired by biology, whereas HTM proponents argue that biology is important and it should play a more central role. Deep learning folks are more geared towards applied AI, whereas HTM's are way more ambitious and are trying to solve the problem of intelligence/AGI. Numenta, the company behind HTM's has a platform called NuPic for "intelligent computing": https://github.com/numenta/nupic https://github.com/numenta/nupic But as far as I know HTM's have never been applied to anything non-trivial successfully.

paladin314159 9y ago
I recently read On Intelligence, very interesting take on the core principles of the brain. Perhaps someone more knowledgeable can comment on this, but it feels like deep recurrent networks do a pretty good job of capturing the concepts that Jeff Hawkins considers really important: hierarchy/invariance and sequences/prediction. Plus they're really effective in practice!
- computerex 9y ago
  The concept Hawkins considers really important above all else is the biology. He rejects deep learning as being a fundamentally different form of computation than the one the brain performs. There are are no parallels to be drawn between RNN and HTM's, really. Apples and oranges. HTM's goal is literally to try and emulate the neocortex.
  - etiam 9y ago
    "Emulate the neocortex" is at best a metaphoric shorthand that will be misunderstood by many, and quite likely the wrong way of thinking about it. It is certainly not to be taken literally. There are plenty of things going on in a real neocortex that are not essential to intelligence and Numenta was working hard from the start to pick out the important ones at an appropriate level of modeling. This distinction becomes especially important in contrast to things like Henry Markram's projects, where people seem to be happily spending major-multi-million-euro sums of public funds to run poorly understood simulations with insufficient detail on a really big computation cluster. RNN:s are crude compared to the HTM as well as to the more recent CLA model, but they do share for instance the formulation as a dynamical system with recurrence. Some of publications coming from people at Numenta have made direct comparison between the CLA to for instance LSTM-based networks, and in my opinion they are entirely reasonable to do that. They are all suited to sequence modelling. Having said that, I agree the comparison to RNN:s isn't to be made without understanding some of what goes into the HTM or CLA, and I agree that Hawkins emphasizes biological intelligence.
ghthor 9y ago
They built a infrastructure monitor using the HTM https://grokstream.com https://grokstream.com Also http://www.cortical.io http://www.cortical.io is pretty dann cool.

paladin314159 9y ago

I recently read On Intelligence, very interesting take on the core principles of the brain. Perhaps someone more knowledgeable can comment on this, but it feels like deep recurrent networks do a pretty good job of capturing the concepts that Jeff Hawkins considers really important: hierarchy/invariance and sequences/prediction. Plus they're really effective in practice!

computerex 9y ago
The concept Hawkins considers really important above all else is the biology. He rejects deep learning as being a fundamentally different form of computation than the one the brain performs. There are are no parallels to be drawn between RNN and HTM's, really. Apples and oranges. HTM's goal is literally to try and emulate the neocortex.
- etiam 9y ago
  "Emulate the neocortex" is at best a metaphoric shorthand that will be misunderstood by many, and quite likely the wrong way of thinking about it. It is certainly not to be taken literally. There are plenty of things going on in a real neocortex that are not essential to intelligence and Numenta was working hard from the start to pick out the important ones at an appropriate level of modeling. This distinction becomes especially important in contrast to things like Henry Markram's projects, where people seem to be happily spending major-multi-million-euro sums of public funds to run poorly understood simulations with insufficient detail on a really big computation cluster. RNN:s are crude compared to the HTM as well as to the more recent CLA model, but they do share for instance the formulation as a dynamical system with recurrence. Some of publications coming from people at Numenta have made direct comparison between the CLA to for instance LSTM-based networks, and in my opinion they are entirely reasonable to do that. They are all suited to sequence modelling. Having said that, I agree the comparison to RNN:s isn't to be made without understanding some of what goes into the HTM or CLA, and I agree that Hawkins emphasizes biological intelligence.

ghthor 9y ago

They built a infrastructure monitor using the HTM https://grokstream.com https://grokstream.com Also http://www.cortical.io http://www.cortical.io is pretty dann cool.