3 ms·

I recently read On Intelligence, very interesting take on the core principles of the brain. Perhaps someone more knowledgeable can comment on this, but it feels

I recently read On Intelligence, very interesting take on the core principles of the brain. Perhaps someone more knowledgeable can comment on this, but it feels like deep recurrent networks do a pretty good job of capturing the concepts that Jeff Hawkins considers really important: hierarchy/invariance and sequences/prediction. Plus they're really effective in practice!

computerex 9y ago
The concept Hawkins considers really important above all else is the biology. He rejects deep learning as being a fundamentally different form of computation than the one the brain performs. There are are no parallels to be drawn between RNN and HTM's, really. Apples and oranges. HTM's goal is literally to try and emulate the neocortex.
- etiam 9y ago
  "Emulate the neocortex" is at best a metaphoric shorthand that will be misunderstood by many, and quite likely the wrong way of thinking about it. It is certainly not to be taken literally. There are plenty of things going on in a real neocortex that are not essential to intelligence and Numenta was working hard from the start to pick out the important ones at an appropriate level of modeling. This distinction becomes especially important in contrast to things like Henry Markram's projects, where people seem to be happily spending major-multi-million-euro sums of public funds to run poorly understood simulations with insufficient detail on a really big computation cluster. RNN:s are crude compared to the HTM as well as to the more recent CLA model, but they do share for instance the formulation as a dynamical system with recurrence. Some of publications coming from people at Numenta have made direct comparison between the CLA to for instance LSTM-based networks, and in my opinion they are entirely reasonable to do that. They are all suited to sequence modelling. Having said that, I agree the comparison to RNN:s isn't to be made without understanding some of what goes into the HTM or CLA, and I agree that Hawkins emphasizes biological intelligence.

computerex 9y ago

The concept Hawkins considers really important above all else is the biology. He rejects deep learning as being a fundamentally different form of computation than the one the brain performs. There are are no parallels to be drawn between RNN and HTM's, really. Apples and oranges. HTM's goal is literally to try and emulate the neocortex.

etiam 9y ago
"Emulate the neocortex" is at best a metaphoric shorthand that will be misunderstood by many, and quite likely the wrong way of thinking about it. It is certainly not to be taken literally. There are plenty of things going on in a real neocortex that are not essential to intelligence and Numenta was working hard from the start to pick out the important ones at an appropriate level of modeling. This distinction becomes especially important in contrast to things like Henry Markram's projects, where people seem to be happily spending major-multi-million-euro sums of public funds to run poorly understood simulations with insufficient detail on a really big computation cluster. RNN:s are crude compared to the HTM as well as to the more recent CLA model, but they do share for instance the formulation as a dynamical system with recurrence. Some of publications coming from people at Numenta have made direct comparison between the CLA to for instance LSTM-based networks, and in my opinion they are entirely reasonable to do that. They are all suited to sequence modelling. Having said that, I agree the comparison to RNN:s isn't to be made without understanding some of what goes into the HTM or CLA, and I agree that Hawkins emphasizes biological intelligence.