6 ms·

Numenta's based upon the hierarchical temporal memory model described in the book On Intelligence. It's an attempt to replicate the structure of the neocortex.

by s_q_b 12y ago

Numenta's based upon the hierarchical temporal memory model described in the book On Intelligence. It's an attempt to replicate the structure of the neocortex. Mathematically it appears to be just another neural network. IMHO, deep learning is probably more likely to give systems approximating general intelligence.

chestervonwinch 12y ago
> Mathematically it appears to be just another neural network. What's the difference, broadly speaking? > IMHO, deep learning is probably more likely to give systems approximating general intelligence. If HTMs are just another neural net, why do you suspect deep learning will excel comparatively? I don't know the first thing about HTMs - just curious.
- c54 12y ago
  HTMs are biologically accurate -- they do the same thing as the neurons in our neocortex! The most exciting aspect of Numenta's research is that they are building algorithms which think the way we do (albeit in early stages right now).
  - dj-wonk 12y ago
    Re: "HTMs are biologically accurate": It depends on what you mean by accurate. Hawkins argues that HTMs capture the systems-level effect of a brain, even if software models of neurons are not the same as biological neurons. I can't personally comment on what others say about the accuracy, even at the systems-level. Re: "The most exciting aspect of Numenta's research is that they are building algorithms which think the way we do (albeit in early stages right now)." It is a stretch to use the word "think" in this way, since thinking suggests consciousness. One wouldn't say we "think" when we simply observe. You might argue that HTMs pass signals in an analogous way -- that might be a fair statement, from Hawkins' point of view. Yes, I realize that Hawkins says some interesting things about 'forward' and 'backward' processing even for 'low-level' processing such as edge detection.
31reasons 12y ago
Can deep learning algorithms learn from temporal data ? So far I have seen object recognition from a single image but not some intelligence gathering from a video.
- Houshalter 12y ago
  Yes, they are called recurrent neural networks. The disadvantage is higher memory and time requirements. You need to store the state of the neural network at every time step, and then at the end iterate through the whole process in reverse. Whereas best I understand it, HTM is online and updates the weights every time step without needing to keep a running memory of the states it's been in.
yogthos 12y ago
That's not really correct actually. A better way to look at it is that each neuron in the HTM is itself effectively a deep neural network. The HTM is a temporally aware hierarchy of these neurons.
saraid216 12y ago
> described in the book On Intelligence Said book spends a good deal of time on criticizing neural networks, FYI.
- s_q_b 12y ago
  Of course I read it. Fascinating concept, but I'm unimpressed by Numenta's execution. Certainly the ideas of a single neocortical algorithm, a temporal feedback loop minimizing error, the relationship between the hippocampus and the rest of the memory-learn-predict-observe-correct cycle are inspired.

by s_q_b 12y ago

chestervonwinch 12y ago

> Mathematically it appears to be just another neural network. What's the difference, broadly speaking? > IMHO, deep learning is probably more likely to give systems approximating general intelligence. If HTMs are just another neural net, why do you suspect deep learning will excel comparatively? I don't know the first thing about HTMs - just curious.

c54 12y ago
HTMs are biologically accurate -- they do the same thing as the neurons in our neocortex! The most exciting aspect of Numenta's research is that they are building algorithms which think the way we do (albeit in early stages right now).
- dj-wonk 12y ago
  Re: "HTMs are biologically accurate": It depends on what you mean by accurate. Hawkins argues that HTMs capture the systems-level effect of a brain, even if software models of neurons are not the same as biological neurons. I can't personally comment on what others say about the accuracy, even at the systems-level. Re: "The most exciting aspect of Numenta's research is that they are building algorithms which think the way we do (albeit in early stages right now)." It is a stretch to use the word "think" in this way, since thinking suggests consciousness. One wouldn't say we "think" when we simply observe. You might argue that HTMs pass signals in an analogous way -- that might be a fair statement, from Hawkins' point of view. Yes, I realize that Hawkins says some interesting things about 'forward' and 'backward' processing even for 'low-level' processing such as edge detection.

31reasons 12y ago

Can deep learning algorithms learn from temporal data ? So far I have seen object recognition from a single image but not some intelligence gathering from a video.

Houshalter 12y ago
Yes, they are called recurrent neural networks. The disadvantage is higher memory and time requirements. You need to store the state of the neural network at every time step, and then at the end iterate through the whole process in reverse. Whereas best I understand it, HTM is online and updates the weights every time step without needing to keep a running memory of the states it's been in.

yogthos 12y ago

That's not really correct actually. A better way to look at it is that each neuron in the HTM is itself effectively a deep neural network. The HTM is a temporally aware hierarchy of these neurons.

saraid216 12y ago

> described in the book On Intelligence Said book spends a good deal of time on criticizing neural networks, FYI.

s_q_b 12y ago
Of course I read it. Fascinating concept, but I'm unimpressed by Numenta's execution. Certainly the ideas of a single neocortical algorithm, a temporal feedback loop minimizing error, the relationship between the hippocampus and the rest of the memory-learn-predict-observe-correct cycle are inspired.