3 ms·
"I am certain that HTM networks applied to applications like vision will not exhibit the problems talked about in this paper. Brains won’t either." Put up or s
by modeless 12y ago
"I am certain that HTM networks applied to applications like vision will not exhibit the problems talked about in this paper. Brains won’t either."
Put up or shut up, I say. If you have a model that works better, demonstrate it. ANNs simply crush every competitor in real-world performance.
Modeling brains is a cool idea but it doesn't have any practical advantages unless it improves learning performance or has medical applications. Neither is true of any ANN competitor so far.
Honestly, I think we're much more likely to eventually understand the brain by first focusing on building working AI systems of our own design and then comparing them to the brain (with the help of our AIs!), rather than reverse engineering the brain first and building systems that mimic it.
- sgk284 12y agoHe has "put up". He's been doing and funding neuroscience research for over a decade and has built some impressive predictive systems. Leading researchers, such as Andrew Ng (the Director of the Stanford Artificial Intelligence Lab), have stated that Jeff's publications have directly influenced their thinking. That said, I agree machine intelligence will initially be the equivalent of building machines that fly without flapping, but Jeff's approaches don't forbid this (he often deviates from nature in his models, but tries to stay close).
- modeless 12y ago> has built some impressive predictive systems Do you have any links to examples? I'm not aware of any systems built with HTMs that I'd consider "impressive" relative to what ANNs are doing these days, e.g. learning to play Atari games by watching the screen [1], approaching human performance on object classification in unrestricted images [2], or learning to execute simple python programs by reading the source code [3]. [1] https://medium.com/the-physics-arxiv-blog/the-last-ai-breakthrough-deepmind-made-before-google-bought-it-for-400m-7952031ee5e1 https://medium.com/the-physics-arxiv-blog/the-last-ai-breakt... [2] http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/ http://karpathy.github.io/2014/09/02/what-i-learned-from-com... [3] http://arxiv.org/abs/1410.4615 http://arxiv.org/abs/1410.4615
- sgk284 12y agoOh don't get me wrong, the things being done with ANNs are incredibly impressive. Far more so than anything I've seen with HTMs to date. I was just pointing out that Jeff is putting his money where his mouth is. Also, much of the resurgence in interest in AI came about 12 years ago in large part due to Jeff's thinking and writings (which was mostly consolidating existing ideas into a cohesive theory), as Andrew Ng and others have stated[1]. Jeff helped kick AI out of a plateau and avoid another AI winter. For interesting things that Numenta has done, just check their website[2]. Their most popular product is likely Grok[3], which predicts things like server loads. It's nothing that ANNs couldn't do, and ultimately isn't that impressive at face value, but the implementation is interesting and has far reaching implications (w/regards to things like energy efficiency and other areas). I think the future holds incredible things for ANNs and we'll continue to see great breakthroughs with them, but HTMs hold a lot of promise too. ANNs have been around quite a bit longer and have orders of magnitude more researchers working with them, so I'd argue it's premature to dismiss HTMs just because a handful of researchers in Redwood, CA haven't solved every AI problem within 12 years. It was only 15-20 years ago that ANNs were considered to have peaked and been surpassed by bayesian networks and support vector machines. How times have changed since then! [1] http://www.forbes.com/sites/roberthof/2014/08/28/interview-inside-google-brain-founder-andrew-ngs-plans-to-transform-baidu/ http://www.forbes.com/sites/roberthof/2014/08/28/interview-i... [2] http://numenta.com/ http://numenta.com/ [3] http://numenta.com/grok/ http://numenta.com/grok/
- deleted 12y ago[deleted]
- ezy 12y agoBeyond the Hawkins-driven hype, I do think, however that it is worth attempting to model the brain more closely in ML. It may be that, practically, the whole HTM thing doesn't really work out -- but it will be interesting to see how it fails or succeeds as a research project. Just like NNs in the 90's, there is this tendency to write it all off when some other technology does better, but I have a vague hope that at least some of the ideas will be more widely useful and that it's isn't a totally wasted effort.