16 ms·
Jeff Hawkins: Thousand Brains Theory of Intelligence [video]
- jcims 7y agoJust a side note, Lex has been killing it with this podcast. He’s been able to land some remarkable guests for a relatively new show with a niche market. I personally think he does a great job of mixing up the conversation with both technical and philosophical angles. Highly recommended.
- nabla9 7y agoMy favorite podcasts are Lex Friedman and Sean Carroll. Anyone, please feel free to suggest other similar ones.
- espeed 7y agoEric Weinstein: Revolutionary Ideas in Science, Math, and Society https://www.youtube.com/watch?v=2wq9x2QcZN0 https://www.youtube.com/watch?v=2wq9x2QcZN0 https://lexfridman.com/eric-weinstein/ https://lexfridman.com/eric-weinstein/ Discussion: https://news.ycombinator.com/item?id=19458503 https://news.ycombinator.com/item?id=19458503
- nabla9 7y agoEric Weinstein one was great. What I meant is other podcast channels.
- c1b 7y agoNot too similar but Sam Harris has some very good content on his podcast.
- FeeJai 7y agoYeah. It is impressive how he manages to self-market and how different he is on various media channels to his real-life personality
- mark_l_watson 7y agoAbsolutely! His interviews are my favorite recurring podcasts. A distant second, but still very good, is Azeem Ahzar’s Exponential View podcasts.
- melling 7y agoLearning a lot about the neocortex. Any recommendations on how to learn more? @7:45 It’s the size of a dinner napkin, 2.5mm think, uniform, and similar in other animals. @9:50 if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. @34:30 Thousand brains theory of intelligence.
- phreeza 7y agoNote that the "uniform" part, on which the second part ist also conditioned, ist not universally accepted. There are certainly shared elements, but most likely also specialisations.
- gnode 7y agoLearn about the rest of the brain's components. Intelligence and cognition is greatly dependent on the limbic system. For example, while memory appears to be stored throughout the cortex, we can't form new spatial and event memories if our hippocampi are removed. It seems to me that the cortex functions like a kind of FPGA which the brain's core machinery uses to expand its capabilities. It has a uniform micro-anatomy, regardless of function; functions correlate with connectivity to nerves and other brain regions; in cases of damage to the cortex, it is capable of remapping and rerouting functions and connectivity.
- Balgair 7y ago> if you took the optic nerve and attached it to another part of the neocortex, that part would become the visual region. FYI, that is super hand-wavey and covers over a lot about how the path of the information from the cones/rods gets into V1. The chain of neurons that pass infomation from your eyes to V1 is well studied [0]. Interruptions in that path cause a lot of sightedness issues and are not fun diseases to have. The musician, Stevie Wonder, among others, aledgedly has a form of blindness known as blindsight [1] where relfexes to motion are perserved, but information is not passed into the conscious mind. In the end, though neuroscience is a facinating subject, we're just in the beginning of our understanding of the brain. More research is needed. [0] https://en.wikipedia.org/wiki/Optic_chiasm https://en.wikipedia.org/wiki/Optic_chiasm a good place to start learning about the chain of information transfer. [1] https://en.wikipedia.org/wiki/Blindsight https://en.wikipedia.org/wiki/Blindsight EDIT: Additionally, if you want to learn more about neuroscience, the best place to look is at Kandel's Principals of Neural Science [2]. It is a tome of a book, but is the best place to get a deep dive into the brain and our understanding of it. I've not yet seen anything else that is somewhat accessible to the general public but also gets into all the issues with any particular experiment. Most pop-sci book brush over a lot of the very important and thorny issues that each experiment has. I'd also love to know of a good book that is more accessible than Kandel. [2] https://www.amazon.com/Principles-Neural-Science-Fifth-Kandel/dp/0071390111 https://www.amazon.com/Principles-Neural-Science-Fifth-Kande...
- devoply 7y agoOnce intelligence is "solved", most humans are completely and utterly screwed.
- UnpossibleJim 7y agoThere's the "Wally vs. Star Trek" theory of Artificial Intelligence (I know it sounds goofy, but it makes a difference when looking towards the direction of AI). If you look at the direction AI will take, science fiction is really all we have to go on and we have three platforms: Terminator - To be avoided at all costs (everyone agrees) Wally - AI does its best to "take care" of us, turning us into infantile, unthinking, pleasure seeking dweebs. The ultimate basal brain utopia. But empty. Star Trek (TNG) - AI as an assistant to help us fulfill our higher purpose and make us better at being a higher order being. There is no doubt that the ship's computer has advanced AI if you look at the questions posed to it by the engineers and the crew, but it does not impose its "will" upon them. It is an observer until directly asked, even when death is at hand (seemingly). There is a very good video on YouTube about it, but I can't find it (right now) that explains it in a very accessible way (that's pretty entertaining) with no real knowledge of the underlying functionality necessary. I suggest you watch it. It's great! EDIT: https://www.youtube.com/watch?v=48mf2QUtUmg&t=949s https://www.youtube.com/watch?v=48mf2QUtUmg&t=949s
- ruskerdax 7y agoThat does sound like an interesting video, and I'd like to see it if you find it (or I might search for it later). As for the Star Trek (TNG) interpretation of AGI, I suspect something like this in the real world would give rise to a kind of "oracle" situation, which creates its own dangerous problems. A system which functions as described only by observing and giving input where asked still has the potential to affect the outcome of events simply by being a reliable predictor. A paper explaining this concept far better than I would be able to in a HN comment is available here: http://www.aleph.se/papers/oracleAI.pdf http://www.aleph.se/papers/oracleAI.pdf
- quirkot 7y agoIt's worth noting that the Wally and Star Trek interpretations are not incompatible. What are the trillions of people that can't qualify to be a red shirt doing? Probably zonked out in a holodeck
- monkeydust 7y agoBeen following hawkins and Numenta for a while. They have never really taken off in commercial applications as far as I know despite some sound reasoning behind their theories. Hopefully they will make an impactful breakthrough soon.
- KingFelix 7y agosame, I have been following before they changed the name to Numenta. Interesting stuff, will listen to this podcast soon
- bhouston 7y agoJeff has been at this for a really long time now. If it is this hard to make progress, it is likely basic research that just requires a hard slog and also may not turn out in the end as all basic research is risky. My default bet then is against him because most basic research fails, with a small chance of success.
- tolstoy77 7y agoGot introduced to Jeff after watching him keynote a conference about a decade ago. He's a great speaker. Also thought they had some interesting tech when it came to computer vision at the time. Sadly it never took off and then ConvNets blew whatever Numenta was doing out of the water. Surprised even with someone like Jeff that Numenta hasn't had any commercial success.
- RosanaAnaDana 7y agoI'm like 50/50 on whether or not Lex is himself an artificial intelligence.
- TaupeRanger 7y agoJeff and Subutai have been on this hobby horse for a very long time. They don't have much to show for it, and many of their ideas are controversial even within neuroscience (for example, some think that their venerated "cortical columns" are structures without meaningful functions). It is very easy to bet against them given the fact that nothing much has happened, and "theories" about the brain are a dime-a-dozen. I'm glad that they have the gumption to go all-in on these ideas, but I'm not expecting anything earth shattering anytime soon.
- nabla9 7y agoThat's the blessing/curse with completely self funded research group. No strict timelines, yearly review, benchmarks or external pressure. There is no hurry to deliver, you can dream on and keep developing your pet idea indefinitely. Nobody can tell you that you were wrong and you are now done.
- tux1968 7y agoThat explains why they can continue, but it doesn't make it more likely that the idea will ever bare fruit.
- p1esk 7y agoNo strict timelines, yearly review, benchmarks or external pressure. Maybe not imposed on Jeff himself, at least not in this narrow sense, but the people who work for him most likely have all that.
- espeed 7y agoUnderstanding the underlying structure is key. And the significance of structure -- the ability to see it, understand it, and re-create it -- has been recognized and pointed out by Feynman [1] and some of the world's best programmers [2]. Using data-structure representations that best match reality's underlying structure may be key too. With this in mind I had a thought -- a "dopey idea" in the Steve Jobs to Jony Ive sense of the word [3] -- the brain doesn't do backprop [4] -- at least not directly (as far as we know) -- but neural nets using backprop/gradient-descent work pretty well. Why so? What if the underlying structure these nets are modeling isn't the brain but something else...maybe something more fundamental...like the curvature of spacetime, a path integral representation of information, or some other crude representation of an underlying quantum structure. And if so, what if learning is just the brain optimizing itself in accordance with this structure, and has this idea been explored before? Evidently it has. The idea of the brain as a quantum information processor has been explored to some degree in holonomic brain theory [5]. [1] "What I cannot create, I do not understand" https://en.wikiquote.org/wiki/Richard_Feynman https://en.wikiquote.org/wiki/Richard_Feynman [2] Programmers on Data Structures https://news.ycombinator.com/item?id=20047607 https://news.ycombinator.com/item?id=20047607 [3] Jony Ive talking about Steve Jobs' Dopey Ideas [video] https://www.youtube.com/watch?v=4WNYYCX3Tg0 https://www.youtube.com/watch?v=4WNYYCX3Tg0 [4] Artificial intelligence pioneer says we need to start over https://www.axios.com/artificial-intelligence-pioneer-says-we-need-to-start-over-1513305524-f619efbd-9db0-4947-a9b2-7a4c310a28fe.html https://www.axios.com/artificial-intelligence-pioneer-says-w... [5] Holonomic Brain Theory https://en.wikipedia.org/wiki/Holonomic_brain_theory https://en.wikipedia.org/wiki/Holonomic_brain_theory
- sysbin 7y agoCan anyone clarify Jeff Hawkins usage of the word time? I'm wondering why he isn't using rate or frequency instead of time.
- p1esk 7y ago“Time” as in “predicting the future”. Same as in RNNs.
- yters 7y agoNo fundamental difference between centralized and decentralized computation, even if you throw in stochasticity. It is all reducible to a deterministic Turing machine.
- canistel 7y agoDileep George, one of the original co-founders of Numenta have since moved on to co-found Vicarious.They were in the news after breaking captcha, that too with commonplace hardware. They are far too secretive, but there is reason to believe that they too are part of the HTM legacy...
- p1esk 7y agoThey publish occasional DL papers, but results are not impressive, so mainstream DL people ignore them.
- raindeer3 7y agoThe don't do deep learning. Their papers are based on graphical models. Their RCN model can be said to be inspired by Numentas earlier HTM model that dileep developed there, but is very different in fundamental ways.
- p1esk 7y agoYeah they do, here are 3 papers on convnets: https://arxiv.org/abs/1611.02252 https://arxiv.org/abs/1611.02252 https://arxiv.org/abs/1611.02767 https://arxiv.org/abs/1611.02767 https://arxiv.org/abs/1706.04313 https://arxiv.org/abs/1706.04313 But it seems like lately they focus more on RL for robotics.
- gnode 7y agoI think the cortex is attractive to AI research because its highly uniform micro-anatomy suggests there is a simple algorithm behind general intelligence to be found within. While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. Additionally, animals lacking a cortex, like birds, are still capable of many intelligent behaviours typically regarded as cortical. I wonder whether the cortex may be a kind of FPGA wired up by the limbic system, not naturally containing the machinery for general learning, and a red herring to the pursuit of general intelligence.
- daphneokeefe 7y agoFor people unfamiliar with the acronym (from Wikipedia): "A field-programmable gate array (FPGA) is an integrated circuit designed to be configured by a customer or a designer after manufacturing..."
- deleted 7y ago[deleted]
- derefr 7y ago> While intelligent behaviours appear to reside in the cortex, the limbic system is heavily involved in learning and cognition generally. We understand the limbic system to process motivation and emotion; could the limbic system's activation during cognition not be "explained away" by it being recruited to process the emotional or motivational aspects of sensory or memory data? (Just like how there is visual cortex recruitment when using visual imagination, etc.)
- howlin 7y agoIt's important to recognize that the cortex is a relatively late addition to the structure of the brain. Brains have been learning about the world and how to behave appropriately long before some mammals developed such a large region here. It's much more likely that the core functionality to be an autonomous agent in the world is encoded in the limbic system (e.g. basal ganglia, hypothalamus, mid brain and brain stem regions).
- thrwaway3742 7y agoIs there a transcript somewhere? (It's 2 hours 9 minutes)
- mr__y 7y agoyou can increase playback speed. 1.1..1.25x speed is usually OK in terms of ability to understand the speech while saving several percent in time. Sure, the transcript would probably require significantly less time to read and is my usual preferred way to consume content, but when none is available the faster playback allows for saving at least a fraction of time.
- tim333 7y agoThere's a youtube generated one if you go to https://www.youtube.com/watch?v=-EVqrDlAqYo https://www.youtube.com/watch?v=-EVqrDlAqYo and click the three dots under the vid. I don't know if there's a way to get those things into an easy to read format? Clicking select all and pasting to the text editor sort of works but could be better.
- KirinDave 7y agoDoes someone have links to written variants of this material rather than a podcast? No offense if you like podcasts but I just don't have time for them even at 2x speed. they're just too slow and disorganized.
- neural_thing 7y agohttps://www.frontiersin.org/articles/10.3389/fncir.2018.00121/full https://www.frontiersin.org/articles/10.3389/fncir.2018.0012...
- patpending 7y agoThis looks relevant: https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-intelligence/ https://numenta.com/blog/2019/01/16/the-thousand-brains-theo...
- neural_thing 7y agoI have a metric ton of respect for Jeff and his team. However, I do think that they did not go far enough in figuring out the computational capabilities of individual pyramidal neurons, which IMO is the foundation to figuring out how the neocortex works. I wrote a short book about that once - if anyone's interested, check it out, it's free and meant to be readable for AI/neuroscience enthusiasts: http://www.corticalcircuitry.com/ http://www.corticalcircuitry.com/
- teabee89 7y agoThanks for sharing! Curious to learn more about this. Have you posted it in the HTM discourse forum? I would be interested in what the community thinks of it!
- neural_thing 7y agoYes, there is a discussion thread here: https://discourse.numenta.org/t/a-book-about-dendritic-computation-and-cortical-layers/2810 https://discourse.numenta.org/t/a-book-about-dendritic-compu...
- fjfaase 7y agoWhat about bird intelligence? The modern view is that the parts of the bird brain use to be called the pallium actually are organized in a similar way as the neocortex. Although a bird brain is usually smaller than the mammal brain, it has in many case the same amount of neurons. Actually, a high percentage of neurons are located in the pallium than in the neocortex. Please note that our cerebellum contains more than half of the number of neurons in the brain.