4 ms·

Jeff Hawkins Announces the Thousand Brains Project for AI

30 points by kjhughes 2y ago

eimrine 2y ago
I appreciate Jeff Hawkins the author of 'On Intelligence' book which has shaped my understanding of brains.
deleted 2y ago
[deleted]
bongodongobob 2y ago
> Neural networks also often prove unstable, with slight alterations in the data they receive leading to wild changes in outcomes. For instance, previous research found that changing a single pixel on an image can make an AI think a horse is a frog. Referencing a 7 year old paper. > To overcome these challenges, the Thousand Brains Project aims to develop a new AI platform by reverse engineering the neocortex Oh just going to reverse-engineer the brain, why didn't anyone think of that before > Today’s neural networks are based on basic neuroscience from 80 years ago. We’ve learned a lot about neuroscience since then, and we want to use that knowledge to advance AI Huh? I wasn't aware any neural networks were based on biology at all other than the loose activate and fire idea. > “Once we learn how to build one cortical column, we can build as many as we want,” Hawkins says. Easy, just have to learn how to do it > When you look at deep networks, they don’t fundamentally understand the world, which is why if you just change one tiny feature of an image, they often don’t recognize it Not really true unless you develop something specifically to do this. Also, "understand" needs to be defined Fluff piece that reads as an ad hoping for funding. Sounds like a money pit to me.
- eimrine 2y ago
  "To understand" means "to make a prediction" in Hawkins' paradigm.
- spywaregorilla 2y ago
  > Huh? I wasn't aware any neural networks were based on biology at all other than the loose activate and fire idea. Sounds like you are aware then.
  - bongodongobob 2y ago
    That's like saying integrated circuits are based on neurons because they route signals. It's more of an analogy than a model.
    - spywaregorilla 2y ago
      Integrated circuits are similar to neurons in that way. They are not based on neurons. Neural networks significantly diverge from actual brain mechanisms. But they are originally based on the original concept as you seem to know. And this person is, perhaps unjustifiably, claiming that they want to bring it closer.
      - bongodongobob 2y ago
        Fair point. I just feel like his statement saying current neural networks are based on the brain is way off. Especially in the popular LLM paradigm, it's not something anyone is trying to mimic.
padolsey 2y ago
This seems essentially an announcement of an SDK that is a lot like older stuff from Numenta–is that wrong? I remember playing around with their stuff a few years ago. Is there something new? Related to all this: In light of the recent AI Spring ("Attention is all you need" era), is anyone finding ML/AI slowly converging with our understanding of neurology or are we drifting to a different model where 'True AI' won't be found in emulating our own brains? It seems the latter. Our weird and wonderful brains were brewed under evolutionary constraints that have produced remarkable amounts of redundancy, because, well... a brain is incredibly precious to the body its operating within. I don't know if building an AI computer needs to be burdened with concepts born of those constraints?
ilaksh 2y ago
It sounds like what he has always been talking about. Did he get new funding? It seems like there could be a loose relationship between having "many brains" and having many experts in MoE. That's why I thought something like the Arctic Snowflake LLM or whatever they called it, or a model with LOTs of experts, could have an advantage. Although that one hasn't taken over any leaderboard, obviously. The new DeepMind paper about transformers with NARs is really interesting also. It seems like there are eventually going to be several really powerful but different approaches to general purpose AI.
- AlanYx 2y ago
  He got new funding, but it's relatively modest ($2.69 million spread over two years).
OutOfHere 2y ago
He looks to be peddling the same info that he has been peddling for twenty years. If he has had no large scale projects or results to show in this time, why would someone listen to him now? I am not saying his ideas don't have merit, only that it doesn't look like he can be trusted to implement them well. Perhaps someone else can take over his ideas and do a better job.
- observationist 2y ago
  Some things are incremental and tedious. Hawkins is taking a rigorous, incremental, and tedious approach to realizing his dream of brain like AI. That means synthesizing thousands of neurology papers and isolating each component of neurons and searching for evidence of algorithms and so on. It doesn't mean glomming on to the latest and greatest deep learning algorithms, optimizing for a particular application, product, or market. Hawkins is doing research; not productization.
  - OutOfHere 2y ago
    Yes, there is need for research, but the proof is always in the pudding. Is his goal to help along AI architectures, or is it to efficiently simulate the brain instead? If it's the former, the perhaps the incremental lessons ought to be productionized as they are discovered. Either way, there is need for a benchmark score to be attacked, so the score can be targeted for improvement as time progresses. In contrast, LLMs are tested on numerous benchmarks, improving steadily year after year. All the best to his dedicated efforts.
    - aeternum 2y ago
      Remember that neural networks were invented in 1943 and were not productized until much later. Often it takes time for the engineering to catch up to the theory.

30 points by kjhughes 2y ago

eimrine 2y ago

I appreciate Jeff Hawkins the author of 'On Intelligence' book which has shaped my understanding of brains.

deleted 2y ago

[deleted]

bongodongobob 2y ago

> Neural networks also often prove unstable, with slight alterations in the data they receive leading to wild changes in outcomes. For instance, previous research found that changing a single pixel on an image can make an AI think a horse is a frog. Referencing a 7 year old paper. > To overcome these challenges, the Thousand Brains Project aims to develop a new AI platform by reverse engineering the neocortex Oh just going to reverse-engineer the brain, why didn't anyone think of that before > Today’s neural networks are based on basic neuroscience from 80 years ago. We’ve learned a lot about neuroscience since then, and we want to use that knowledge to advance AI Huh? I wasn't aware any neural networks were based on biology at all other than the loose activate and fire idea. > “Once we learn how to build one cortical column, we can build as many as we want,” Hawkins says. Easy, just have to learn how to do it > When you look at deep networks, they don’t fundamentally understand the world, which is why if you just change one tiny feature of an image, they often don’t recognize it Not really true unless you develop something specifically to do this. Also, "understand" needs to be defined Fluff piece that reads as an ad hoping for funding. Sounds like a money pit to me.

eimrine 2y ago
"To understand" means "to make a prediction" in Hawkins' paradigm.
spywaregorilla 2y ago
> Huh? I wasn't aware any neural networks were based on biology at all other than the loose activate and fire idea. Sounds like you are aware then.
- bongodongobob 2y ago
  That's like saying integrated circuits are based on neurons because they route signals. It's more of an analogy than a model.
  - spywaregorilla 2y ago
    Integrated circuits are similar to neurons in that way. They are not based on neurons. Neural networks significantly diverge from actual brain mechanisms. But they are originally based on the original concept as you seem to know. And this person is, perhaps unjustifiably, claiming that they want to bring it closer.
    - bongodongobob 2y ago
      Fair point. I just feel like his statement saying current neural networks are based on the brain is way off. Especially in the popular LLM paradigm, it's not something anyone is trying to mimic.

padolsey 2y ago

This seems essentially an announcement of an SDK that is a lot like older stuff from Numenta–is that wrong? I remember playing around with their stuff a few years ago. Is there something new? Related to all this: In light of the recent AI Spring ("Attention is all you need" era), is anyone finding ML/AI slowly converging with our understanding of neurology or are we drifting to a different model where 'True AI' won't be found in emulating our own brains? It seems the latter. Our weird and wonderful brains were brewed under evolutionary constraints that have produced remarkable amounts of redundancy, because, well... a brain is incredibly precious to the body its operating within. I don't know if building an AI computer needs to be burdened with concepts born of those constraints?

ilaksh 2y ago

It sounds like what he has always been talking about. Did he get new funding? It seems like there could be a loose relationship between having "many brains" and having many experts in MoE. That's why I thought something like the Arctic Snowflake LLM or whatever they called it, or a model with LOTs of experts, could have an advantage. Although that one hasn't taken over any leaderboard, obviously. The new DeepMind paper about transformers with NARs is really interesting also. It seems like there are eventually going to be several really powerful but different approaches to general purpose AI.

AlanYx 2y ago
He got new funding, but it's relatively modest ($2.69 million spread over two years).

OutOfHere 2y ago

He looks to be peddling the same info that he has been peddling for twenty years. If he has had no large scale projects or results to show in this time, why would someone listen to him now? I am not saying his ideas don't have merit, only that it doesn't look like he can be trusted to implement them well. Perhaps someone else can take over his ideas and do a better job.

observationist 2y ago
Some things are incremental and tedious. Hawkins is taking a rigorous, incremental, and tedious approach to realizing his dream of brain like AI. That means synthesizing thousands of neurology papers and isolating each component of neurons and searching for evidence of algorithms and so on. It doesn't mean glomming on to the latest and greatest deep learning algorithms, optimizing for a particular application, product, or market. Hawkins is doing research; not productization.
- OutOfHere 2y ago
  Yes, there is need for research, but the proof is always in the pudding. Is his goal to help along AI architectures, or is it to efficiently simulate the brain instead? If it's the former, the perhaps the incremental lessons ought to be productionized as they are discovered. Either way, there is need for a benchmark score to be attacked, so the score can be targeted for improvement as time progresses. In contrast, LLMs are tested on numerous benchmarks, improving steadily year after year. All the best to his dedicated efforts.
  - aeternum 2y ago
    Remember that neural networks were invented in 1943 and were not productized until much later. Often it takes time for the engineering to catch up to the theory.