4 ms·

Hofstadter (GEB) and later Jeff Hawkins (Numenta) have long postulated this theory - and indeed it seems to be compelling - but the prediction-by-analogy paradi

Hofstadter (GEB) and later Jeff Hawkins (Numenta) have long postulated this theory - and indeed it seems to be compelling - but the prediction-by-analogy paradigm has yet to deliver in a convincing way. Would love to see some progress on this front.

ska 5y ago
Hofstadters lab did a bunch of related work but it hasn't been taken up broadly and I'm not sure how much real traction they got. I agree it's interesting.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
_0ffh 5y ago
Yeah, I had to immediately think of "The Subtlety of Sameness" by Robert French. https://en.wikipedia.org/wiki/Robert_M._French https://en.wikipedia.org/wiki/Robert_M._French
gwern 5y ago
If anyone was curious about GPT-3, which draws on zero of this work but is remarkably intelligent nevertheless, Mitchell has access and has found that (when used correctly https://twitter.com/MelMitchell1/status/1285270704313610241 https://twitter.com/MelMitchell1/status/1285270704313610241) that it solves a surprising number of Copycat analogies: https://medium.com/@melaniemitchell.me/can-gpt-3-make-analogies-16436605c446 https://medium.com/@melaniemitchell.me/can-gpt-3-make-analog... More amusing is a comparison with a 5yo human child: https://twitter.com/lacker/status/1294341796831477761 https://twitter.com/lacker/status/1294341796831477761 A pity that she didn't mention any of that, it's pretty interesting.
- Hammershaft 5y ago
  > More amusing is a comparison with a 5yo human child: https://twitter.com/lacker/status/1294341796831477761 https://twitter.com/lacker/status/1294341796831477761 > Q: If axbxcx goes to abc, what does xpxqxr go to? > A: s I've gotta be missing something here?
  - brabel 5y ago
    5-year-olds are prone to answering questions they don't understand randomly?!
joe_the_user 5y ago
I've looked at Jeff Hawkin's On Intelligence. I don't see how it bears similarities Hofstadter's work. Notably, Jeff Hawkins is highly focused on brain anatomy while Hofstadster's work mostly starts at the level of logic and abstraction. Anyway, Hofstadter's (and his student Cabrera's) aim actually hasn't been to deliver a convincing industrial scale system but produce models that illuminate our understanding of thought. I'm not actually sure if that is the right way of doing it but it's worth noting they're not trying to "deliver" on the terms akin to today's neural networks.
Lyapunov_Lover 5y ago
The ideas of Hofstadter and Hawkins aren't really related. Hawkins postulated a singular predictive algorithm as the explanation of how our neocortices works. Hofstadter has focused on analogies and have had some ideas similar to Hopfield networks/the Boltzmann machine. The latter fits in the trend of complex systems research, inspired as it was by spin glasses. Hofstadter worked with ideas that emerged from cybernetics, such as recursive causation and the vital role of feedback. Hawkins came from a different direction and ended up with a wholly predictive theory. It should be noted that predictive coding predates Hawkins' work, perhaps best exemplified by Rao and Ballard's hierarchical predictive coding model from 1999. Various researchers working from similar assumptions put their thoughts together in Bayesian Brain in 2006. Since then, predictive processing has grown to be a very popular framework in neuroscience and cognitive science in general. Friston's free energy principle and philosopher Andy Clark's popularization of it have a lot to do with that. There's been a ton of progress so far, so there's plenty of academic material for you to dive into.
- d0mine 5y ago
  Without googling I can't say at the first glance whether the parent comment is bot-generated
  - deleted 5y ago
    [deleted]

ska 5y ago

Hofstadters lab did a bunch of related work but it hasn't been taken up broadly and I'm not sure how much real traction they got. I agree it's interesting.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

_0ffh 5y ago

Yeah, I had to immediately think of "The Subtlety of Sameness" by Robert French. https://en.wikipedia.org/wiki/Robert_M._French https://en.wikipedia.org/wiki/Robert_M._French

gwern 5y ago

If anyone was curious about GPT-3, which draws on zero of this work but is remarkably intelligent nevertheless, Mitchell has access and has found that (when used correctly https://twitter.com/MelMitchell1/status/1285270704313610241 https://twitter.com/MelMitchell1/status/1285270704313610241) that it solves a surprising number of Copycat analogies: https://medium.com/@melaniemitchell.me/can-gpt-3-make-analogies-16436605c446 https://medium.com/@melaniemitchell.me/can-gpt-3-make-analog... More amusing is a comparison with a 5yo human child: https://twitter.com/lacker/status/1294341796831477761 https://twitter.com/lacker/status/1294341796831477761 A pity that she didn't mention any of that, it's pretty interesting.

Hammershaft 5y ago
> More amusing is a comparison with a 5yo human child: https://twitter.com/lacker/status/1294341796831477761 https://twitter.com/lacker/status/1294341796831477761 > Q: If axbxcx goes to abc, what does xpxqxr go to? > A: s I've gotta be missing something here?
- brabel 5y ago
  5-year-olds are prone to answering questions they don't understand randomly?!

joe_the_user 5y ago

I've looked at Jeff Hawkin's On Intelligence. I don't see how it bears similarities Hofstadter's work. Notably, Jeff Hawkins is highly focused on brain anatomy while Hofstadster's work mostly starts at the level of logic and abstraction. Anyway, Hofstadter's (and his student Cabrera's) aim actually hasn't been to deliver a convincing industrial scale system but produce models that illuminate our understanding of thought. I'm not actually sure if that is the right way of doing it but it's worth noting they're not trying to "deliver" on the terms akin to today's neural networks.

Lyapunov_Lover 5y ago

The ideas of Hofstadter and Hawkins aren't really related. Hawkins postulated a singular predictive algorithm as the explanation of how our neocortices works. Hofstadter has focused on analogies and have had some ideas similar to Hopfield networks/the Boltzmann machine. The latter fits in the trend of complex systems research, inspired as it was by spin glasses. Hofstadter worked with ideas that emerged from cybernetics, such as recursive causation and the vital role of feedback. Hawkins came from a different direction and ended up with a wholly predictive theory. It should be noted that predictive coding predates Hawkins' work, perhaps best exemplified by Rao and Ballard's hierarchical predictive coding model from 1999. Various researchers working from similar assumptions put their thoughts together in Bayesian Brain in 2006. Since then, predictive processing has grown to be a very popular framework in neuroscience and cognitive science in general. Friston's free energy principle and philosopher Andy Clark's popularization of it have a lot to do with that. There's been a ton of progress so far, so there's plenty of academic material for you to dive into.

d0mine 5y ago
Without googling I can't say at the first glance whether the parent comment is bot-generated
- deleted 5y ago
  [deleted]