5 ms·

Surely this wasn't a novel concept in 2018? It seems a trivial idea, and I'm pretty sure the machine learning course I followed that explained deep convolutiona

by tinco 4y ago

Surely this wasn't a novel concept in 2018? It seems a trivial idea, and I'm pretty sure the machine learning course I followed that explained deep convolutional neural networks as being made up of workers identifying various parts of an image and passing their conclusions on to next layers that aggregate and decide, is both older than 2018 and basically the same idea. I'd be very surprised if this line of thinking wasn't thoroughly explored in the 70's. Of course that doesn't mean there aren't novel concepts in their book and research, but just a bit weird to pose the base "thousand brains" idea as novel.

musingsole 4y ago
"Novel" is relative. However, this concept dates back at least to the 1980s with Marvin Minsky's "The Society of the Mind": https://www.amazon.com/Society-Mind-Marvin-Minsky/dp/0671657135?ref_=d6k_applink_bb_dls&dplnkId=1cea33f2-6d66-4de9-8339-076e56ce276e https://www.amazon.com/Society-Mind-Marvin-Minsky/dp/0671657...
iamgopal 4y ago
When knowledge surpasses memory, repeated knowledge feels like discovery.
derefr 4y ago
You’re talking about a pure hierarchy — “passing their conclusions onto next layers to decide.” This “thousand brains” concept refers instead to a peerwise mesh, where “workers” at each layer receive both lower-layer inputs, and also their peers’ outputs, and use both to derive an output that evolves over time, as outputs are passed between peers, until reaching a layerwise equilibrium-state (a “consensus.”) The (again, many) models in the layer above receive this evolving consensus as the output from subsets of nodes in the layer below — potentially with overlap in which upper-layer models receive outputs of which lower-layer models; and each of these models themselves evolve to a consensus. By analogy: DCNN is like representative democracy, where each layer is a representative with the layer below as its “constituents”, forming a belief based on the multitude of lower-layer beliefs it hears, and then acting on its own formed belief when reporting to the layer above. This “thousand brains” model is more like grouping constituents into juries and running them through the Delphi-model iterative consensus; and then having representatives who act on continuous-feed statistical aggregations of the Delphi-model iteration-step outputs of multiple juries.
jeffffff 4y ago
it sounds an awful lot like boosting https://en.wikipedia.org/wiki/Boosting_(machine_learning) https://en.wikipedia.org/wiki/Boosting_(machine_learning)

by tinco 4y ago

musingsole 4y ago

"Novel" is relative. However, this concept dates back at least to the 1980s with Marvin Minsky's "The Society of the Mind": https://www.amazon.com/Society-Mind-Marvin-Minsky/dp/0671657135?ref_=d6k_applink_bb_dls&dplnkId=1cea33f2-6d66-4de9-8339-076e56ce276e https://www.amazon.com/Society-Mind-Marvin-Minsky/dp/0671657...

iamgopal 4y ago

When knowledge surpasses memory, repeated knowledge feels like discovery.

derefr 4y ago

You’re talking about a pure hierarchy — “passing their conclusions onto next layers to decide.” This “thousand brains” concept refers instead to a peerwise mesh, where “workers” at each layer receive both lower-layer inputs, and also their peers’ outputs, and use both to derive an output that evolves over time, as outputs are passed between peers, until reaching a layerwise equilibrium-state (a “consensus.”) The (again, many) models in the layer above receive this evolving consensus as the output from subsets of nodes in the layer below — potentially with overlap in which upper-layer models receive outputs of which lower-layer models; and each of these models themselves evolve to a consensus. By analogy: DCNN is like representative democracy, where each layer is a representative with the layer below as its “constituents”, forming a belief based on the multitude of lower-layer beliefs it hears, and then acting on its own formed belief when reporting to the layer above. This “thousand brains” model is more like grouping constituents into juries and running them through the Delphi-model iterative consensus; and then having representatives who act on continuous-feed statistical aggregations of the Delphi-model iteration-step outputs of multiple juries.

jeffffff 4y ago

it sounds an awful lot like boosting https://en.wikipedia.org/wiki/Boosting_(machine_learning) https://en.wikipedia.org/wiki/Boosting_(machine_learning)