6 ms·
Disclaimer: I have a PhD from the department Stephen Grossberg founded. Gail Carpenter, the co-creator of ART, was my advisor. I am also co-authoring a neurosc
by ChaitanyaSai 5y ago
Disclaimer: I have a PhD from the department Stephen Grossberg founded. Gail Carpenter, the co-creator of ART, was my advisor. I am also co-authoring a neuroscience non-fiction book that features Steve's work, along with other stuff. (Just so you know I am a reasonably biased observer).
Most of you have probably not heard of Stephen Grossberg. He is in my opinion one of the greatest neuroscientists of this generation. Maybe the greatest theoretical neuroscientist ever.
Neuroscience is swimming (barely) in a ton of data, but very little in the way of a unifying framework. Steve's body of work over 60 years provides that. ART is only one part of it. ART has some way to go before proving itself as undeniably useful as deep-learning on the application side of things. But it is very compelling as a model for how our (mammalian) brains might be operating.
One fundamental idea, which is worth considering in applications is how almost all of deep-learning is founded on the idea of an "error" (A mis-match between expectation and the prediction) and how this flows through the system. Backprop is obviously the most popularly known method for this flow. ART is fundamentally different in that it works with the idea of a match rather than a mis-match. This foundational even philosophical difference is interesting because if allows for learning to happen all the time even while perceiving is simultaneously happening. It also allows for learning to be unsupervised. This philosophical departure point is actually very interesting: If you rely entirely on the idea of an error and throw out the rest, can you even hope to build a conscious system?
Steve btw also has what I think is the most elegant and mechanistically precise explanation of consciousness, which builds on ART. Even if you are skeptical of the current promise of ART as a replacement for deep-learning, his explanation of how ART-like systems enable consciousness (and different conscious percepts) are alone worth the price of admission.
I’ve written more about Stephen Grossberg and his book here: https://saigaddam.medium.com/the-greatest-neuroscientist-youve-never-heard-of-17c61b654a3e https://saigaddam.medium.com/the-greatest-neuroscientist-you...
- Isinlor 5y agoIt's weird how many unrecognized geniuses there are in the neuroscience all claiming to have explained the brain and people just need to recognize their genius. All sounds very similar to Jeff Hawkins.
- ChaitanyaSai 5y agoYour skepticism is warranted :) Grossberg is not that unrecognized. I'd recommend Talking Nets by Anderson and Rosenfeld and Margaret Boden's Mind as Machine. Both are scholarly histories of the field. Boden's book is far more comprehensive. Worth reading in any case if you are interested in the study of the brain and mind.
- captaincaveman 5y agoI remember ART from 25 years ago, sounded very promising even if I didn't fully understand at the time (nor now, but not revisited). Obviously back-propagation based NN have become very popular (even got a rebrand as Deep Learning), how has ART changed in the last 25 years, is there any interesting results using ART instead of DL?
- ChaitanyaSai 5y agoThere have been updates on the supervised versions of ART (ARTMAP) but none I know of, which have been shown to be as as effective as DL methods on very large datasets. Not sure if there have been many/any attempts to tackle large datasets. DL methods, while opaque, often result in good feature basis sets. ART given it was devised to model perception in brains, assumes features are created by lower network layers (V1, primary auditory cortex etc). I don't think there's any reason to suspect ART could not be extended to aid feature discovery, but it hasn't been done as far as I know.