3 ms·
Amazing article, and for a layperson who’s been reading a lot of neuroscience, the perfect level of complexity. I love an article that makes you (internally) sh
by bbor 3y ago
Amazing article, and for a layperson who’s been reading a lot of neuroscience, the perfect level of complexity. I love an article that makes you (internally) shout “why haven’t I wondered about that before?” over and over, so thanks for that. A compliment as to clarity of purpose, I suppose.
Materially;
That’s bad news for anyone hoping to simulate a brain digitally. It means there’s a lot more relevant stuff to simulate (like the learning that goes on within cells) than the connectionist paradigm of treating each biological neuron like a neural-net “neuron” would imply, and thus the computational requirements of simulating a brain are higher — maybe vastly higher — than connectionists hope.
I get where she’s coming from, and she’s not wrong, but it seems like unnecessary detour to dunk on another AI “camp” in the field for drama points and satisfaction - the Marcus Maneuver, if you will.
Connectionism isn’t a cult or an institution, it’s a paradigm that emphasis the utility of big nets of interconnected smaller pieces. Any self-avowed connectionist (are there any left? Honest question) could just retreat to “ok well it’s networks of brain cells plus smaller intracellular networks” and keep their paradigm. And all we can say to that is “ugh, I guess”, as far as I can tell!
- eurekin 3y agoI was also stumped by this exact quote. The whole article was in the best spirit, until this. It's a model we have, it will get updated in order to be more useful. Every engineering field builds a model of reality. Who are the connectionists? Is this some kind of "they people" label for, what causes fear in a typical layperson?
- einsum 3y agoPrecisely, connectionism in modern AI boils down to the idea that learning should be expressed in terms of DAGs that are composed of simpler units. It’s quite likely that the units that are currently used are too abstract, but this doesn’t necessarily mean the paradigm itself is flawed.
- jampekka 3y agoI don't think connectionism is restricted to acyclic graphs. Or even graphs in general. But you're right that the connectionism as an approach is more abstract than just simulating neuron behavior.
- jampekka 3y agoAt least one self-avowed professional connectionist here. I was coming to make similar critique here. Connectionism isn't and never was about trying to simulate the biological neural networks and other anatomy. As you said, it's about emphasizing the network and network emergent phenomena over isolated pieces (e.g. "grandmother neurons" or strict localization of brain function). At the information processing level the contrast is to classicism/symbolicism that tries to explain cognition as atomic and modular operations on symbols.
- bbor 3y agoGood to hear from one, thanks! My big insight when I first started really getting into ai was “we should unite the two camps now!”, only to find out we’ve basically been doing that since the 90s. So I'm glad the debate survives a bit to this day! Honestly, after 2023 I think we’re all connectionist in a way lol, except for the old guard, of which Chomsky might be the only (adjacent) one left. Godspeed to Chomsky, I honestly wouldn’t be surprised if he has one last scientific revolution left in him