2 ms·
As a former neuroscientist, I welcome attempts to add more biologically-inspired complexity to the relatively simple computer-based NNs. That being said, I sus
by KingMob 3y ago
As a former neuroscientist, I welcome attempts to add more biologically-inspired complexity to the relatively simple computer-based NNs.
That being said, I suspect the authors still need to take a Neuroscience 101 class. There's any number of neuronal features missing from NNs that would be more interesting to incorporate before looking towards glial cells like astrocytes. E.g., different neurotransmitters, G proteins, gap junctions/electrical synapses, pyramidal neurons, really all large-scale and long-range organizational structures in brain.
- pushfoo 3y agoTo my understanding, pyramidal cells are already partially emulated by certain structures of weights. It's not the same as dedicated multi-layer mechanisms outside the local flow, but that option seems difficult to implement efficiently with current compute technology. Maybe more VRAM to hold the added inter-layer features would allow it? NVidia seems reluctant or unable to do that on the consumer side. I haven't had time to thoroughly read up on why. All of these seem plausible: * Offsetting rising per-unit costs from other components * Intentional market segmentation * Export restrictions
- KingMob 3y agoI suspect cost/efficiency is a driving factor. Existing layered NN designs seem quite local, with fewer long-range connections than brains have. I'd bet that updating non-local weights increases memory requirements, slows down computation by orders of magnitude, or both.