5 ms·
We already do know[1] the a single neuron has the same level of complexity as multilayered digital "neural network". [1] https://www.youtube.com/watch?v=hmtQPr
by xvilka 3y ago
We already do know[1] the a single neuron has the same level of complexity as multilayered digital "neural network".
[1] https://www.youtube.com/watch?v=hmtQPrH-gC4 https://www.youtube.com/watch?v=hmtQPrH-gC4
- seydor 3y agoThere are different studies proposing 2 or 3 layer network for representing the input-firing curve of neurons (Usually hippocampal). Of course, neural networks are abritrary approximators so the size of the network determines the fidelity of the reproduction. But it 's not clear what the firing does and what amount of complexity in the firing code is reduntant or useful for making AI systems
- uoaei 3y agoWhat is clear however is the evident power savings in implementing cultured neural networks vs digital ones for a given network capacity.
- p1esk 3y agoEven that is not clear. A model like GPT-4 can read an entire book in seconds, and produce an intelligent answer about its content [1]. A human would need at least several hours to perform the same task. [1] https://www.anthropic.com/index/100k-context-windows https://www.anthropic.com/index/100k-context-windows
- uoaei 3y ago> for a given network capacity You'd be hard-pressed to find an expert who believes any of the current crop of LLMs have a similar capacity to human brains.
- p1esk 3y agoUnlike the capacity of human brains the capacity of ML models has been growing very fast in the last 10 years. The number of tasks AI cannot do is shrinking fast.
- uoaei 3y agoPower consumption is what's relevant to this discussion thread here. We're not talking about possible capacity but possible efficiency for given capacity as implemented in analog vs digital circuits.
- p1esk 3y agoIf a "given capacity" is the ability to read books and answer questions about the content, then LLMs are more power efficient (4kW for 20 seconds beats 20W for 3 hours). That's on standard power hungry GPUs (8xA100 server consumes about 4kW). If we switch to analog deep learning accelerators, we gain even better power efficiency. There's simply no chance for brains to compete with electronic devices - as long as we match brain's "capacity".
- uoaei 3y agoOk so it's clear you are not familiar with the vocabulary of this domain. "Capacity" means something specific (if extremely hard to measure precisely) about the informational throughput of a system, not just "the ability to do high-level task X as judged by a naive human observer". I urge you to study more seriously about the things you seem so eager to speculate about before publishing underinformed opinions in public spaces.