8 ms·
> On the other hand, these cultures have essentially no advantage over digital computers and modern machine learning models. Absolutely false. While it's indee
by xvilka 3y ago
> On the other hand, these cultures have essentially no advantage over digital computers and modern machine learning models.
Absolutely false. While it's indeed hard to keep it alive, real neurons are far more sophisticated than what AI researchers think they are. Modern digital so called neural networks are built on the outdated and oversimplified knowledge of neuron model, almost a century-old by now.
- seydor 3y agosophisticated is not a scientific word. they are complex and complicated, and the voltage dynamics across their elaborate membrane takes a lot of computers to simulate. But we don't really know what it is doing or if it is particularly sophisticated. Nature has found a lot of complex solutions to simple problems because it does not know better. We don't know how well it did with intelligence
- xvilka 3y agoWe already do know[1] the a single neuron has the same level of complexity as multilayered digital "neural network". [1] https://www.youtube.com/watch?v=hmtQPrH-gC4 https://www.youtube.com/watch?v=hmtQPrH-gC4
- seydor 3y agoThere are different studies proposing 2 or 3 layer network for representing the input-firing curve of neurons (Usually hippocampal). Of course, neural networks are abritrary approximators so the size of the network determines the fidelity of the reproduction. But it 's not clear what the firing does and what amount of complexity in the firing code is reduntant or useful for making AI systems
- uoaei 3y agoWhat is clear however is the evident power savings in implementing cultured neural networks vs digital ones for a given network capacity.
- p1esk 3y agoEven that is not clear. A model like GPT-4 can read an entire book in seconds, and produce an intelligent answer about its content [1]. A human would need at least several hours to perform the same task. [1] https://www.anthropic.com/index/100k-context-windows https://www.anthropic.com/index/100k-context-windows
- uoaei 3y ago> for a given network capacity You'd be hard-pressed to find an expert who believes any of the current crop of LLMs have a similar capacity to human brains.
- p1esk 3y agoUnlike the capacity of human brains the capacity of ML models has been growing very fast in the last 10 years. The number of tasks AI cannot do is shrinking fast.
- uoaei 3y agoPower consumption is what's relevant to this discussion thread here. We're not talking about possible capacity but possible efficiency for given capacity as implemented in analog vs digital circuits.
- p1esk 3y agoIf a "given capacity" is the ability to read books and answer questions about the content, then LLMs are more power efficient (4kW for 20 seconds beats 20W for 3 hours). That's on standard power hungry GPUs (8xA100 server consumes about 4kW). If we switch to analog deep learning accelerators, we gain even better power efficiency. There's simply no chance for brains to compete with electronic devices - as long as we match brain's "capacity".
- uoaei 3y ago
- p1esk 3y agoModern CMOS transistors are extremely sophisticated devices (you need a very complicated model with hundreds of parameters to simulate all kinds of quantum effects to predict its behavior). Yet all it does is one simple function - it's an on/off switch.
- xvilka 3y agoEvolution would not allow waste of energy and complexity for just one on/off switch. Inefficient things die out in the course of millions of years. Neural tissue on itself is far older than humanity, so it had much more time to perfect.
- lukebitts 3y agoThat’s not true at all. If something is a waste but doesn’t meaningfully change yours odds of suvival, “evolution” won’t care
- xvilka 3y agoOut there food and energy are often scarce. Evolution does care about efficiency in that particular case a lot. True, there are ecosystems with plenty of free food but they are rarity.
- jacquesm 3y agoEvolution doesn't care about anything.
- runsWphotons 3y agoit is possible this is not quite true
- jacquesm 3y agoNo, that is not possible.