6 ms·
One thing I note from listening to my wife talking about her work, which is in molecular biology of human neurons, is that there seems to be so many signalling
by jakewins 4y ago
One thing I note from listening to my wife talking about her work, which is in molecular biology of human neurons, is that there seems to be so many signalling mechanisms, vs just one in the neural nets I’ve seen in software.
Like, there are more than 40 separate neurotransmitters, each effectively a float output, I think? So at each synapse at every instant, potentially 40 separate floats can be communicated.
And then there’s adjacency - the network they were studying this spring they suspect is communicating using the electric fields around the neurons, so adjacent but unconnected neurons influence each other via electrical induction.
And then it’s like - and my wife says there’s no evidence neurons do this - but cells can rewrite their own DNA to store information. So each neuron is a tiny microcomputer with 40 pwm outputs and what, thousands of inputs, and 350Mib of read/write memory. And there’s 86 Billion of them in each brain?!
It just seems so much larger than a matrix of floats and weights.
- zitterbewegung 4y agoIf we were to rename Neural Networks it would probably be Multiplicative Tensor based machines for neural networks / deep neural networks and then denote the subtypes Backpropigative Tensor Machines , Adversarial Tensor Machines and Attention Tensor machines.
- tux3 4y agoRewrite their own DNA is a little misleading, because the actual DNA codons don't normally change. But cells can affect gene expression throught multiple mechanisms, so that part of the DNA gets activated or de-activated. You can use those mechanisms to react to the environment in various ways. For instance when you see chemical signals A and B, start making protein C. Or, when you see some signal, make the neuron more sensitive at the synapse (add receptors). You can see it as storing state in a sort of memory, but that's pushing the computer analogy to its limits. Cells are pretty unlike chips :)
- epgui 4y agoEven though it is approximately correct that the codons don’t change, the codons can be chemically modified in a bunch of different ways (eg.: methylation, acetylation, phosphorylation, ubiquitination, sumoylation…)
- melony 4y ago> the actual DNA codons don't normally change. Unless you are bacteria! I will see myself out.
- capitainenemo 4y agoPerhaps was thinking of https://www.insideprecisionmedicine.com/news-and-features/extensive-dna-breaking-needed-to-make-memories/ https://www.insideprecisionmedicine.com/news-and-features/ex... ? Although AFAIK this deliberate DNA breakage doesn't involve actual modification to the DNA...
- deltaonefour 4y agoThere must be something "turing complete-ish" about neural nets. As in although the primitive element of a human neuron is complex, you can achieve identical computational power with a much simpler model.
- skohan 4y agoWait are you saying you can achieve identical computational power to a biological neural network with an ANN? How do you justify that assumption?
- deltaonefour 4y agoNo. I'm not. I'm saying there must be simple model primitives that don't involve electromagnetic fields and other strange communication channels that arise out of evolution that can be used to model intelligence. We aren't fully clear about what this model of intelligence is, but I'm instinctively sure that it won't have to involve anything as complex as biological neurons that communicate through thousands of different pathways and electromagnetic side effects too.
- levitatorius 4y agoIt's fascinating that we have simple primitives or notions of analysis, deduction, causation, yet no artificial system where those features of intelligence emerge on their own.
- deltaonefour 4y agoNot true. It's called unsupervised learning.
- levitatorius 4y agoWhat do you think is missing then? Unsupervised learning == general intelligence?
- feet 4y agoThose float inputs are even more complex than that. Some inputs are excitatory and some are inhibitory, but some also modify the response of the neuron to other inputs
- skohan 4y agoIt's more complicated than one neurotransmitter == 1 float. For instance you can have more than one receptor type for each neurotransmitter. And each one might be sensitive to different concentrations of that neurotransmitter, and have different signaling modalities. And the behavior of those receptors might also be modulated by other signaling mechanisms inside the cell: i.e. neurotransmitter A might modulate the signaling of neurotransmitter B. In other words, there is a lot of complex information processing going on there.
- brundolf 4y agoSome people think we're on track to approximate the brain with a computer on the order of a couple decades from now. But I wonder if we'll discover it's physically impossible to do that with transistor-based chips at all (density of information, energy limits, etc). Of course that wouldn't mean AGI itself is impossible - biology's own mechanisms could be recreated artificially - but AGI estimates would be way off if we find out in a few years that actually we have to start over from scratch.