7 ms·
Except that every synapse is not a dumb weight but a highly complex system connected to an even more complex system (aka neuron) which might each be a (super)co
by singularity2001 5y ago
Except that every synapse is not a dumb weight but a highly complex system connected to an even more complex system (aka neuron) which might each be a (super)computer on its own.
Given how extremely bad we are at computing, there is hope (for ai) that the neurons or their circuits are not _that_ powerful after all.
- alecst 5y agoLast I heard (and I believe this could be wrong) my professor said that we basically understand how a single neuron works. That like basically if we do X input we get Y output, up to some accuracy. He used this to discuss the idea behind neural networks -- that each neuron is simple enough to model, all we need to worry about is the weights and the dynamics of the network as a whole. How much of a simplification is that? And how much does the accuracy of such a model matter, in the grand scheme of things?
- andyxor 5y agoThat’s correct understanding as of 1943 when the “artificial neural network“ model your professor is teaching was developed. There is a whole lot of new knowledge on how live neurons and networks of neurons work that had been collected in the last 75 years in the neuroscience domain but it’s mostly ignored by computer scientists.
- visarga 5y ago> All we need to worry about is the weights and the dynamics of the network as a whole. How much of a simplification is that? A lot. Parallel optimization is an art form. These models are trained on static datasets, they can't intervene in the environment to infer causal relations, so they need legs and hands.
- google234123 5y agoThey totally can https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e...
- manquer 5y agoI would say quite a bit. Adding even a third body makes it impossible to calculate physics with certainty. A complex system with any number of individual components is hard to understand with certainty and/or calculations can become exponentially more complex .
- google234123 5y agoCiting the fact that the 3 body problem doesn't always have an exact solution is a straw man argument.
- manquer 5y agoThe intent was to illustrate that complexities of a system of simple components can be pretty difficult. Automata theory has more appropriate examples perhaps.
- gisely 5y agoYour professor lied.
- sydd 5y agooh finally something that I learned a lot about :) Such research is the area of computational neuroscience - one thing that such people do is try to model parts of the brain (or just a single neuron) with computers. A Neuron (=nerve cell in the brain) is a very complex beast. In rough terms they work like this: They collect signals (electrical impulses) via their small appendages called dendrites. when the sum of the signals reaches a certain threshold a large electrical impulse is generated at the cell body that will travel trough its "output" appendage (called axon) that connected to another neuron's cell body or to its dendrite. Neurons display a dazzling variety in all these parameters: - In morphology, e.g. they can look like a pine tree http://www.scholarpedia.org/article/Pyramidal_neuron http://www.scholarpedia.org/article/Pyramidal_neuron (I really recommend scholarpedia, also this article has a nice animation on how electrical impulses propagate) or like a sea urchin. - it really matters where the cell gets its impulse from: A neuron stimulated near its cell body will be much more sensitive to the input than being stimulated far away. - Their response characteristics are wildly varied too. Some give off one large impulse, some a quick burst of impulses. Some are preventing others from giving out impulses from stimulation (inhibitor neurons) - This whole mess can be modulated with chemical compounds that are released by the body -- some make some neurons more sensitive, some less. - Also we still discover every year some new mechanism that modulates how they function. The issue is that this results in such a complex system that a modern PC cant even simulate 1 detailed neuron model realtime (these tools are open source, try them out! for example https://neuron.yale.edu/ https://neuron.yale.edu/ ). Now we know that we're simulating things that likely do not matter (e.g. we don't need a neuron model that consist of 10.000+ segments), but we do not know which parts we need to remove to have a faithful simulation. Also we might simply simulate some parts wrong because our knowledge of the subject is not enough. But on the upside we've reached some great things already, for example we know how our brain calculates from our head and eye position the orientation of the things we're looking at
- abecedarius 5y agoEmulating a neuron != taking a comparable part in a computation. Probably the former is a lot more complex. For instance, an artificial net can take advantage of backpropagation in a separated training phase -- that's a lot of complexity that's factored out of the runtime phase.
- TeMPOraL 5y agoWonder how this architecture is limiting the space, though - all biological brains train continuously. Our DNNs are more like a brain upload snapshot that's always run for one cycle and then rebooted.
- abecedarius 5y agoYeah, that's worth exploring more. It's just that it's not safe to depend on "real neurons are complicated, and therefore artificial nets of simple units won't have transformative capabilities".
- tudorw 5y agoWon't somebody think of the exosomes and telocytes?