3 ms·

You probably already know this (since you wrote "silly thought"), but real-life neurons are ridiculously more complex than simulated "neurons" in an NN. So the

You probably already know this (since you wrote "silly thought"), but real-life neurons are ridiculously more complex than simulated "neurons" in an NN. So the analogy doesn't really hold.

pishpash 5y ago
They're more complex in biological construction and in signaling mechanism, but no proof that they are more complex in function.
- mattkrause 5y ago
  An individual biological neuron can compute a variety of functions, including max and xor, that a single perceptron can't (e.g., https://www.science.org/doi/10.1126/science.aax6239 https://www.science.org/doi/10.1126/science.aax6239 ). In general, one needs a fairly elaborate ANN to approximate the behavior of a single biological neuron. OTOH, a three-layer network is a universal function approximator and RNNs are universal dynamical systems approximators, so they are sort of trivially equivalent.
- csomar 5y ago
  Do you think they have accumulated complexity for no reason. Because usually evolution kicks out things that aren't needed.
  - pishpash 5y ago
    For one they need to engage in metabolism and reproduction, but I'd like to see some argument of how neurons are more complex without those needs, e.g. do they compute some radically different class of functions than typical ANN's do, or require entirely different interconnections, etc.
XnoiVeX 5y ago
I think a lot of people on this thread are missing this critical insight.
- visarga 5y ago
  You can simulate the data processing of a real neuron with 1000 digital ones, a small neural net. I think we read too much into the complexity of biological neurons. Remember they need to do much more than compute signals. They need to self assemble, self replicate and pass through various stages of growth. They need to function for 80-100 years. Many of those neurons and synapses exist only for redundancy and other biological constraints. A digital neuron doesn't care about its physical substrate and can be millions of times faster. They can be copied identically for no cost and cheaply fine-tuned for new tasks. Their architecture and data can evolve much faster than ours, and the physical implementation can remain the same during this process.

You probably already know this (since you wrote "silly thought"), but real-life neurons are ridiculously more complex than simulated "neurons" in an NN. So the analogy doesn't really hold.

pishpash 5y ago

They're more complex in biological construction and in signaling mechanism, but no proof that they are more complex in function.

mattkrause 5y ago
An individual biological neuron can compute a variety of functions, including max and xor, that a single perceptron can't (e.g., https://www.science.org/doi/10.1126/science.aax6239 https://www.science.org/doi/10.1126/science.aax6239 ). In general, one needs a fairly elaborate ANN to approximate the behavior of a single biological neuron. OTOH, a three-layer network is a universal function approximator and RNNs are universal dynamical systems approximators, so they are sort of trivially equivalent.
csomar 5y ago
Do you think they have accumulated complexity for no reason. Because usually evolution kicks out things that aren't needed.
- pishpash 5y ago
  For one they need to engage in metabolism and reproduction, but I'd like to see some argument of how neurons are more complex without those needs, e.g. do they compute some radically different class of functions than typical ANN's do, or require entirely different interconnections, etc.

XnoiVeX 5y ago

I think a lot of people on this thread are missing this critical insight.

visarga 5y ago
You can simulate the data processing of a real neuron with 1000 digital ones, a small neural net. I think we read too much into the complexity of biological neurons. Remember they need to do much more than compute signals. They need to self assemble, self replicate and pass through various stages of growth. They need to function for 80-100 years. Many of those neurons and synapses exist only for redundancy and other biological constraints. A digital neuron doesn't care about its physical substrate and can be millions of times faster. They can be copied identically for no cost and cheaply fine-tuned for new tasks. Their architecture and data can evolve much faster than ours, and the physical implementation can remain the same during this process.