3 ms·

My point isn’t about whether ANNs work, it’s that they’re so fundamentally different (both locally at the level of the neuron and globally at the level of the b

My point isn’t about whether ANNs work, it’s that they’re so fundamentally different (both locally at the level of the neuron and globally at the level of the brain) that calling them both “neurons” is pretty imprecise. Silicon simulations of brains may “suffer” from being faithful but this also discounts the advantages that brains have. As I mentioned for example, brains can repurpose neurons for other tasks. Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. If I ask you to study a textbook on cats you won’t suddenly forget the difference between cats and dogs.

famouswaffles 3y ago
>Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. There is not a single brain on earth that is the blank slate a typical ANN is. "Brains generalize from one example" is pretty dubious. Millions of years of evolution matter. >As I mentioned for example, brains can repurpose neurons for other tasks. Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation ? >Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. This suggests CF may well be a simple matter of scale - https://palm-e.github.io/ https://palm-e.github.io/ Since individual anns are much closer to synapses, we don't have anything near the scale of the brain yet.
- janalsncm 3y ago
  > There is not a single brain on earth that is the blank slate a typical ANN is. Of course structure matters, but biological neurons have far more degrees of freedom than those in ANNs. The fact that we even need to keep differentiating between the two is an indication that classifying both as “neurons” is not accurate. > Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation? It’s a difference in capabilities of the things themselves. A biological neuron organically seeks out new connections. Sure we could program that into an ANN somehow but the fact that nodes in an ANN don’t have this capability out of the box is a fundamental difference. > CF may well be a simple matter of scale For a moment, a big enough network might be able to mirror an entire brain with the lottery ticket hypothesis. But if it takes two or ten or a thousand ANN neurons to simulate the degrees of freedom of a biological neuron, are they really the same?
- freejazz 3y ago
  >There is not a single brain on earth that is the blank slate a typical ANN is. You are the one saying that biological neurons and ANN are similar...
greiskul 3y ago
> Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. Since the other comment already went for the evolution and structure angle, I'll go for the other part. What single example? What test have you seen done on the brain capacity of few weeks old fetuses? Our brains start learning patterns in the world before we are even born. How much input does a baby receive every single second from it's eyes and ears and every other sense? Even when you are "analyzing" a new object for the first time, you receive a continuous stream of sensory input of it. Our brain even requires that to work, if you put a single frame different in a fast enough display, most times you won't even notice the extra frame and your brain will just ignore it.

famouswaffles 3y ago

>Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. There is not a single brain on earth that is the blank slate a typical ANN is. "Brains generalize from one example" is pretty dubious. Millions of years of evolution matter. >As I mentioned for example, brains can repurpose neurons for other tasks. Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation ? >Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. This suggests CF may well be a simple matter of scale - https://palm-e.github.io/ https://palm-e.github.io/ Since individual anns are much closer to synapses, we don't have anything near the scale of the brain yet.

janalsncm 3y ago
> There is not a single brain on earth that is the blank slate a typical ANN is. Of course structure matters, but biological neurons have far more degrees of freedom than those in ANNs. The fact that we even need to keep differentiating between the two is an indication that classifying both as “neurons” is not accurate. > Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation? It’s a difference in capabilities of the things themselves. A biological neuron organically seeks out new connections. Sure we could program that into an ANN somehow but the fact that nodes in an ANN don’t have this capability out of the box is a fundamental difference. > CF may well be a simple matter of scale For a moment, a big enough network might be able to mirror an entire brain with the lottery ticket hypothesis. But if it takes two or ten or a thousand ANN neurons to simulate the degrees of freedom of a biological neuron, are they really the same?
freejazz 3y ago
>There is not a single brain on earth that is the blank slate a typical ANN is. You are the one saying that biological neurons and ANN are similar...

greiskul 3y ago

> Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. Since the other comment already went for the evolution and structure angle, I'll go for the other part. What single example? What test have you seen done on the brain capacity of few weeks old fetuses? Our brains start learning patterns in the world before we are even born. How much input does a baby receive every single second from it's eyes and ears and every other sense? Even when you are "analyzing" a new object for the first time, you receive a continuous stream of sensory input of it. Our brain even requires that to work, if you put a single frame different in a fast enough display, most times you won't even notice the extra frame and your brain will just ignore it.