3 ms·

>Biological neurons and ANN neurons aren’t the same or even all that similar. They're not the same but they're definitely similar. >Brains don’t do backprop f

>Biological neurons and ANN neurons aren’t the same or even all that similar. They're not the same but they're definitely similar. >Brains don’t do backprop for example. We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead. The only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it. The secret of bird flight was wings. Not feathers. Not flapping.

tsimionescu 3y ago
> The only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it. And yet, there's no ANN that's as good at interacting with the real world as the simplest worms we've studied, despite having many times more neurons than those worms have cells. We are clearly still missing some key pieces of the puzzle for intelligence, so claiming that the difference between ANNs and biological neurons is irrelevant is quite premature. We are far away from having an airfoil moment in AI research.
- FeepingCreature 3y ago
  Simple organisms are not necessarily easier to imitate than more complex ones. Our vehicles can drive faster than the fastest animals, but no car can absorb nutrients across a cell membrane. The simpler a system is, the more an imitation has to stick to the specifics to the system's particular implementation. Larger, more complex systems can usually perform more abstract tasks, which allows more diverse approaches.
- ben_w 3y ago
  I assert that most robotics AI, including self driving cars, are at least as good at interacting with the real world as the simplest worms we've studied. Do different things, but at least as good.
janalsncm 3y ago
My point isn’t about whether ANNs work, it’s that they’re so fundamentally different (both locally at the level of the neuron and globally at the level of the brain) that calling them both “neurons” is pretty imprecise. Silicon simulations of brains may “suffer” from being faithful but this also discounts the advantages that brains have. As I mentioned for example, brains can repurpose neurons for other tasks. Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. If I ask you to study a textbook on cats you won’t suddenly forget the difference between cats and dogs.
- famouswaffles 3y ago
  >Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. There is not a single brain on earth that is the blank slate a typical ANN is. "Brains generalize from one example" is pretty dubious. Millions of years of evolution matter. >As I mentioned for example, brains can repurpose neurons for other tasks. Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation ? >Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. This suggests CF may well be a simple matter of scale - https://palm-e.github.io/ https://palm-e.github.io/ Since individual anns are much closer to synapses, we don't have anything near the scale of the brain yet.
  - janalsncm 3y ago
    > There is not a single brain on earth that is the blank slate a typical ANN is. Of course structure matters, but biological neurons have far more degrees of freedom than those in ANNs. The fact that we even need to keep differentiating between the two is an indication that classifying both as “neurons” is not accurate. > Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation? It’s a difference in capabilities of the things themselves. A biological neuron organically seeks out new connections. Sure we could program that into an ANN somehow but the fact that nodes in an ANN don’t have this capability out of the box is a fundamental difference. > CF may well be a simple matter of scale For a moment, a big enough network might be able to mirror an entire brain with the lottery ticket hypothesis. But if it takes two or ten or a thousand ANN neurons to simulate the degrees of freedom of a biological neuron, are they really the same?

tsimionescu 3y ago

> The only reason modern NNs aren't closer to their biological counterparts is because they genuinely suffer for it. And yet, there's no ANN that's as good at interacting with the real world as the simplest worms we've studied, despite having many times more neurons than those worms have cells. We are clearly still missing some key pieces of the puzzle for intelligence, so claiming that the difference between ANNs and biological neurons is irrelevant is quite premature. We are far away from having an airfoil moment in AI research.

FeepingCreature 3y ago
Simple organisms are not necessarily easier to imitate than more complex ones. Our vehicles can drive faster than the fastest animals, but no car can absorb nutrients across a cell membrane. The simpler a system is, the more an imitation has to stick to the specifics to the system's particular implementation. Larger, more complex systems can usually perform more abstract tasks, which allows more diverse approaches.
ben_w 3y ago
I assert that most robotics AI, including self driving cars, are at least as good at interacting with the real world as the simplest worms we've studied. Do different things, but at least as good.

janalsncm 3y ago

My point isn’t about whether ANNs work, it’s that they’re so fundamentally different (both locally at the level of the neuron and globally at the level of the brain) that calling them both “neurons” is pretty imprecise. Silicon simulations of brains may “suffer” from being faithful but this also discounts the advantages that brains have. As I mentioned for example, brains can repurpose neurons for other tasks. Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. If I ask you to study a textbook on cats you won’t suddenly forget the difference between cats and dogs.

famouswaffles 3y ago
>Brains can also generalize from a single example, unlike neural networks which require thousands if not millions of examples. There is not a single brain on earth that is the blank slate a typical ANN is. "Brains generalize from one example" is pretty dubious. Millions of years of evolution matter. >As I mentioned for example, brains can repurpose neurons for other tasks. Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation ? >Brains also generally do not suffer from catastrophic forgetting in the same way that our simulations tend to. This suggests CF may well be a simple matter of scale - https://palm-e.github.io/ https://palm-e.github.io/ Since individual anns are much closer to synapses, we don't have anything near the scale of the brain yet.
- janalsncm 3y ago
  > There is not a single brain on earth that is the blank slate a typical ANN is. Of course structure matters, but biological neurons have far more degrees of freedom than those in ANNs. The fact that we even need to keep differentiating between the two is an indication that classifying both as “neurons” is not accurate. > Isn't this just a matter of the practical distinction between training and inference and not some fundamental structural limitation? It’s a difference in capabilities of the things themselves. A biological neuron organically seeks out new connections. Sure we could program that into an ANN somehow but the fact that nodes in an ANN don’t have this capability out of the box is a fundamental difference. > CF may well be a simple matter of scale For a moment, a big enough network might be able to mirror an entire brain with the lottery ticket hypothesis. But if it takes two or ten or a thousand ANN neurons to simulate the degrees of freedom of a biological neuron, are they really the same?