4 ms·

I don't doubt that there are many similarities between how machine learning works and how brains work, but this seems like a pretty myopic trend of confirmation

by yosito 9y ago

I don't doubt that there are many similarities between how machine learning works and how brains work, but this seems like a pretty myopic trend of confirmation bias. Neurons and brains are so much more complex than machine learning and it will be really unfortunate if we limit ourselves to the machine learning model in neuroscience.

2bitencryption 9y ago
maybe the functional nodes (wetware neurons vs software neurons) are very different, but it seems like the way they are manipulated via back-propagation, pooling, recurrence, layerings, etc, are similar, right? because, at the end of the day, it's more about how behavior is emergent than how behavior functions physically, I would say. I would guess that if we ever get to some true sci-fi AI "consciousness", it would just be a hyper-scaled version of what we already have. But that's just fun speculation.
- tehsauce 9y ago
  This is not correct, artificial NNs are not functionally related to neurons in the brain. Nothing like back-propagation has been observed in real neurons. The early layers of a CNN may be similar to early perceptrons in the brain, but beyond this any connection between the two is fantasy.
- Retric 9y ago
  Nope. Brains don't use layers or back propagation. CS NN's are really just a cool name for math that has almost nothing in common with how the brain functions. There is a lot of misinformation about how the brain works. For example you see a lot of drawings with different parts of the brain doing different things. However, if you look at an actual brain almost none of this is physically obvious. At best those diagrams show what stops working when that part is damaged, though again plenty of people have very different structures and it still mostly works.
- yosito 9y ago
  Honestly, this is more like having invented solar cells and then saying that that's how trees work. Turning light into energy is just a small part of what trees do, and solar cells are only an approximation of the idea but the actual implementation is very very different from photosynthesis.
- tormeh 9y ago
  Real cells use Hebbian learning, which in some cases is equivalent to back-propagation, but is way less efficient. Otherwise, yes, many of the techniques used in ML is also used in the body. Not only ML, actually, but also electrical engineering and surely many other fields. The more you learn about the body the more machine-like it will look to you. In some respects. In others, it's fucking space technology. Nanobots exist, they are called proteins, and each of your cells have hordes of them, for example.
  - stochastic_monk 9y ago
    The big difference is that biological neurons emit binary outputs. Because of this, there’s no gradient and they can’t be trained by SGD.
    - danharaj 9y ago
      There's phase and frequency information in neural spike trains.
    - yorwba 9y ago
      But biological neurons are also stochastic, and the firing probabilities are continuous, so you can do SGD on them.
  - traject_ 9y ago
    Well, I think it may not just be Hebbian learning but likely Balanced Amplification learning [1] as well which propagates quite a bit faster. Promising paper came out recently echoing this phenomena globally with a model of visual perception in the macaque brain[2]. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2667957/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2667957/ [2] https://www.cell.com/neuron/fulltext/S0896-6273(18)30152-1 https://www.cell.com/neuron/fulltext/S0896-6273(18)30152-1?
eli_gottlieb 9y ago
Yes and no. On the one hand, yes, theoretical neuroscience does use some ideas originating in machine learning and computational statistics. So does computational cognitive science. On the other hand, in these fields we set ourselves a different explanatory task: point to the real brain, the real body, and real behavior, and explain them in their full complexity. When asked, "how does this explain the real world", machine learning feels no more compunction than an aeronautical engineer asked to explain wings. This would be great, if only machine learning actually built its engineering artifacts from solid normative principles! Instead, it mostly just hacks things together, as a result of which, its lack of compunction comes across as a reluctance to stand up to the challenge of explaining any aspect of the real world at all, which becomes embarrassing which the machine learning world begins opining on the nature of intelligence and such. Meanwhile, if you ask cognitive scientists and neuroscientists to opine on the nature of intelligence, we have many fewer cool demos, but many more principles and better scientific evidence.
- pishpash 9y ago
  Hacks as a human-driven search driven by competitive forces to find structures that solve human-like tasks. Though it's not guaranteed to produce biological neurology (unless it is somehow a singular solution), it is at least convergent evolution.

by yosito 9y ago

2bitencryption 9y ago

maybe the functional nodes (wetware neurons vs software neurons) are very different, but it seems like the way they are manipulated via back-propagation, pooling, recurrence, layerings, etc, are similar, right? because, at the end of the day, it's more about how behavior is emergent than how behavior functions physically, I would say. I would guess that if we ever get to some true sci-fi AI "consciousness", it would just be a hyper-scaled version of what we already have. But that's just fun speculation.

tehsauce 9y ago
This is not correct, artificial NNs are not functionally related to neurons in the brain. Nothing like back-propagation has been observed in real neurons. The early layers of a CNN may be similar to early perceptrons in the brain, but beyond this any connection between the two is fantasy.
Retric 9y ago
Nope. Brains don't use layers or back propagation. CS NN's are really just a cool name for math that has almost nothing in common with how the brain functions. There is a lot of misinformation about how the brain works. For example you see a lot of drawings with different parts of the brain doing different things. However, if you look at an actual brain almost none of this is physically obvious. At best those diagrams show what stops working when that part is damaged, though again plenty of people have very different structures and it still mostly works.
yosito 9y ago
Honestly, this is more like having invented solar cells and then saying that that's how trees work. Turning light into energy is just a small part of what trees do, and solar cells are only an approximation of the idea but the actual implementation is very very different from photosynthesis.
tormeh 9y ago
Real cells use Hebbian learning, which in some cases is equivalent to back-propagation, but is way less efficient. Otherwise, yes, many of the techniques used in ML is also used in the body. Not only ML, actually, but also electrical engineering and surely many other fields. The more you learn about the body the more machine-like it will look to you. In some respects. In others, it's fucking space technology. Nanobots exist, they are called proteins, and each of your cells have hordes of them, for example.
- stochastic_monk 9y ago
  The big difference is that biological neurons emit binary outputs. Because of this, there’s no gradient and they can’t be trained by SGD.
  - danharaj 9y ago
    There's phase and frequency information in neural spike trains.
  - yorwba 9y ago
    But biological neurons are also stochastic, and the firing probabilities are continuous, so you can do SGD on them.
- traject_ 9y ago
  Well, I think it may not just be Hebbian learning but likely Balanced Amplification learning [1] as well which propagates quite a bit faster. Promising paper came out recently echoing this phenomena globally with a model of visual perception in the macaque brain[2]. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2667957/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2667957/ [2] https://www.cell.com/neuron/fulltext/S0896-6273(18)30152-1 https://www.cell.com/neuron/fulltext/S0896-6273(18)30152-1?

eli_gottlieb 9y ago

Yes and no. On the one hand, yes, theoretical neuroscience does use some ideas originating in machine learning and computational statistics. So does computational cognitive science. On the other hand, in these fields we set ourselves a different explanatory task: point to the real brain, the real body, and real behavior, and explain them in their full complexity. When asked, "how does this explain the real world", machine learning feels no more compunction than an aeronautical engineer asked to explain wings. This would be great, if only machine learning actually built its engineering artifacts from solid normative principles! Instead, it mostly just hacks things together, as a result of which, its lack of compunction comes across as a reluctance to stand up to the challenge of explaining any aspect of the real world at all, which becomes embarrassing which the machine learning world begins opining on the nature of intelligence and such. Meanwhile, if you ask cognitive scientists and neuroscientists to opine on the nature of intelligence, we have many fewer cool demos, but many more principles and better scientific evidence.

pishpash 9y ago
Hacks as a human-driven search driven by competitive forces to find structures that solve human-like tasks. Though it's not guaranteed to produce biological neurology (unless it is somehow a singular solution), it is at least convergent evolution.