3 ms·

What makes you think an ANN has very much in common with neurons at all? The model of activation as a weight times a simple activation function is laughably si

by skohan 4y ago

What makes you think an ANN has very much in common with neurons at all? The model of activation as a weight times a simple activation function is laughably simple compared to what's actually going on in the brain.

epgui 4y ago
Well, the “neurons” in each case are obviously very different, but you can express computations with varying levels of abstraction. Trying to perfectly emulate atom for atom or neuron for neuron will never work, I think we need to move past that. What matters is overall information flow and processing, and that’s an emergent phenomenon for both brain and ANN. Imagine a program written with OOP, and the same program written with FP. The implementation details may be rather different, but this doesn’t prevent it from doing the same thing in a different way. You have to think of artificial neurons and real neurons in a similar way (though they are rather more dissimilar than OOP and FP are to each other). The main problem that backpropagation solves, I believe, is roughly equivalent to feedback loops in real neural networks.
- skohan 4y ago
  Which networks? There are dozens, or maybe hundreds of different mechanisms the brain uses to adapt the signaling of neural networks. Back propagation might resemble some of them, but others not at all. Back propagation assumes a synapse can be modeled as a weight times an input. We know of tons of types of synapse which don't work like that.
  - epgui 4y ago
    Is the question “do all real neural networks work exactly like backpropagation”, or is the question “can real neural networks do backpropagation”? Because you’re pointing to the first, whereas I’m pointing to the second, and the article points to the second as well. I agree with you that not all arrangements of real neurons look like backpropagation: neurons are way messier and more diverse in their configurations and in the way they interact with each other.
    - skohan 4y ago
      Diverse yes, messy no. Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. If your point is, that something approximating back propagation probably happens somewhere in the brain, then I would agree with you. But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence.
      - epgui 4y ago
        > Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. Agree generally > messy no Well, the way I originally meant it was that it's a very high-entropy, chaotic organ, similarly to kidneys. There is order in the chaos, obviously, but the fact that it can work at all is very remarkable (just like kidneys, where you have messy and complex chemical soups and active concentration gradients all over the place in every which way). In fact all of biochemistry and cell biology, generally speaking, is a very remarkable juxtaposition of chaos and order together and at the same time. My area of research was studying cell-derived extracellular vesicles (of which you can count viruses, extracellular mitochondria, exosomes, microparticles, apoptotic bodies and others), which are a vehicle for the horizontal transfer of information (adding a whole other dimension to cell signalling). Studying this is a great way to appreciate the amount of messiness and chaos we're dealing with... both in pathology and in physiologic conditions :) > But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. I mean, I agree that artificial neurons don't look like real neurons, but I prefer to think of the higher-order function, at the level of the organ, or group of neurons, where you get emergent behaviour and where meaningful computations happen. I think the strongest analogies are found when you relax the rules of what you're comparing (to find the most similarities, we should compare groups of artificial neurons to groups of real neurons). > And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence. Strongly, *strongly agree*. I believe if we want to create a computational network that resembles the way a brain works, we need to start modelling our networks like brains at the macro-level (not on a neuron-to-neuron basis of course, but on a neuroanatomic, group-of-neuron basis). When's the last time you've heard of AI researchers modeling a spinal cord, or a pons, or some thalamic nuclei, or specific neocortical functions? I think if you try to compare a layered sandwich of artificial neurons to a brain, you're necessarily bound to find all sorts of important differences in performance or function. Neuroanatomy has a lot yet to learn, but the structure-function relationship is a lesson that was learned a long time ago.

by skohan 4y ago

epgui 4y ago

Well, the “neurons” in each case are obviously very different, but you can express computations with varying levels of abstraction. Trying to perfectly emulate atom for atom or neuron for neuron will never work, I think we need to move past that. What matters is overall information flow and processing, and that’s an emergent phenomenon for both brain and ANN. Imagine a program written with OOP, and the same program written with FP. The implementation details may be rather different, but this doesn’t prevent it from doing the same thing in a different way. You have to think of artificial neurons and real neurons in a similar way (though they are rather more dissimilar than OOP and FP are to each other). The main problem that backpropagation solves, I believe, is roughly equivalent to feedback loops in real neural networks.

skohan 4y ago
Which networks? There are dozens, or maybe hundreds of different mechanisms the brain uses to adapt the signaling of neural networks. Back propagation might resemble some of them, but others not at all. Back propagation assumes a synapse can be modeled as a weight times an input. We know of tons of types of synapse which don't work like that.
- epgui 4y ago
  Is the question “do all real neural networks work exactly like backpropagation”, or is the question “can real neural networks do backpropagation”? Because you’re pointing to the first, whereas I’m pointing to the second, and the article points to the second as well. I agree with you that not all arrangements of real neurons look like backpropagation: neurons are way messier and more diverse in their configurations and in the way they interact with each other.
  - skohan 4y ago
    Diverse yes, messy no. Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. If your point is, that something approximating back propagation probably happens somewhere in the brain, then I would agree with you. But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence.
    - epgui 4y ago
      > Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. Agree generally > messy no Well, the way I originally meant it was that it's a very high-entropy, chaotic organ, similarly to kidneys. There is order in the chaos, obviously, but the fact that it can work at all is very remarkable (just like kidneys, where you have messy and complex chemical soups and active concentration gradients all over the place in every which way). In fact all of biochemistry and cell biology, generally speaking, is a very remarkable juxtaposition of chaos and order together and at the same time. My area of research was studying cell-derived extracellular vesicles (of which you can count viruses, extracellular mitochondria, exosomes, microparticles, apoptotic bodies and others), which are a vehicle for the horizontal transfer of information (adding a whole other dimension to cell signalling). Studying this is a great way to appreciate the amount of messiness and chaos we're dealing with... both in pathology and in physiologic conditions :) > But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. I mean, I agree that artificial neurons don't look like real neurons, but I prefer to think of the higher-order function, at the level of the organ, or group of neurons, where you get emergent behaviour and where meaningful computations happen. I think the strongest analogies are found when you relax the rules of what you're comparing (to find the most similarities, we should compare groups of artificial neurons to groups of real neurons). > And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence. Strongly, *strongly agree*. I believe if we want to create a computational network that resembles the way a brain works, we need to start modelling our networks like brains at the macro-level (not on a neuron-to-neuron basis of course, but on a neuroanatomic, group-of-neuron basis). When's the last time you've heard of AI researchers modeling a spinal cord, or a pons, or some thalamic nuclei, or specific neocortical functions? I think if you try to compare a layered sandwich of artificial neurons to a brain, you're necessarily bound to find all sorts of important differences in performance or function. Neuroanatomy has a lot yet to learn, but the structure-function relationship is a lesson that was learned a long time ago.