4 ms·

Which networks? There are dozens, or maybe hundreds of different mechanisms the brain uses to adapt the signaling of neural networks. Back propagation might re

by skohan 4y ago

Which networks? There are dozens, or maybe hundreds of different mechanisms the brain uses to adapt the signaling of neural networks. Back propagation might resemble some of them, but others not at all. Back propagation assumes a synapse can be modeled as a weight times an input. We know of tons of types of synapse which don't work like that.

epgui 4y ago
Is the question “do all real neural networks work exactly like backpropagation”, or is the question “can real neural networks do backpropagation”? Because you’re pointing to the first, whereas I’m pointing to the second, and the article points to the second as well. I agree with you that not all arrangements of real neurons look like backpropagation: neurons are way messier and more diverse in their configurations and in the way they interact with each other.
- skohan 4y ago
  Diverse yes, messy no. Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. If your point is, that something approximating back propagation probably happens somewhere in the brain, then I would agree with you. But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence.
  - epgui 4y ago
    > Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. Agree generally > messy no Well, the way I originally meant it was that it's a very high-entropy, chaotic organ, similarly to kidneys. There is order in the chaos, obviously, but the fact that it can work at all is very remarkable (just like kidneys, where you have messy and complex chemical soups and active concentration gradients all over the place in every which way). In fact all of biochemistry and cell biology, generally speaking, is a very remarkable juxtaposition of chaos and order together and at the same time. My area of research was studying cell-derived extracellular vesicles (of which you can count viruses, extracellular mitochondria, exosomes, microparticles, apoptotic bodies and others), which are a vehicle for the horizontal transfer of information (adding a whole other dimension to cell signalling). Studying this is a great way to appreciate the amount of messiness and chaos we're dealing with... both in pathology and in physiologic conditions :) > But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. I mean, I agree that artificial neurons don't look like real neurons, but I prefer to think of the higher-order function, at the level of the organ, or group of neurons, where you get emergent behaviour and where meaningful computations happen. I think the strongest analogies are found when you relax the rules of what you're comparing (to find the most similarities, we should compare groups of artificial neurons to groups of real neurons). > And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence. Strongly, *strongly agree*. I believe if we want to create a computational network that resembles the way a brain works, we need to start modelling our networks like brains at the macro-level (not on a neuron-to-neuron basis of course, but on a neuroanatomic, group-of-neuron basis). When's the last time you've heard of AI researchers modeling a spinal cord, or a pons, or some thalamic nuclei, or specific neocortical functions? I think if you try to compare a layered sandwich of artificial neurons to a brain, you're necessarily bound to find all sorts of important differences in performance or function. Neuroanatomy has a lot yet to learn, but the structure-function relationship is a lesson that was learned a long time ago.
    - epgui 4y ago
      > to find the most similarities, we should compare groups of artificial neurons to groups of real neurons And to be clear, in brains there are very few structures that are self-defined and not connected to a whole lot of other inputs and outputs... So while the group-to-group comparison may be broadly similar at the core, the comparison will break down at the boundaries. In other words, the brain is a very side-effecting computational structure, and it is difficult (if not impossible) to isolate a purely-functional set of neurons with clearly defined inputs and outputs.

by skohan 4y ago

epgui 4y ago

Is the question “do all real neural networks work exactly like backpropagation”, or is the question “can real neural networks do backpropagation”? Because you’re pointing to the first, whereas I’m pointing to the second, and the article points to the second as well. I agree with you that not all arrangements of real neurons look like backpropagation: neurons are way messier and more diverse in their configurations and in the way they interact with each other.

skohan 4y ago
Diverse yes, messy no. Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. If your point is, that something approximating back propagation probably happens somewhere in the brain, then I would agree with you. But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence.
- epgui 4y ago
  > Brain structure is extremely specific and tailored to function. Actually "messy" neural structure is tied to serious disorders like schizophrenia. Agree generally > messy no Well, the way I originally meant it was that it's a very high-entropy, chaotic organ, similarly to kidneys. There is order in the chaos, obviously, but the fact that it can work at all is very remarkable (just like kidneys, where you have messy and complex chemical soups and active concentration gradients all over the place in every which way). In fact all of biochemistry and cell biology, generally speaking, is a very remarkable juxtaposition of chaos and order together and at the same time. My area of research was studying cell-derived extracellular vesicles (of which you can count viruses, extracellular mitochondria, exosomes, microparticles, apoptotic bodies and others), which are a vehicle for the horizontal transfer of information (adding a whole other dimension to cell signalling). Studying this is a great way to appreciate the amount of messiness and chaos we're dealing with... both in pathology and in physiologic conditions :) > But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all. I mean, I agree that artificial neurons don't look like real neurons, but I prefer to think of the higher-order function, at the level of the organ, or group of neurons, where you get emergent behaviour and where meaningful computations happen. I think the strongest analogies are found when you relax the rules of what you're comparing (to find the most similarities, we should compare groups of artificial neurons to groups of real neurons). > And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence. Strongly, *strongly agree*. I believe if we want to create a computational network that resembles the way a brain works, we need to start modelling our networks like brains at the macro-level (not on a neuron-to-neuron basis of course, but on a neuroanatomic, group-of-neuron basis). When's the last time you've heard of AI researchers modeling a spinal cord, or a pons, or some thalamic nuclei, or specific neocortical functions? I think if you try to compare a layered sandwich of artificial neurons to a brain, you're necessarily bound to find all sorts of important differences in performance or function. Neuroanatomy has a lot yet to learn, but the structure-function relationship is a lesson that was learned a long time ago.
  - epgui 4y ago
    > to find the most similarities, we should compare groups of artificial neurons to groups of real neurons And to be clear, in brains there are very few structures that are self-defined and not connected to a whole lot of other inputs and outputs... So while the group-to-group comparison may be broadly similar at the core, the comparison will break down at the boundaries. In other words, the brain is a very side-effecting computational structure, and it is difficult (if not impossible) to isolate a purely-functional set of neurons with clearly defined inputs and outputs.