5 ms·

> The work suggests that existing approaches to neural network architecture would benefit from more closely emulating the operation of the brain in this regard.

> The work suggests that existing approaches to neural network architecture would benefit from more closely emulating the operation of the brain in this regard. Does it? My read is that biological neutral architecture is a consequence of biological constraints that artificial NNs mostly don't face, so we don't necessarily need to try to copy it.

kelseyfrog 3y ago
There's a reason airplanes were invented before helicopters.
spenczar5 3y ago
Depends on your goal. If you want to simulate human intelligence, maybe you want to replicate constraints. A fair chunk of AI work boils down to “make something that acts like a human.” On the other end of the spectrum is stuff that is more specialized, like very targeted classifiers; there is no reason to expect those would benefit from this.
- andrewflnr 3y ago
  Eh, maybe. The question there becomes, how much of human behavior is specifically a consequence of the physical constraints of laying out our neurons? I'm guessing not much. The article doesn't mention any change in output prompted by the change in internal architecture, which doesn't really confirm my guess, but doesn't falsify it either.
  - stefs 3y ago
    there's been physics simulation of bipedal models learning to walk via reinforcement learning. the results always were a bit choppy and robotic, until they implemented signals propagation delays. this led to more natural, fluid movements that definitely looked more "human". sorry, can't find the video. personally i absolutely do think that for generating convincingly human-like intelligence you also need some human constraints, otherwise you will get some uncanny valley. another example would be alpha-zeros play style. AIs don't play like humans, they maximize their chance of winning in the long term without going for good looking opportunities that hurt their chances in the long run (like human players do).
    - andrewflnr 3y ago
      Did the bipedal walking people try optimizing for energy instead? If so, and the models were still choppy, maybe that's actually better. I guess I just find the goal of imitating human intelligence including all its mistakes to be a silly goal. The only time you want that instead of an actual human is if you're trying to deceive people into thinking your AI is a human. Otherwise, you just want the correct answer (or, if you're afraid, you want a strictly sub-human intelligence).
- SoftTalker 3y ago
  Has there been research on whether it's possible to similate human intelligence without also getting all of its flaws? Susceptibility to logical fallacies, random difficulty with factual recall, fading memory over time, biases, prejudice, etc.
  - pfsalter 3y ago
    That's something you could philosophise about but not research unless you already have a human level intelligence to test. We won't know if it's even possible to replicate in silicon for a very long time
- stillwithit 3y ago
  No, AI work boils down to “output computation results that humans hallucinate could have been generated by a human”” We’re just normalizing to our innate sensibilities. We have no idea if we’re making intelligence or what that even means as us humans must work within the constraints we evolved into. We have no idea if we generalized consciousness as it could exist across space time. You and I will never exist outside our universe and observe what makes it tick. We’re hanging out on Earth making mannequins talk, hallucinating we’re gods because of it. Humans are artificial intelligence given their lack of direct observation of so much of the universe. Conway came up with his game of life after the universe. Sorry, AI researchers, life, consciousness and visualization were already created by reality. We’re just working on an easy to use Dewey Decimal system to catalog it.
mkoubaa 3y ago
We don't need to copy but I think the hypothesis is that constraints like ours help networks optimize towards outcomes like ours, and it seems to test well in this case.
- andrewflnr 3y ago
  That's true, but it's a quite distinct hypothesis from the bit I quoted.
famouswaffles 3y ago
You're dead on. We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead. Biology is Biology and Silicon is Silicon. Sometimes, Constraints can be just that - Constraints and not some secret sauce.
- tudorw 3y ago
  I'm interested in the 'slower/tougher' route you mentioned, do you recall the final performance, am assuming it was equivalent or worse. Given the vast disparity in power consumption between silicon and meat space it's surely a rich area to explore, especially given the feedback loop present where AI is being effectively used in more accurate observation and interpretation of brain activity.
- kaba0 3y ago
  Also, biological “architecture” is so different compared to silicon design, that it might just the case that different algorithms perform best for the respective design. Biological systems are effectively huge distributed computing systems, where each unit is very weak. Backpropagation probably can’t easily be implemented for that.
tudorw 3y ago
I agree with this, I should have been more specific, I really meant it from the researchers perspective, they are trying to make NN's that more closely model the brain for their brain research, not for AGI, so in that regard, the research was looking at how constraints affect learning, and whether applying constraints might result in a network that more closely mirrors brain activity, for AI purposes, we want smart machines, not necessarily a model of the brain with all it's additional functions aside from 'intelligence' :)

kelseyfrog 3y ago

There's a reason airplanes were invented before helicopters.

spenczar5 3y ago

Depends on your goal. If you want to simulate human intelligence, maybe you want to replicate constraints. A fair chunk of AI work boils down to “make something that acts like a human.” On the other end of the spectrum is stuff that is more specialized, like very targeted classifiers; there is no reason to expect those would benefit from this.

andrewflnr 3y ago
Eh, maybe. The question there becomes, how much of human behavior is specifically a consequence of the physical constraints of laying out our neurons? I'm guessing not much. The article doesn't mention any change in output prompted by the change in internal architecture, which doesn't really confirm my guess, but doesn't falsify it either.
- stefs 3y ago
  there's been physics simulation of bipedal models learning to walk via reinforcement learning. the results always were a bit choppy and robotic, until they implemented signals propagation delays. this led to more natural, fluid movements that definitely looked more "human". sorry, can't find the video. personally i absolutely do think that for generating convincingly human-like intelligence you also need some human constraints, otherwise you will get some uncanny valley. another example would be alpha-zeros play style. AIs don't play like humans, they maximize their chance of winning in the long term without going for good looking opportunities that hurt their chances in the long run (like human players do).
  - andrewflnr 3y ago
    Did the bipedal walking people try optimizing for energy instead? If so, and the models were still choppy, maybe that's actually better. I guess I just find the goal of imitating human intelligence including all its mistakes to be a silly goal. The only time you want that instead of an actual human is if you're trying to deceive people into thinking your AI is a human. Otherwise, you just want the correct answer (or, if you're afraid, you want a strictly sub-human intelligence).
SoftTalker 3y ago
Has there been research on whether it's possible to similate human intelligence without also getting all of its flaws? Susceptibility to logical fallacies, random difficulty with factual recall, fading memory over time, biases, prejudice, etc.
- pfsalter 3y ago
  That's something you could philosophise about but not research unless you already have a human level intelligence to test. We won't know if it's even possible to replicate in silicon for a very long time
stillwithit 3y ago
No, AI work boils down to “output computation results that humans hallucinate could have been generated by a human”” We’re just normalizing to our innate sensibilities. We have no idea if we’re making intelligence or what that even means as us humans must work within the constraints we evolved into. We have no idea if we generalized consciousness as it could exist across space time. You and I will never exist outside our universe and observe what makes it tick. We’re hanging out on Earth making mannequins talk, hallucinating we’re gods because of it. Humans are artificial intelligence given their lack of direct observation of so much of the universe. Conway came up with his game of life after the universe. Sorry, AI researchers, life, consciousness and visualization were already created by reality. We’re just working on an easy to use Dewey Decimal system to catalog it.

mkoubaa 3y ago

We don't need to copy but I think the hypothesis is that constraints like ours help networks optimize towards outcomes like ours, and it seems to test well in this case.

andrewflnr 3y ago
That's true, but it's a quite distinct hypothesis from the bit I quoted.

famouswaffles 3y ago

You're dead on. We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead. Biology is Biology and Silicon is Silicon. Sometimes, Constraints can be just that - Constraints and not some secret sauce.

tudorw 3y ago
I'm interested in the 'slower/tougher' route you mentioned, do you recall the final performance, am assuming it was equivalent or worse. Given the vast disparity in power consumption between silicon and meat space it's surely a rich area to explore, especially given the feedback loop present where AI is being effectively used in more accurate observation and interpretation of brain activity.
kaba0 3y ago
Also, biological “architecture” is so different compared to silicon design, that it might just the case that different algorithms perform best for the respective design. Biological systems are effectively huge distributed computing systems, where each unit is very weak. Backpropagation probably can’t easily be implemented for that.

tudorw 3y ago

I agree with this, I should have been more specific, I really meant it from the researchers perspective, they are trying to make NN's that more closely model the brain for their brain research, not for AGI, so in that regard, the research was looking at how constraints affect learning, and whether applying constraints might result in a network that more closely mirrors brain activity, for AI purposes, we want smart machines, not necessarily a model of the brain with all it's additional functions aside from 'intelligence' :)