6 ms·

> There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks What evidence can you provide

by davedx 2y ago

> There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks What evidence can you provide to back up the statement of this "significant possibility"? Human brains use neural networks...

The_Colonel 2y ago
Neural networks in machine learning bear only a surface level similarity to human brain structure.
- whimsicalism 2y ago
  do you all not see how this is a completely different question?
  - janalsncm 2y ago
    It seems to be intrinsically related. The argument goes something like: 1. Humans have general intelligence. 2. Human brains use biological neurons. 3. Human biological neurons give rise to human general intelligence. 4. Artificial neural networks (ANNs) are similar to human brains. 5. Therefore an ANN could give rise to artificial general intelligence. Many people are objecting to #4 here. However in writing this out, I think #3 is suspect as well: many animals who do not have general intelligence have biologically identical neurons, and although they have clear structural differences with humans, we don’t know how that leads to general intelligence. We could also criticize #1 as well, since human brains are pretty bad at certain things like memorization or calculation. Therefore if we built an ANN with only human capabilities it should also have those weaknesses.
- davedx 2y ago
  Physically, sure. But 1) feedback (more synapses/backprop) and 2) connectedness (huge complex graphs) of both produce very similar intelligent (or "pseudo-intelligent" if you like) emergent properties. I'm pretty sure 5 years ago nobody would have believed ANN's could produce something as powerful as ChatGPT.
Yizahi 2y ago
There are two possibilities. 1. Either you are correct and the neural networks humans have are exactly the same or very similar to the programs in the LLMs. Then it will be relatively easy to verify this - just scale one LLN to the human brain neuron count and supposedly it will acquire consciousness and start rapidly learning and creating on its own without prompts. 2. Or what we call neural networks in the computer programs is radically different and or insufficient to create AI. I'm leaning to the second option, just from the very high level and rudimentary reading about current projects. Can be wrong of course. But I have yet to see any paper that refutes option 2, so it means that it is still possible.
- barrell 2y ago
  I agree with your stance - that being said there aren’t two options, one being identical or radically different. It’s not even a gradient between two choices, because there are several dimensions involved and nobody even knows what Superintelligence is anyways. If you wanted to reduce it down, I would say there are two possibilities: 1. Our understanding of Neurel Nets is currently sufficient to recreate intelligence, consciousness, or what have you 2. We’re lacking some understanding critical to intelligence/conciousness. Given that with a mediocre math education and a week you could pretty completely understand all of the math that goes into these neurel nets, I really hope there’s some understand we don’t yet have
  - shwaj 2y ago
    There are layers of abstraction on top of “the math”. The back propagation math for a transformer is no different than for a multi-layer perception, yet a transformer is vastly more capable than a MLP. More to the point, it took a series of non-trivial steps to arrive at the transformer architecture. In other words, understanding the lowest-level math is no guarantee that you understand the whole thing, otherwise the transformer architecture would have been obvious.
    - theGnuMe 2y ago
      We know architecture and training procedures matter in practice. MLPs and transformers are ultimately theoretically equivalent. That means there is an MLP that represent the any function a given transformer can. However, that MLP is hard to identify and train. Also the transformer contains MLPs as well...
waveBidder 2y ago
no, there's really no comparing barely nonlinear algrebra that makes up transformers and the tangled mess that is human neurons. the name is an artifact and a useful bit of salesmanship.
- davedx 2y ago
  Sure, it's a model. But don't we think neural networks and human brains are primarily about their connectedness and feedback mechanisms though? (I did AI and Psychology at degree level, I understand there are definitely also big differences too, like hormones and biological neurones being very async)
  - waveBidder 2y ago
    You could maybe make a case for CNNs, but the fact that they're feed-forward rather than feedback means they're fundamentally representing a different object (CNN is a function, whereas the visual system is a feedback network). Transformers, while not exactly functions, don't have a feedback mechanism similar to e.g. the cortical algorithm or any other neuronal structure I'm aware of. In general, the ML field is less concerned with replicating neural mechanisms than following the objective gradient.
    - davedx 2y ago
      Thanks for the considered answer. What is the cortical algorithm? (Yeah, it's been quite a few years since I did any bio psych...)
      - waveBidder 2y ago
        As far as I understand it, there's a standing hypothesis that cortical columns have a similar structure that is designed to learn arbitrary patterns via predictive coding, and that a lot of human plasticity arises from the interaction and flexibility of these columns. Numenta has attempted to implement a system to this effect (see the wiki page https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://en.wikipedia.org/wiki/Hierarchical_temporal_memory) for quite some time with not particularly much success. Personally I think the kinds of minds we create in silico will end up being very different, because the advantages and disadvantages of the medium are just very different; for example, having a much stronger central processor and much weaker distributed memory, along with specialized precise circuits in addition to probabilistic ones.
consp 2y ago
I would replace "use" with "vaguely look like".
sva_ 2y ago
The neural networks in human brains are very different from artificial neural networks though. In particular, they seem to learn in a very different way than backprop. But there is no reason the company can't come up with a different paradigm.
- whimsicalism 2y ago
  that is very weak evidence for the impossibility claim
  - janalsncm 2y ago
    It was refuting the weak evidence for possibility stated above.
    - whimsicalism 2y ago
      cheers i missed that
- calf 2y ago
  Do we know that? I've seem some articles and lectures this year that kind of almost loosely argue and reach for the notion that "human backprop" happens when we sleep and dream, etc. I know that's handwavy and not rigorous, but who knows what's going on at this point.
  - sva_ 2y ago
    I've only heard of one researcher who believes the brain does something similar to backprop and has gradients, but it sounded extremely handwavy to me. I think it is more likely the brain does something resembling active inference. But I suppose you could say we don't know 100% since we don't fully understand how the brain learns.
semiquaver 2y ago
There’s always a “significant possibility” that something unprecedented will turn out to be infeasible with any particular approach. How could it be otherwise? Smart people have incorrectly believed we were on the precipice of AGI many times in the 80 years that artificial neural networks have been part of the AI toolbox. https://en.m.wikipedia.org/wiki/AI_winter https://en.m.wikipedia.org/wiki/AI_winter
aithrowaway1987 2y ago
There was a very good paper in Nature showing this definitively: https://news.ycombinator.com/item?id=41437933 https://news.ycombinator.com/item?id=41437933 Modern ANN architectures are not actually capable of long-term learning in the same way animals are, even stodgy old dogs that don't learn new tricks. ANNs are not a plausible model for the brain, even if they emulate certain parts of the brain (the cerebellum, but not the cortex) I will add that transformers are not capable of recursion, so it's impossible for them to realistically emulate a pigeon's brain. (you would need millions of layers that "unlink chains of thought" purely by exhaustion)
- whimsicalism 2y ago
  this paper is far from “showing this definitively” even if we bought this negative result as somehow “proving impossibility”, i’m not convinced plasticity is necessary for intelligence huge respect for richard sutton though
  - aithrowaway1987 2y ago
    Isn't "plasticity is not necessary for intelligence" just defining intelligence downwards? It seems like you want to restrict "intelligence" to static knowledge and (apparent) short-term cleverness, but being able to make long-term observation and judgements about a changing world is a necessary component of intelligence in vertebrates. Why exclude that from consideration? More specifically: it is highly implausible that an AI system could learn to improve itself beyond human capability if it does not have long-term plasticity: how would it be able to reflect upon and extend its discoveries if it's not able to learn new things during its operation?
    - scarmig 2y ago
      Anterograde amnesia is a significant disruption of plasticity, and yet people who have it are still intelligent. (That said, I agree plasticity is key to the most powerful systems. A human race with anterograde amnesia would have long ago gone extinct.)
    - 2y ago
layer8 2y ago
Read up on astrocytes.
zeroonetwothree 2y ago
For any technology we haven’t achieved yet there’s some probability we never achieve it (say, at least in the next 100 years). Why would AI be different?
fsndz 2y ago
exactly, we probably can't even build super intelligence. frankly what we need is more useful tools, we have to quit the idea of creating gods: https://medium.com/@fsndzomga/there-will-be-no-agi-d9be9af4428d https://medium.com/@fsndzomga/there-will-be-no-agi-d9be9af44...
otabdeveloper4 2y ago
> Human brains use neural networks... They don't, actually.

by davedx 2y ago

The_Colonel 2y ago

Neural networks in machine learning bear only a surface level similarity to human brain structure.

whimsicalism 2y ago
do you all not see how this is a completely different question?
- janalsncm 2y ago
  It seems to be intrinsically related. The argument goes something like: 1. Humans have general intelligence. 2. Human brains use biological neurons. 3. Human biological neurons give rise to human general intelligence. 4. Artificial neural networks (ANNs) are similar to human brains. 5. Therefore an ANN could give rise to artificial general intelligence. Many people are objecting to #4 here. However in writing this out, I think #3 is suspect as well: many animals who do not have general intelligence have biologically identical neurons, and although they have clear structural differences with humans, we don’t know how that leads to general intelligence. We could also criticize #1 as well, since human brains are pretty bad at certain things like memorization or calculation. Therefore if we built an ANN with only human capabilities it should also have those weaknesses.
davedx 2y ago
Physically, sure. But 1) feedback (more synapses/backprop) and 2) connectedness (huge complex graphs) of both produce very similar intelligent (or "pseudo-intelligent" if you like) emergent properties. I'm pretty sure 5 years ago nobody would have believed ANN's could produce something as powerful as ChatGPT.

Yizahi 2y ago

There are two possibilities. 1. Either you are correct and the neural networks humans have are exactly the same or very similar to the programs in the LLMs. Then it will be relatively easy to verify this - just scale one LLN to the human brain neuron count and supposedly it will acquire consciousness and start rapidly learning and creating on its own without prompts. 2. Or what we call neural networks in the computer programs is radically different and or insufficient to create AI. I'm leaning to the second option, just from the very high level and rudimentary reading about current projects. Can be wrong of course. But I have yet to see any paper that refutes option 2, so it means that it is still possible.

barrell 2y ago
I agree with your stance - that being said there aren’t two options, one being identical or radically different. It’s not even a gradient between two choices, because there are several dimensions involved and nobody even knows what Superintelligence is anyways. If you wanted to reduce it down, I would say there are two possibilities: 1. Our understanding of Neurel Nets is currently sufficient to recreate intelligence, consciousness, or what have you 2. We’re lacking some understanding critical to intelligence/conciousness. Given that with a mediocre math education and a week you could pretty completely understand all of the math that goes into these neurel nets, I really hope there’s some understand we don’t yet have
- shwaj 2y ago
  There are layers of abstraction on top of “the math”. The back propagation math for a transformer is no different than for a multi-layer perception, yet a transformer is vastly more capable than a MLP. More to the point, it took a series of non-trivial steps to arrive at the transformer architecture. In other words, understanding the lowest-level math is no guarantee that you understand the whole thing, otherwise the transformer architecture would have been obvious.
  - theGnuMe 2y ago
    We know architecture and training procedures matter in practice. MLPs and transformers are ultimately theoretically equivalent. That means there is an MLP that represent the any function a given transformer can. However, that MLP is hard to identify and train. Also the transformer contains MLPs as well...

waveBidder 2y ago

no, there's really no comparing barely nonlinear algrebra that makes up transformers and the tangled mess that is human neurons. the name is an artifact and a useful bit of salesmanship.

davedx 2y ago
Sure, it's a model. But don't we think neural networks and human brains are primarily about their connectedness and feedback mechanisms though? (I did AI and Psychology at degree level, I understand there are definitely also big differences too, like hormones and biological neurones being very async)
- waveBidder 2y ago
  You could maybe make a case for CNNs, but the fact that they're feed-forward rather than feedback means they're fundamentally representing a different object (CNN is a function, whereas the visual system is a feedback network). Transformers, while not exactly functions, don't have a feedback mechanism similar to e.g. the cortical algorithm or any other neuronal structure I'm aware of. In general, the ML field is less concerned with replicating neural mechanisms than following the objective gradient.
  - davedx 2y ago
    Thanks for the considered answer. What is the cortical algorithm? (Yeah, it's been quite a few years since I did any bio psych...)
    - waveBidder 2y ago
      As far as I understand it, there's a standing hypothesis that cortical columns have a similar structure that is designed to learn arbitrary patterns via predictive coding, and that a lot of human plasticity arises from the interaction and flexibility of these columns. Numenta has attempted to implement a system to this effect (see the wiki page https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://en.wikipedia.org/wiki/Hierarchical_temporal_memory) for quite some time with not particularly much success. Personally I think the kinds of minds we create in silico will end up being very different, because the advantages and disadvantages of the medium are just very different; for example, having a much stronger central processor and much weaker distributed memory, along with specialized precise circuits in addition to probabilistic ones.

consp 2y ago

I would replace "use" with "vaguely look like".

sva_ 2y ago

The neural networks in human brains are very different from artificial neural networks though. In particular, they seem to learn in a very different way than backprop. But there is no reason the company can't come up with a different paradigm.

whimsicalism 2y ago
that is very weak evidence for the impossibility claim
- janalsncm 2y ago
  It was refuting the weak evidence for possibility stated above.
  - whimsicalism 2y ago
    cheers i missed that
calf 2y ago
Do we know that? I've seem some articles and lectures this year that kind of almost loosely argue and reach for the notion that "human backprop" happens when we sleep and dream, etc. I know that's handwavy and not rigorous, but who knows what's going on at this point.
- sva_ 2y ago
  I've only heard of one researcher who believes the brain does something similar to backprop and has gradients, but it sounded extremely handwavy to me. I think it is more likely the brain does something resembling active inference. But I suppose you could say we don't know 100% since we don't fully understand how the brain learns.

semiquaver 2y ago

There’s always a “significant possibility” that something unprecedented will turn out to be infeasible with any particular approach. How could it be otherwise? Smart people have incorrectly believed we were on the precipice of AGI many times in the 80 years that artificial neural networks have been part of the AI toolbox. https://en.m.wikipedia.org/wiki/AI_winter https://en.m.wikipedia.org/wiki/AI_winter

aithrowaway1987 2y ago

There was a very good paper in Nature showing this definitively: https://news.ycombinator.com/item?id=41437933 https://news.ycombinator.com/item?id=41437933 Modern ANN architectures are not actually capable of long-term learning in the same way animals are, even stodgy old dogs that don't learn new tricks. ANNs are not a plausible model for the brain, even if they emulate certain parts of the brain (the cerebellum, but not the cortex) I will add that transformers are not capable of recursion, so it's impossible for them to realistically emulate a pigeon's brain. (you would need millions of layers that "unlink chains of thought" purely by exhaustion)

whimsicalism 2y ago
this paper is far from “showing this definitively” even if we bought this negative result as somehow “proving impossibility”, i’m not convinced plasticity is necessary for intelligence huge respect for richard sutton though
- aithrowaway1987 2y ago
  Isn't "plasticity is not necessary for intelligence" just defining intelligence downwards? It seems like you want to restrict "intelligence" to static knowledge and (apparent) short-term cleverness, but being able to make long-term observation and judgements about a changing world is a necessary component of intelligence in vertebrates. Why exclude that from consideration? More specifically: it is highly implausible that an AI system could learn to improve itself beyond human capability if it does not have long-term plasticity: how would it be able to reflect upon and extend its discoveries if it's not able to learn new things during its operation?
  - scarmig 2y ago
    Anterograde amnesia is a significant disruption of plasticity, and yet people who have it are still intelligent. (That said, I agree plasticity is key to the most powerful systems. A human race with anterograde amnesia would have long ago gone extinct.)
  - 2y ago

layer8 2y ago

Read up on astrocytes.

zeroonetwothree 2y ago

For any technology we haven’t achieved yet there’s some probability we never achieve it (say, at least in the next 100 years). Why would AI be different?

fsndz 2y ago

exactly, we probably can't even build super intelligence. frankly what we need is more useful tools, we have to quit the idea of creating gods: https://medium.com/@fsndzomga/there-will-be-no-agi-d9be9af4428d https://medium.com/@fsndzomga/there-will-be-no-agi-d9be9af44...

otabdeveloper4 2y ago

> Human brains use neural networks... They don't, actually.