5 ms·

It's unclear that it's even possible to emulate general intelligence by computable functions, let alone that it's possible to improve it to superhuman capacity.

by atroyn 9y ago

It's unclear that it's even possible to emulate general intelligence by computable functions, let alone that it's possible to improve it to superhuman capacity. There are clear and present threats to civilization needing to be dealt with - superhuman A.I is, to quote Maciej/Pinboard, the 'Idea that eats smart people'.

maldeh 9y ago
My takeaway is that there's a more pressing risk from humans automating too much too fast with something approximate and suboptimal, but "good enough". There're already plenty of situations where city planners enforce macroscopic policies that result in widespread discrimination and such, and that was with best intents. The promise of AI is that entire horizons of industry open up with opportunities for automation, making us tackle historically difficult-to-get-right objective functions, potentially with bad macroscopic results.
- atroyn 9y ago
  This is a whole other set of problems, which I agree will only get worse. However they don't really represent the threat Musk discusses - superhuman / superintelligent paperclip maximizers and other Yudkowskian nonsense.
emmab 9y ago
> It's unclear that it's even possible to emulate general intelligence by computable functions, let alone that it's possible to improve it to superhuman capacity. Rejecting the physical Church-Turing thesis is not really plausible. https://plato.stanford.edu/entries/church-turing/ https://plato.stanford.edu/entries/church-turing/
- atroyn 9y ago
  This is the wrong way around. It's possible for a physical system to perform computation. It's not necessarily true that all physical systems can be emulated by computation.
simonh 9y ago
>It's unclear that it's even possible to emulate general intelligence by computable functions You've been reading Searle and Penrose. I can tell. Their proofs are based on the assumption that any AI must be a consistent system built using only computable functions. Have you ever met a human mind that was completely consistent? I Haven't. It's easy to set up a straw man to get demolished if you get to design the exact properties of the straw, flaws and all. Of course who would imagine that perfect internal consistency would be a flaw? But then again why should we assume that it's a prerequisite of artificial intelligence if it isn't for humans?
- atroyn 9y ago
  I came to my conclusion independently by observing that computing the time evolution of most physical systems to arbitrary precision is impossible in finite time. More formally, the state space grows much much faster than polynomial time. Finding out if we can do better with quantum computing is an active area of research. I haven't read Searle/Penrose
  - simonh 9y ago
    If humans can't do that either, and we cant't, why would you conclude that it's necessary to be able to do that in order to match human intelligence? Or are you specifically talking about perfectly simulating human brains? Human brain emulations are only one very specific and narrow form a strong AI might take. But even in that specific subset of possible AIs, we have no real idea how precise the simulation might have to be. It might be perfectly acievable without even simulating individual molecules.
    - atroyn 9y ago
      I don't agree with your assertion that humans can't do that. Whether or not human cognition is a superset of computation is an unanswered question. That aside, even if human cognition is a computable function, there are no guarantees that the physical process giving rise to human cognition is computable, nor that any process giving rise to cognition is computable.
sullyj3 9y ago
Suppose we grant the premise that human intelligence is doing something over and above computing functions in the Turing sense (I'm personally not convinced that anything in the universe can do that, but that's beside the point). That still doesn't preclude the possibility of artificial intelligence. Why? Because, you can't claim with a straight face that human brains are the only arrangements of matter that physics will grant general intelligence. That would just be too insanely arbitrary. There are a lot of possible arrangements of matter. You would expect, a priori, for a tiny proportion of those arrangements of matter to be smart. But a small fraction of an enormous, combinatorially large number is still combinatorially large. Note that consciousness isn't necessary or relevant for the purposes of this argument. We only care about what the AI is capable of, whether it's experiencing qualia while it's taking over the world or whatever is immaterial. Once you grant that there are other possible configurations of matter that could be generally intelligent, the only other thing you have to grant is that humanity will one day be capable of manufacturing some of them. Assuming a wide variety of possible arrangements, it would seem unlikely that there was some fundamental law of the universe that prevented us from manufacturing intelligence.
- atroyn 9y ago
  But what characterizes those other arrangements of matter? What is their energy consumption? What are their switching speeds? Can cognition grow unbounded, and if so, why? Can any brain improve on its own process of cognition? If other arrangements of matter perform cognition equally well, why don't we see them in the independent evolution of life with wildly different structures for everything else? There is far too much woo and not enough rigorous thinking around the notion of AI superintelligence, singularity, or whatever other incantation people care to use.
  - sullyj3 9y ago
    > why don't we see them in the independent evolution of life Because evolution is a very weak optimisation process. It has very limited materials to work with - everything has to be made of meat. The only changes it can make are incredibly indirect ones - mutations in DNA base pairs, leading to different proteins being manufactured or some such (I'm not a biologist, obviously) leading to subtle changes in the makeup of the resulting organism. Can you imagine how difficult engineering would be through a medium like that? It'd be like trying to perform brain surgery with a sledgehammer. And it's not even purposefully optimising for anything, let alone intelligence. The changes all happen completely at random, and the ones that manage to reproduce stick around. Moreover, the fact that such a weak process as natural selection can produce something as smart as us, should really make you consider that humans, with optimisation power far superior to natural selection, can produce something far smarter.
sullyj3 9y ago
You can attack the idea more directly as well. Whilst we don't have machines running computable functions that exhibit general intelligence, we unequivocally have ones that exhibit narrow intelligence. Things like telling the difference between cats and dogs by looking at photos, playing superhuman go, learning to play Mario, etc. This gives us an indication that computable functions are capable of a very broad range of interesting behaviours. So then, the question becomes, what is the fundamental, lawful difference between narrow and general intelligence, that would cause effectively computable functions to be able to perform behaviours that we classify as narrow, but not broad ones? This would also seem surprising and arbitrary a priori.

by atroyn 9y ago

maldeh 9y ago

My takeaway is that there's a more pressing risk from humans automating too much too fast with something approximate and suboptimal, but "good enough". There're already plenty of situations where city planners enforce macroscopic policies that result in widespread discrimination and such, and that was with best intents. The promise of AI is that entire horizons of industry open up with opportunities for automation, making us tackle historically difficult-to-get-right objective functions, potentially with bad macroscopic results.

atroyn 9y ago
This is a whole other set of problems, which I agree will only get worse. However they don't really represent the threat Musk discusses - superhuman / superintelligent paperclip maximizers and other Yudkowskian nonsense.

emmab 9y ago

> It's unclear that it's even possible to emulate general intelligence by computable functions, let alone that it's possible to improve it to superhuman capacity. Rejecting the physical Church-Turing thesis is not really plausible. https://plato.stanford.edu/entries/church-turing/ https://plato.stanford.edu/entries/church-turing/

atroyn 9y ago
This is the wrong way around. It's possible for a physical system to perform computation. It's not necessarily true that all physical systems can be emulated by computation.

simonh 9y ago

>It's unclear that it's even possible to emulate general intelligence by computable functions You've been reading Searle and Penrose. I can tell. Their proofs are based on the assumption that any AI must be a consistent system built using only computable functions. Have you ever met a human mind that was completely consistent? I Haven't. It's easy to set up a straw man to get demolished if you get to design the exact properties of the straw, flaws and all. Of course who would imagine that perfect internal consistency would be a flaw? But then again why should we assume that it's a prerequisite of artificial intelligence if it isn't for humans?

atroyn 9y ago
I came to my conclusion independently by observing that computing the time evolution of most physical systems to arbitrary precision is impossible in finite time. More formally, the state space grows much much faster than polynomial time. Finding out if we can do better with quantum computing is an active area of research. I haven't read Searle/Penrose
- simonh 9y ago
  If humans can't do that either, and we cant't, why would you conclude that it's necessary to be able to do that in order to match human intelligence? Or are you specifically talking about perfectly simulating human brains? Human brain emulations are only one very specific and narrow form a strong AI might take. But even in that specific subset of possible AIs, we have no real idea how precise the simulation might have to be. It might be perfectly acievable without even simulating individual molecules.
  - atroyn 9y ago
    I don't agree with your assertion that humans can't do that. Whether or not human cognition is a superset of computation is an unanswered question. That aside, even if human cognition is a computable function, there are no guarantees that the physical process giving rise to human cognition is computable, nor that any process giving rise to cognition is computable.

sullyj3 9y ago

Suppose we grant the premise that human intelligence is doing something over and above computing functions in the Turing sense (I'm personally not convinced that anything in the universe can do that, but that's beside the point). That still doesn't preclude the possibility of artificial intelligence. Why? Because, you can't claim with a straight face that human brains are the only arrangements of matter that physics will grant general intelligence. That would just be too insanely arbitrary. There are a lot of possible arrangements of matter. You would expect, a priori, for a tiny proportion of those arrangements of matter to be smart. But a small fraction of an enormous, combinatorially large number is still combinatorially large. Note that consciousness isn't necessary or relevant for the purposes of this argument. We only care about what the AI is capable of, whether it's experiencing qualia while it's taking over the world or whatever is immaterial. Once you grant that there are other possible configurations of matter that could be generally intelligent, the only other thing you have to grant is that humanity will one day be capable of manufacturing some of them. Assuming a wide variety of possible arrangements, it would seem unlikely that there was some fundamental law of the universe that prevented us from manufacturing intelligence.

atroyn 9y ago
But what characterizes those other arrangements of matter? What is their energy consumption? What are their switching speeds? Can cognition grow unbounded, and if so, why? Can any brain improve on its own process of cognition? If other arrangements of matter perform cognition equally well, why don't we see them in the independent evolution of life with wildly different structures for everything else? There is far too much woo and not enough rigorous thinking around the notion of AI superintelligence, singularity, or whatever other incantation people care to use.
- sullyj3 9y ago
  > why don't we see them in the independent evolution of life Because evolution is a very weak optimisation process. It has very limited materials to work with - everything has to be made of meat. The only changes it can make are incredibly indirect ones - mutations in DNA base pairs, leading to different proteins being manufactured or some such (I'm not a biologist, obviously) leading to subtle changes in the makeup of the resulting organism. Can you imagine how difficult engineering would be through a medium like that? It'd be like trying to perform brain surgery with a sledgehammer. And it's not even purposefully optimising for anything, let alone intelligence. The changes all happen completely at random, and the ones that manage to reproduce stick around. Moreover, the fact that such a weak process as natural selection can produce something as smart as us, should really make you consider that humans, with optimisation power far superior to natural selection, can produce something far smarter.

sullyj3 9y ago

You can attack the idea more directly as well. Whilst we don't have machines running computable functions that exhibit general intelligence, we unequivocally have ones that exhibit narrow intelligence. Things like telling the difference between cats and dogs by looking at photos, playing superhuman go, learning to play Mario, etc. This gives us an indication that computable functions are capable of a very broad range of interesting behaviours. So then, the question becomes, what is the fundamental, lawful difference between narrow and general intelligence, that would cause effectively computable functions to be able to perform behaviours that we classify as narrow, but not broad ones? This would also seem surprising and arbitrary a priori.