3 ms·

There is nothing here that applies particularly to intelligence. A universal Turing machine that "contains all Turing machines" already exists; it's called a co

by pron 5y ago

There is nothing here that applies particularly to intelligence. A universal Turing machine that "contains all Turing machines" already exists; it's called a computer. So yes, computers cannot be "contained," but what does it have to do with intelligence? Even their definition is merely the definition of a general-purpose computer attached to some sensors and actuators: > A superintelligent machine is a programmable machine with a program R, that receives input D from the external world (the state of the world), and is able to act on the external world as a function of the output of its program R(D). The program in this machine must be able to simulate the behavior of a universal Turing machine. There is nothing in the paper that distinguishes between a "superintelligent machine" and my laptop. What has always fascinated me in discussions about the safety and power of artificial intelligence (or "strong" AI as it has unfortunately come to be called [1]) is that the people who are obsessed with the problem are those who like to believe that intelligence carries much more power than the evidence suggests. Over the history of all the intelligent species we know, much more power has been yielded by unintelligent beings (microorganisms and even insects); even the presence agency does not seem to play a role in the potential for harm. Within human society itself, other traits, such as charisma, seem to be correlated with power, and especially harm, much more than intelligence. I think that the reason a certain group of people chooses to focus particularly on the dangers of intelligence [2] have a power fantasy about intelligence because they believe it to be their own extraordinary trait. [1]: Not to be confused with the proven dangers of statistical inference methods that are sometimes called AI these days. [2]: We have no idea what "superintelligence" is, whether or not it could exist (that question is separate from the question of whether human-level intelligence could be achieved in a mechanical computer, to which the answer is most likely yes), what it could do that mere "ordinary" intelligence could not, or even if the term can be meaningfully defined at all.

akyu 5y ago
>A universal Turing machine that "contains all Turing machines" already exists You are misquoting the paper here Quoting the abstract: >Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world... Note the difference between "programs" and "Turing Machines".
- pron 5y ago
  > Note the difference between "programs" and "Turing Machines". There is no such real difference. Such programs also exist, and they're called interpreters.
mrtranscendence 5y ago
> I think that the reason a certain group of people chooses to focus particularly on the dangers of intelligence [2] have a power fantasy about intelligence because they believe it to be their own extraordinary trait. I'm not sure. As a source of problem-solving power intelligence is universal in a way that charisma is not. If a problem did call for charisma, a sufficiently intelligent being would know precisely what to say and how to behave so as to mimic charisma. On the other hand charisma is useless in a variety of contexts. That's not to say intelligence is more important than charisma to 21st century humans, though I'd point out that charisma relies on intelligence to some degree; there's a limit to how charismatic someone can be with profound intellectual disability. But it's definitely true that no thoroughly acharismatic but intelligent human would be intelligent enough to mimic charisma. But we're talking about something that's more intelligent than a human, at least.
- pron 5y ago
  > As a source of problem-solving power intelligence is universal in a way that charisma is not. If a problem did call for charisma, a sufficiently intelligent being would know precisely what to say and how to behave so as to mimic charisma. But that defines "intelligence" as a general problem-solving power, and that is probably wrong. There are many problems that require computational power that we simply don't have (even collectively, as humanity with all its technology) despite our intelligence. For example, to predict the weather further ahead (or, say, the behaviour of society assuming we had some mathematical model), doesn't require more intelligence, but exponentially more computational resources. Moreover, I don't know what "sufficiently intelligent" means, but I certainly don't see any correlation between intelligence and charisma in humans, so that claim rests on a fictional story about what "superintelligence" is, rather than on any evidence about the intelligence we already know. > But we're talking about something that's more intelligent than a human, at least. While I certainly accept that a machine that's as intelligent as humans is possible (although I have no idea if we're 50 or 100 years away from achieving it), we don't know what "more intelligent" even means, let alone that it's possible. For all we know, a machine that's as intelligent as humans but just processes faster will be more prone to depression. Also, problem-solving has rarely been a bottleneck. What we lack is resources. For example, physicists now believe that to answer some questions in physics what we need isn't smarter physicists (assuming such a thing is possible), but resources to build ever larger particle accelerators. The belief that "more intelligence" -- if it means anything at all -- equals more power simply goes against all the evidence we have. It currently rests on nothing more than science fiction and the power fantasies expressed in science fiction, that is both written and targeted at people who wish intelligence translated to power.

by pron 5y ago

akyu 5y ago

>A universal Turing machine that "contains all Turing machines" already exists You are misquoting the paper here Quoting the abstract: >Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world... Note the difference between "programs" and "Turing Machines".

pron 5y ago
> Note the difference between "programs" and "Turing Machines". There is no such real difference. Such programs also exist, and they're called interpreters.

mrtranscendence 5y ago

> I think that the reason a certain group of people chooses to focus particularly on the dangers of intelligence [2] have a power fantasy about intelligence because they believe it to be their own extraordinary trait. I'm not sure. As a source of problem-solving power intelligence is universal in a way that charisma is not. If a problem did call for charisma, a sufficiently intelligent being would know precisely what to say and how to behave so as to mimic charisma. On the other hand charisma is useless in a variety of contexts. That's not to say intelligence is more important than charisma to 21st century humans, though I'd point out that charisma relies on intelligence to some degree; there's a limit to how charismatic someone can be with profound intellectual disability. But it's definitely true that no thoroughly acharismatic but intelligent human would be intelligent enough to mimic charisma. But we're talking about something that's more intelligent than a human, at least.

pron 5y ago
> As a source of problem-solving power intelligence is universal in a way that charisma is not. If a problem did call for charisma, a sufficiently intelligent being would know precisely what to say and how to behave so as to mimic charisma. But that defines "intelligence" as a general problem-solving power, and that is probably wrong. There are many problems that require computational power that we simply don't have (even collectively, as humanity with all its technology) despite our intelligence. For example, to predict the weather further ahead (or, say, the behaviour of society assuming we had some mathematical model), doesn't require more intelligence, but exponentially more computational resources. Moreover, I don't know what "sufficiently intelligent" means, but I certainly don't see any correlation between intelligence and charisma in humans, so that claim rests on a fictional story about what "superintelligence" is, rather than on any evidence about the intelligence we already know. > But we're talking about something that's more intelligent than a human, at least. While I certainly accept that a machine that's as intelligent as humans is possible (although I have no idea if we're 50 or 100 years away from achieving it), we don't know what "more intelligent" even means, let alone that it's possible. For all we know, a machine that's as intelligent as humans but just processes faster will be more prone to depression. Also, problem-solving has rarely been a bottleneck. What we lack is resources. For example, physicists now believe that to answer some questions in physics what we need isn't smarter physicists (assuming such a thing is possible), but resources to build ever larger particle accelerators. The belief that "more intelligence" -- if it means anything at all -- equals more power simply goes against all the evidence we have. It currently rests on nothing more than science fiction and the power fantasies expressed in science fiction, that is both written and targeted at people who wish intelligence translated to power.