5 ms·

It's past time to start calling these treatises on hyperintelligence what they are—theology—and treating them with the respect they deserve, which is a lot less

It's past time to start calling these treatises on hyperintelligence what they are—theology—and treating them with the respect they deserve, which is a lot less than they currently get on this site. People have been theorizing about the attributes of the Absolute since forever. Just because you start talking about building a god, rather than positing one already in existence, doesn't make the discussions about the nature of such hypothetical superbeings any more fruitful.

apsec112 6y ago
[deleted]
- heavyset_go 6y ago
  "Argument from fallacy Argument from fallacy is the formal fallacy of analyzing an argument and inferring that, since it contains a fallacy, its conclusion must be false. It is also called argument to logic (argumentum ad logicam), the fallacy fallacy, the fallacist's fallacy, and the bad reasons fallacy." https://en.wikipedia.org/wiki/Argument_from_fallacy https://en.wikipedia.org/wiki/Argument_from_fallacy
  - idlewords 6y ago
    And let us not forget the Argument from Cut and Paste, beloved of this forum.
  - drdeca 6y ago
    Saying that an argument is fallacious isn’t the same thing as saying that because the argument is fallacious, it is therefore wrong. Surely you don’t think that all cases of pointing out that something is a fallacy need to include a disclaimer saying “but of course, that doesn’t in itself imply that the conclusion is wrong”. After all, you did not include such a disclaimer yourself. That being said, it seems that the back and forth here doesn’t really seem to have any statements of the form “X (and also Y), therefore Z”. So, I guess that makes it hard to analyze formally as an argument, as instead much of the things like that being explicitly said, there are things being mentioned, with a number of things left implicit.
- idlewords 6y ago
  I gave a whole talk about how this form of mind wank is theological cosplay, but it may not be Dark Enlightenment enough for your tastes. I'm no Scott Alexander. https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm
  - idlebirds 6y ago
    Interested in setting up a clubhouse on this topic?
  - mistermann 6y ago
    I like this part best: "What I hope I've done today is shown you the dangers of being too smart. Hopefully you'll leave this talk a little dumber than you started it, and be more immune to the seductions of AI that seem to bedevil smarter people."
idlebirds 6y ago
If artificial intelligence is a religion, it is the only religion with a plausible mechanism of action. Building something that is more powerful and intelligent than any human does not look to violate any law of physics; calling such a thing a god does not make it any less possible. We have proof it can exist (as humans are just machines). Both the top AI companies in the world (Deepmind and OpenAI) explicitly are trying to build AGI; The fact that it can be built and people desire to build it makes informed speculation about it useful.
- idlewords 6y ago
  Every religion has an explanation, completely persuasive to the devout, for why it's the only true one.
nomic 6y ago
You should setup a clubhouse with Gwern and hash it out. Would love a causal debate on this topic.
- idlewords 6y ago
  I am a being of pure text, and I suspect the same might be true of Gwern.
layoutIfNeeded 6y ago
How many AIs can dance on the head of a pin?
thesteamboat 6y ago
I have seen you and others make this point in the past, it and always seems equivalent to creationists shouting "Scientists who believe in evolution are the REAL dogmatists who don't want to look at the facts! We are really being persecuted by the religion of Science!" Your talk on this subject (helpfully posted below) only furthers this when you present a variety of arguments of wildly varying strengths against AI, in the same vein as "37 challenges to evolution". It makes me feel like you have some valid criticism and a lot of bad faith argumentation. Sadly, I think this characterization is only mostly unfair rather than entirely so. I enjoy your writing and thoughts, and you speak with clarity, but on this subject I think you have constructed a mold of bad-AI-argumentation that you squeeze all AI-argumentation into and in so doing fail to rebut any of it.
- idlewords 6y ago
  The difference here (that I hardly believe needs explaining) is that the argument for evolution and natural selection is firmly rooted in observed reality. Even if you are a die-hard creationist, you don't dispute the existence of a wealth of evidence ready at hand that can be marshalled in the argument, for or against. We live on a planet teeming with life, and we are all agreed on the problem that needs explaining (we started with hydrogen and got armadillos, what happened?) Compare this to the debates about hyperintelligence and the anticipated behavior of posited hyperintelligent beings. These consist of (1) an argument by extrapolation that machines or other organisms can surpass human intelligence bolted onto (2) endless from-first-principles blathering about how such a posited hyperintelligent entity would behave. It's like looking at the vial of hydrogen in an otherwise empty universe and trying to deduce things about armadillos from it. In fact, it's much worse, since we are trying to infer things about hypothesized beings who are, by definition, beyond the ability of our minds to encompass. This is exactly unlike any scientific discourse, and exactly like deist arguments about the nature of the gods where (1) you first prove a God must exist, from whatever 'unmoved mover' argument you find personally convincing, and then (2) infer a huge mount of information about that God's behavior (omniscient, benevolent, likes justice) through a series of intellectual non sequiturs. The only innovation here is people swearing up and down that we can build the god ourselves. The one thing we know about hyperintelligence, in any form, is that it is fundamentally not something whose behavior and nature we can infer from first principles, for the same reasons your cat will never understand why you didn't get tenure. I see nothing wrong in the intellectual project of trying to frame questions about the nature of intelligence, and how computers can behave in intelligent ways. Nor do I think it's foolish to wonder about what it would take for machine intelligence for arise, or approach even deeper questions, like the physical basis of consciousness. That's not what we see here, though. Rationalists like gwern take the football and run it into the end zone of GAME THEORY, in the end revealing far more about themselves, their anxieties, and their hopes than shedding any light on the world we inhabit together. They are accompanied by a bunch of otherwise smart people who have scared themselves silly by the prospect that we might build these gods by accident, abruptly, and that we are at imminent risk of doing so. And to the extent that large numbers of smart people (including ones with access to great wealth) have bought into what is fundamentally an apocalyptic religious cult, I think we at least need to call it by the right name.
emtel 6y ago
What an odd reply. Gwern doesn't really talk about the attributes of "the Absolute" in this post. He explains why computational complexity arguments against the possibility of smarter-than-human AIs fail. I have read several of your comments about AI, and never once have I felt that you engaged with the substance of the arguments. This seems to be no exception.

apsec112 6y ago

[deleted]

heavyset_go 6y ago
"Argument from fallacy Argument from fallacy is the formal fallacy of analyzing an argument and inferring that, since it contains a fallacy, its conclusion must be false. It is also called argument to logic (argumentum ad logicam), the fallacy fallacy, the fallacist's fallacy, and the bad reasons fallacy." https://en.wikipedia.org/wiki/Argument_from_fallacy https://en.wikipedia.org/wiki/Argument_from_fallacy
- idlewords 6y ago
  And let us not forget the Argument from Cut and Paste, beloved of this forum.
- drdeca 6y ago
  Saying that an argument is fallacious isn’t the same thing as saying that because the argument is fallacious, it is therefore wrong. Surely you don’t think that all cases of pointing out that something is a fallacy need to include a disclaimer saying “but of course, that doesn’t in itself imply that the conclusion is wrong”. After all, you did not include such a disclaimer yourself. That being said, it seems that the back and forth here doesn’t really seem to have any statements of the form “X (and also Y), therefore Z”. So, I guess that makes it hard to analyze formally as an argument, as instead much of the things like that being explicitly said, there are things being mentioned, with a number of things left implicit.
idlewords 6y ago
I gave a whole talk about how this form of mind wank is theological cosplay, but it may not be Dark Enlightenment enough for your tastes. I'm no Scott Alexander. https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm
- idlebirds 6y ago
  Interested in setting up a clubhouse on this topic?
- mistermann 6y ago
  I like this part best: "What I hope I've done today is shown you the dangers of being too smart. Hopefully you'll leave this talk a little dumber than you started it, and be more immune to the seductions of AI that seem to bedevil smarter people."

idlebirds 6y ago

If artificial intelligence is a religion, it is the only religion with a plausible mechanism of action. Building something that is more powerful and intelligent than any human does not look to violate any law of physics; calling such a thing a god does not make it any less possible. We have proof it can exist (as humans are just machines). Both the top AI companies in the world (Deepmind and OpenAI) explicitly are trying to build AGI; The fact that it can be built and people desire to build it makes informed speculation about it useful.

idlewords 6y ago
Every religion has an explanation, completely persuasive to the devout, for why it's the only true one.

nomic 6y ago

You should setup a clubhouse with Gwern and hash it out. Would love a causal debate on this topic.

idlewords 6y ago
I am a being of pure text, and I suspect the same might be true of Gwern.

layoutIfNeeded 6y ago

How many AIs can dance on the head of a pin?

thesteamboat 6y ago

I have seen you and others make this point in the past, it and always seems equivalent to creationists shouting "Scientists who believe in evolution are the REAL dogmatists who don't want to look at the facts! We are really being persecuted by the religion of Science!" Your talk on this subject (helpfully posted below) only furthers this when you present a variety of arguments of wildly varying strengths against AI, in the same vein as "37 challenges to evolution". It makes me feel like you have some valid criticism and a lot of bad faith argumentation. Sadly, I think this characterization is only mostly unfair rather than entirely so. I enjoy your writing and thoughts, and you speak with clarity, but on this subject I think you have constructed a mold of bad-AI-argumentation that you squeeze all AI-argumentation into and in so doing fail to rebut any of it.

idlewords 6y ago
The difference here (that I hardly believe needs explaining) is that the argument for evolution and natural selection is firmly rooted in observed reality. Even if you are a die-hard creationist, you don't dispute the existence of a wealth of evidence ready at hand that can be marshalled in the argument, for or against. We live on a planet teeming with life, and we are all agreed on the problem that needs explaining (we started with hydrogen and got armadillos, what happened?) Compare this to the debates about hyperintelligence and the anticipated behavior of posited hyperintelligent beings. These consist of (1) an argument by extrapolation that machines or other organisms can surpass human intelligence bolted onto (2) endless from-first-principles blathering about how such a posited hyperintelligent entity would behave. It's like looking at the vial of hydrogen in an otherwise empty universe and trying to deduce things about armadillos from it. In fact, it's much worse, since we are trying to infer things about hypothesized beings who are, by definition, beyond the ability of our minds to encompass. This is exactly unlike any scientific discourse, and exactly like deist arguments about the nature of the gods where (1) you first prove a God must exist, from whatever 'unmoved mover' argument you find personally convincing, and then (2) infer a huge mount of information about that God's behavior (omniscient, benevolent, likes justice) through a series of intellectual non sequiturs. The only innovation here is people swearing up and down that we can build the god ourselves. The one thing we know about hyperintelligence, in any form, is that it is fundamentally not something whose behavior and nature we can infer from first principles, for the same reasons your cat will never understand why you didn't get tenure. I see nothing wrong in the intellectual project of trying to frame questions about the nature of intelligence, and how computers can behave in intelligent ways. Nor do I think it's foolish to wonder about what it would take for machine intelligence for arise, or approach even deeper questions, like the physical basis of consciousness. That's not what we see here, though. Rationalists like gwern take the football and run it into the end zone of GAME THEORY, in the end revealing far more about themselves, their anxieties, and their hopes than shedding any light on the world we inhabit together. They are accompanied by a bunch of otherwise smart people who have scared themselves silly by the prospect that we might build these gods by accident, abruptly, and that we are at imminent risk of doing so. And to the extent that large numbers of smart people (including ones with access to great wealth) have bought into what is fundamentally an apocalyptic religious cult, I think we at least need to call it by the right name.

emtel 6y ago

What an odd reply. Gwern doesn't really talk about the attributes of "the Absolute" in this post. He explains why computational complexity arguments against the possibility of smarter-than-human AIs fail. I have read several of your comments about AI, and never once have I felt that you engaged with the substance of the arguments. This seems to be no exception.