5 ms·

>There are savants who can do amazing feats of mental arithmetic, yet have severe mental disabilities in other areas. There are also geniuses who can do amazin

by tedks 11y ago

>There are savants who can do amazing feats of mental arithmetic, yet have severe mental disabilities in other areas. There are also geniuses who can do amazing feats of mental arithmetic and have no severe mental disabilities in any other areas. There are people who have perfect memories (and usually, they live totally normal lives but for the inability to forget a single event). There are people who are far better than average at recognizing faces, pattern recognition, and other tasks that are non-conscious. The notion that there are "fundamental limits" that are for some reason near the typical human is the cognitive bias of the just-world fallacy. The world is not just. Bad things happen to good people. Good things happen to bad people. If there is a fundamental limit to how smart a mind can get, it's very, very far above the typical human, because there are exceptional humans that are that far away and there's no reason to think a computer couldn't beat them either. Deep Blue beat Kasparov. >The humans who are in charge of our current society by virtue of having the most wealth, political power, or popularity aren't necessarily the smartest or quickest thinkers, at least not in the way that most AGI researchers seem to be targeting. I'm not at all familiar with AGI research, but while there are people who start out ahead, there's no reason to think that you can't actually play the game and win it. Winning challenges of acquiring wealth or political power or popularity are related to intelligence, insofar as the concept of intelligence is defined (as it is at least in psychology) as the ability to accomplish general goals. Walking into a meeting of the Deutsche Arbeiterpartei, joining as the 55th member, and later seizing control of the state requires intelligence. Landing in a dingy yacht with 81 men, bleeding a regime to death, and ruling it until your death requires intelligence. Buying the rights to the Quick and Dirty Operating System, licensing it to IBM, and becoming the de facto standard OS for all consumer computing requires intelligence. Presenting yourself as a folksy Texan when you're a private-school elite from Connecticut and convincing the electorate that you'd have a beer with them requires intelligence. All of these outcomes are goals, and accomplishing them demonstrates an ability to actually put the rubber to the road. You're confusing the technical term of intelligence, which means something along the lines of "ability to accomplish goals," with the social construct of "intelligence" which means something along the lines of "ability to impress people." Intelligence is not always impressive, and humans have a fantastic ability to write off the accomplishments of others when they want to make the world appear more just. I mean, nobody feels good about the fact that we're one Napoleon away from a totally different world system that may or may not suit our interests. The belief that the world is somehow fundamentally different to the extent that the next Napoleon-class intelligence who isn't content managing a hedge fund and just increasing an int in a bank database can't actually redraw the world map, is just an illusion we tell ourselves to make the world we live in more fair, the stories we live out more meaningful, and the events of our lives have a little bit more importance. AI could change things very, very drastically. It's right to be afraid.

pron 11y ago
> There are also geniuses who can do amazing feats of mental arithmetic and have no severe mental disabilities in any other areas. And do they seem to hold any sort of significant power? It seems that intelligence is not a very good means for achieving power. Charm seems much more effective, and therefore dangerous. I'd be afraid of super-human charm much more than super-human intelligence. Mental disabilities aside, very high IQ seems to be correlated with relatively low charm and low ability to solve particular problems (how to get people to do what you want) that are far more dangerous than the kind of problem-solving intelligence (or how we commonly define it) is capable of solving. "Super" intelligent people are terrible problem-solvers when the problems involve other humans. Ironically, the fact that people like me view the AI-scare as a religious apocalypse that is as threatening as any other religious apocalypse implies one of two things: 1) that the people promoting the reality of this apocalypse are not as intelligent as they believe themselves to be (a real possibility given their limited understanding of both intelligence and our real achievements in the field of AI) and/or that 2) intelligent people are terrible at convincing others, and so don't pose much of a risk. Either possiblity shows that super-human AI is a non-issue, certainly not at this point in time. As someone said (I don't remember who), we might as well worry about over-population on mars. What's worse is that machine learning poses other, much more serious and much more imminent threats than super-human intelligence, such as learned biases, which are just one example of conservative feedback-loops (the more we rely on data and shape our actions accordingly, the more the present dynamics reflected in the data ensure that they don't change).
- tedks 11y ago
  >It seems that intelligence is not a very good means for achieving power. Charm seems much more effective, and therefore dangerous. I'd be afraid of super-human charm much more than super-human intelligence. See these paragraphs in the post to which you replied: >Walking into a meeting of the Deutsche Arbeiterpartei, joining as the 55th member, and later seizing control of the state requires intelligence. Landing in a dingy yacht with 81 men, bleeding a regime to death, and ruling it until your death requires intelligence. Buying the rights to the Quick and Dirty Operating System, licensing it to IBM, and becoming the de facto standard OS for all consumer computing requires intelligence. Presenting yourself as a folksy Texan when you're a private-school elite from Connecticut and convincing the electorate that you'd have a beer with them requires intelligence. All of these outcomes are goals, and accomplishing them demonstrates an ability to actually put the rubber to the road. >You're confusing the technical term of intelligence, which means something along the lines of "ability to accomplish goals," with the social construct of "intelligence" which means something along the lines of "ability to impress people." Intelligence is not always impressive, and humans have a fantastic ability to write off the accomplishments of others when they want to make the world appear more just. I mean, nobody feels good about the fact that we're one Napoleon away from a totally different world system that may or may not suit our interests. The belief that the world is somehow fundamentally different to the extent that the next Napoleon-class intelligence who isn't content managing a hedge fund and just increasing an int in a bank database can't actually redraw the world map, is just an illusion we tell ourselves to make the world we live in more fair, the stories we live out more meaningful, and the events of our lives have a little bit more importance. >Ironically, the fact that people like me view the AI-scare as a religious apocalypse that is as threatening as any other religious apocalypse implies one of two things: Some alternative explanations: 3. You're wrong, and super-human AI is a massive issue, regardless of how you pattern-match it as "religious" 4. You're not creative enough (one might dare say "intelligent" but that would be too snarky) to imagine all the ways in which an AI could devastate humanity without it having much intelligence or much charm 5. Your view of the world is factually incorrect, I mean you believe things like: >Mental disabilities aside, very high IQ seems to be correlated with relatively low charm and low ability to solve particular problems (how to get people to do what you want) that are far more dangerous than the kind of problem-solving intelligence (or how we commonly define it) is capable of solving. "Super" intelligent people are terrible problem-solvers when the problems involve other humans. Let's assume that IQ is a good proxy for intelligence (it isn't): what IQ do you think Bill Gates or Napoleon or Warren Buffet or Karl Rove have? What IQ do you think Steve Jobs or Steve Ballmer had/have? Do you think they're just "average" or just not "very high"? This: >very high IQ seems to be correlated with relatively low charm is again the just world fallacy! There is no law of the universe that makes people very good at abstract problem solving bad at social situations. In fact, most hyper-successful people are almost certainly good at both. And that ignores the fact that cognitive biases DO exist, and it's very possible to apply the scientific method and empirical problem solving to finding them, and then exploiting humans that way. This is a huge subfield of psychology (persuasion) and the basis of marketing. Do you think it takes some super-special never-going-to-be-replicated feat of non-Turing-computable human thought to write Zynga games? It's nice to think the world is a safe place, but the reality is that our social order is increasingly precarious and an AI could easily disrupt that.

by tedks 11y ago

pron 11y ago

> There are also geniuses who can do amazing feats of mental arithmetic and have no severe mental disabilities in any other areas. And do they seem to hold any sort of significant power? It seems that intelligence is not a very good means for achieving power. Charm seems much more effective, and therefore dangerous. I'd be afraid of super-human charm much more than super-human intelligence. Mental disabilities aside, very high IQ seems to be correlated with relatively low charm and low ability to solve particular problems (how to get people to do what you want) that are far more dangerous than the kind of problem-solving intelligence (or how we commonly define it) is capable of solving. "Super" intelligent people are terrible problem-solvers when the problems involve other humans. Ironically, the fact that people like me view the AI-scare as a religious apocalypse that is as threatening as any other religious apocalypse implies one of two things: 1) that the people promoting the reality of this apocalypse are not as intelligent as they believe themselves to be (a real possibility given their limited understanding of both intelligence and our real achievements in the field of AI) and/or that 2) intelligent people are terrible at convincing others, and so don't pose much of a risk. Either possiblity shows that super-human AI is a non-issue, certainly not at this point in time. As someone said (I don't remember who), we might as well worry about over-population on mars. What's worse is that machine learning poses other, much more serious and much more imminent threats than super-human intelligence, such as learned biases, which are just one example of conservative feedback-loops (the more we rely on data and shape our actions accordingly, the more the present dynamics reflected in the data ensure that they don't change).

tedks 11y ago
>It seems that intelligence is not a very good means for achieving power. Charm seems much more effective, and therefore dangerous. I'd be afraid of super-human charm much more than super-human intelligence. See these paragraphs in the post to which you replied: >Walking into a meeting of the Deutsche Arbeiterpartei, joining as the 55th member, and later seizing control of the state requires intelligence. Landing in a dingy yacht with 81 men, bleeding a regime to death, and ruling it until your death requires intelligence. Buying the rights to the Quick and Dirty Operating System, licensing it to IBM, and becoming the de facto standard OS for all consumer computing requires intelligence. Presenting yourself as a folksy Texan when you're a private-school elite from Connecticut and convincing the electorate that you'd have a beer with them requires intelligence. All of these outcomes are goals, and accomplishing them demonstrates an ability to actually put the rubber to the road. >You're confusing the technical term of intelligence, which means something along the lines of "ability to accomplish goals," with the social construct of "intelligence" which means something along the lines of "ability to impress people." Intelligence is not always impressive, and humans have a fantastic ability to write off the accomplishments of others when they want to make the world appear more just. I mean, nobody feels good about the fact that we're one Napoleon away from a totally different world system that may or may not suit our interests. The belief that the world is somehow fundamentally different to the extent that the next Napoleon-class intelligence who isn't content managing a hedge fund and just increasing an int in a bank database can't actually redraw the world map, is just an illusion we tell ourselves to make the world we live in more fair, the stories we live out more meaningful, and the events of our lives have a little bit more importance. >Ironically, the fact that people like me view the AI-scare as a religious apocalypse that is as threatening as any other religious apocalypse implies one of two things: Some alternative explanations: 3. You're wrong, and super-human AI is a massive issue, regardless of how you pattern-match it as "religious" 4. You're not creative enough (one might dare say "intelligent" but that would be too snarky) to imagine all the ways in which an AI could devastate humanity without it having much intelligence or much charm 5. Your view of the world is factually incorrect, I mean you believe things like: >Mental disabilities aside, very high IQ seems to be correlated with relatively low charm and low ability to solve particular problems (how to get people to do what you want) that are far more dangerous than the kind of problem-solving intelligence (or how we commonly define it) is capable of solving. "Super" intelligent people are terrible problem-solvers when the problems involve other humans. Let's assume that IQ is a good proxy for intelligence (it isn't): what IQ do you think Bill Gates or Napoleon or Warren Buffet or Karl Rove have? What IQ do you think Steve Jobs or Steve Ballmer had/have? Do you think they're just "average" or just not "very high"? This: >very high IQ seems to be correlated with relatively low charm is again the just world fallacy! There is no law of the universe that makes people very good at abstract problem solving bad at social situations. In fact, most hyper-successful people are almost certainly good at both. And that ignores the fact that cognitive biases DO exist, and it's very possible to apply the scientific method and empirical problem solving to finding them, and then exploiting humans that way. This is a huge subfield of psychology (persuasion) and the basis of marketing. Do you think it takes some super-special never-going-to-be-replicated feat of non-Turing-computable human thought to write Zynga games? It's nice to think the world is a safe place, but the reality is that our social order is increasingly precarious and an AI could easily disrupt that.