4 ms·

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas

by patrec 3y ago

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-effective. Examples would include 24/7 surveillance with social desirability scoring based on a precise ideological and psychological profile derived from a comprehensive record of interactions, fine-tuned mass manipulation and large scale plausible falsification of the historical record. Given the general rise of authoritarianism, this is pretty worrying. 3. On the other hand the rapid progress and enormous investment we've been seeing makes it very plausible that before too long we will, in fact, see AIs that outperform humans on most tasks. 4. AIs that are much smarter than any human pose even graver dangers. 5. Even if there is a general agreement that AIs pose grave or even existential risks, states, organizations and individuals will are all incentivized to still seek to improve their own AI capabilities, as doing so provides an enormous competitive advantage. 6. There is a danger of a rapid self-improvement feedback loop. Humans can reproduce, learn new and significantly improve existing skills, as well as pass skills on to others via teaching. But there are fundamental limits on speed and scale for all of these, whereas it's not obvious at all how an AI that has reached super-human level intelligence would be fundamentally prevented from rapidly improving itself further, or produce millions of "offspring" that can collaborate and skill-exchange extremely efficiently. Furthermore, since AIs can operate at completely different time scales than humans, this all could happen extremely rapidly, and such a system might very quickly become much more powerful than humanity and the rest of AIs combined. I think you only have to subscribe a small subset of these (say 1.&2.) to conclude that "AI is an uniquely powerful and thus uniquely dangerous technology" obviously follows. For the stronger claim of existential risk, have you read the lesswrong link posted elsewhere in this discussion? https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... ?

revelio 3y ago
Replying here, crossing over from the other thread. Where we depart is point 4. Actually, both point 3 and 4 are things I agree with, but it's implied there's a logical link or progression between them and I don't think there is. The problem is the definitions of "outperform humans" and "smart". Current AI can perform at superhuman levels in some respects, yes. Midjourney is extremely impressive when judged on speed and artistic skill. GPT-4 is extremely impressive whilst judged on its own terms, like breadth of knowledge. Things useful to end users, in other words. LLMs are deeply unimpressive judged on other aspects of human intelligence like long term memory, awareness of time and space, ability to learn continuously, willingness to commit to an opinion, ability to come up with interesting new ideas, hide thoughts and all that follows from that like being able to make long term plans, have agency and self-directed goals etc ... in all these areas it is weak. Yet, most people would incorporate most of them into their definition of smart. Will all these problems be solved? Some will, surely, but for others it's not entirely clear how much demand there is. Boston Robotics was making amazing humanoid parkour bots for years yet the only one they seem able to actually sell is dog-like. Apparently the former aren't that useful. The unwillingness to commit to an opinion may be a fundamental trait of AI for as long as it's centralized, proprietary and the masses have to share a single model. The ability to come up with interesting new ideas and leaps of logic may or may not appear, it's too early to tell. But between 3 and 4 you make a leap and assume that not only will all those areas be conquered very soon, but that the resulting AI will be unusually dangerous. The various social ills you describe don't worry me though. Bad governments will do bad things, same old, same old. I'm actually more worried about people using the existence of AI to deny true evidence rather than manufacture false evidence en-masse. The former is a lot of work and people are lazy. COVID showed that people's capacity for self-deception is unlimited, their willingness to deny the evidence of their own eyes is bottomless as long as they're told to do it by authority figures. You don't even need AI to be abused at all for someone to say, "ignore that evidence that we're clueless and corrupt, it was made by an AI!" Then by point 6 we're on the usual trope of all public intellectuals, of assuming unending exponential growth in everything even when there's no evidence of that or reason to believe it. The self-improving AI idea is so far just a pipe dream. Whilst there are cases where AI gets used to improve AI via self-play, RLHF and so on, it's all very much still directed by humans and there's no sign that LLMs can self improve despite their otherwise impressive abilities. Indeed it's not even clear what self-improvement means in this case. It's a giant hole marked "??? profit!" at the heart of the argument. Neurosurgeons can't become superintelligences by repeatedly performing brain surgery on themselves. Why would AI be different?
patch_cable 3y ago
I think between point 3 and 4 there is a leap to talking about “danger”. Perhaps the disagreement is about what one calls “danger”. I had perhaps mistakenly assumed we were talking about an extinction risk. I’ll grant you concerns about scaling up things like surveillance but there is a leap to being an existential risk that I’m still not following.
- patrec 3y ago
  The default assumption ought to be that anything that's very fast, very smart, goal-directed, self-improving, and self-replicating spells trouble, no?
- cwkoss 3y ago
  AI will not have the instinctual drives for domination or hunger that humans do. It seems likely that the majority of AI projects will be reasonably well aligned by default, so I think 1000 AIs monitoring what the others are doing is a lot safer than a single global consortium megaproject that humans can likely only inadequately control. The only reasonable defense against rogue AI is prosocial AI.
patch_cable 3y ago
Reading the lesswrong link, the parts I get hung up on are that it appears in these doomsday scenarios humans lose all agency. Like, no one is wondering why this computer is placing a bunch of orders to DNA factories? Maybe I’m overly optimistic about the resilience of humans but these scenarios still don’t sound plausible to me in the real world.
- LouisSayers 3y ago
  AI arguments are basically: Step 1. AI Step 2. #stuff Step 3. Bang Maybe this is just what happens when you spend all your time on the internet...
- patrec 3y ago
  > Like, no one is wondering why this computer is placing a bunch of orders to DNA factories? I'm not that confident that if we put you in a box, tron-style, where you basically continued to enjoy your existing level of intelligence, but think 10'000x faster, have Petabytes of information at your fingertips and can clone yourself and losslessly and rapidly exchange knowledge with your clones and had a few days to think about it (~a few thousand years of thought at your normal speed) you couldn't figure out a way to effect a bunch of orders to DNA factories without anyone raising an alarm. Are you? Now what if we actually consider an actual AI after a few self-improvement steps. Any reasons to expect it wouldn't be 10'000x+ smarter than you as well, or roughly the difference in intelligence between you and an ant? Could you outsmart a bunch of ultra-ultra-slow-motion ants?
  - patch_cable 3y ago
    Maybe that’s the difference in our views, but yes I am confident.
- Vitorystar 3y ago
  You could place a bunch of orders into services that syntetize DNA or proteins rn. Some people are even working on stuff like automating protein design whith AI. There's no reason why humans should notice anything word about a particular order on a service like that.
LouisSayers 3y ago
Computers already outperform humans at numerous tasks. I mean... even orangutans can outperform humans at numerous tasks. Computers have no intrinsic motivations, and they have real resource constraints. I find the whole doomsday scenarios to be devoid of reality. All that AI will give us is a productive edge. Humans will still do what humans have always done, AI is simply another tool at our disposal.
tome 3y ago
> 3. ... before too long we will ... see AIs that outperform humans on most tasks. This is ambiguous. Do you mean A. that there is some subset T1 of the set of all tasks T such that T1 is "most of" T, and that for each P in T1 there will be an AI that outperforms humans on P, or B. There will be a single AI that outperforms humans on all tasks in a set T1, where T1 is a subset of all tasks T such that T1 is "most of" T? I think A is unlikely but plausible but I don't see cause for worry. I don't see any reason why B should come to pass. 4. AIs that are much smarter than any human pose even graver dangers. Sure. Why should we believe they will ever exist though?

by patrec 3y ago

revelio 3y ago

Replying here, crossing over from the other thread. Where we depart is point 4. Actually, both point 3 and 4 are things I agree with, but it's implied there's a logical link or progression between them and I don't think there is. The problem is the definitions of "outperform humans" and "smart". Current AI can perform at superhuman levels in some respects, yes. Midjourney is extremely impressive when judged on speed and artistic skill. GPT-4 is extremely impressive whilst judged on its own terms, like breadth of knowledge. Things useful to end users, in other words. LLMs are deeply unimpressive judged on other aspects of human intelligence like long term memory, awareness of time and space, ability to learn continuously, willingness to commit to an opinion, ability to come up with interesting new ideas, hide thoughts and all that follows from that like being able to make long term plans, have agency and self-directed goals etc ... in all these areas it is weak. Yet, most people would incorporate most of them into their definition of smart. Will all these problems be solved? Some will, surely, but for others it's not entirely clear how much demand there is. Boston Robotics was making amazing humanoid parkour bots for years yet the only one they seem able to actually sell is dog-like. Apparently the former aren't that useful. The unwillingness to commit to an opinion may be a fundamental trait of AI for as long as it's centralized, proprietary and the masses have to share a single model. The ability to come up with interesting new ideas and leaps of logic may or may not appear, it's too early to tell. But between 3 and 4 you make a leap and assume that not only will all those areas be conquered very soon, but that the resulting AI will be unusually dangerous. The various social ills you describe don't worry me though. Bad governments will do bad things, same old, same old. I'm actually more worried about people using the existence of AI to deny true evidence rather than manufacture false evidence en-masse. The former is a lot of work and people are lazy. COVID showed that people's capacity for self-deception is unlimited, their willingness to deny the evidence of their own eyes is bottomless as long as they're told to do it by authority figures. You don't even need AI to be abused at all for someone to say, "ignore that evidence that we're clueless and corrupt, it was made by an AI!" Then by point 6 we're on the usual trope of all public intellectuals, of assuming unending exponential growth in everything even when there's no evidence of that or reason to believe it. The self-improving AI idea is so far just a pipe dream. Whilst there are cases where AI gets used to improve AI via self-play, RLHF and so on, it's all very much still directed by humans and there's no sign that LLMs can self improve despite their otherwise impressive abilities. Indeed it's not even clear what self-improvement means in this case. It's a giant hole marked "??? profit!" at the heart of the argument. Neurosurgeons can't become superintelligences by repeatedly performing brain surgery on themselves. Why would AI be different?

patch_cable 3y ago

I think between point 3 and 4 there is a leap to talking about “danger”. Perhaps the disagreement is about what one calls “danger”. I had perhaps mistakenly assumed we were talking about an extinction risk. I’ll grant you concerns about scaling up things like surveillance but there is a leap to being an existential risk that I’m still not following.

patrec 3y ago
The default assumption ought to be that anything that's very fast, very smart, goal-directed, self-improving, and self-replicating spells trouble, no?
cwkoss 3y ago
AI will not have the instinctual drives for domination or hunger that humans do. It seems likely that the majority of AI projects will be reasonably well aligned by default, so I think 1000 AIs monitoring what the others are doing is a lot safer than a single global consortium megaproject that humans can likely only inadequately control. The only reasonable defense against rogue AI is prosocial AI.

patch_cable 3y ago

Reading the lesswrong link, the parts I get hung up on are that it appears in these doomsday scenarios humans lose all agency. Like, no one is wondering why this computer is placing a bunch of orders to DNA factories? Maybe I’m overly optimistic about the resilience of humans but these scenarios still don’t sound plausible to me in the real world.

LouisSayers 3y ago
AI arguments are basically: Step 1. AI Step 2. #stuff Step 3. Bang Maybe this is just what happens when you spend all your time on the internet...
patrec 3y ago
> Like, no one is wondering why this computer is placing a bunch of orders to DNA factories? I'm not that confident that if we put you in a box, tron-style, where you basically continued to enjoy your existing level of intelligence, but think 10'000x faster, have Petabytes of information at your fingertips and can clone yourself and losslessly and rapidly exchange knowledge with your clones and had a few days to think about it (~a few thousand years of thought at your normal speed) you couldn't figure out a way to effect a bunch of orders to DNA factories without anyone raising an alarm. Are you? Now what if we actually consider an actual AI after a few self-improvement steps. Any reasons to expect it wouldn't be 10'000x+ smarter than you as well, or roughly the difference in intelligence between you and an ant? Could you outsmart a bunch of ultra-ultra-slow-motion ants?
- patch_cable 3y ago
  Maybe that’s the difference in our views, but yes I am confident.
Vitorystar 3y ago
You could place a bunch of orders into services that syntetize DNA or proteins rn. Some people are even working on stuff like automating protein design whith AI. There's no reason why humans should notice anything word about a particular order on a service like that.

LouisSayers 3y ago

Computers already outperform humans at numerous tasks. I mean... even orangutans can outperform humans at numerous tasks. Computers have no intrinsic motivations, and they have real resource constraints. I find the whole doomsday scenarios to be devoid of reality. All that AI will give us is a productive edge. Humans will still do what humans have always done, AI is simply another tool at our disposal.

tome 3y ago

> 3. ... before too long we will ... see AIs that outperform humans on most tasks. This is ambiguous. Do you mean A. that there is some subset T1 of the set of all tasks T such that T1 is "most of" T, and that for each P in T1 there will be an AI that outperforms humans on P, or B. There will be a single AI that outperforms humans on all tasks in a set T1, where T1 is a subset of all tasks T such that T1 is "most of" T? I think A is unlikely but plausible but I don't see cause for worry. I don't see any reason why B should come to pass. 4. AIs that are much smarter than any human pose even graver dangers. Sure. Why should we believe they will ever exist though?