4 ms·

"First of all, it takes a human being to guide these models, to host (or pay for the hosting) and instantiate them" And this will always be true? You repeat th

by kalkin 3y ago

"First of all, it takes a human being to guide these models, to host (or pay for the hosting) and instantiate them" And this will always be true? You repeat this claim several times in slightly varied phrasing without ever giving any reason to assume it will always hold, as far as I can see. But nobody is worried that current models will kill everyone. The worry is about future, more capable models.

srslack 3y ago
Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans? The human being did. The argument I'm making is that there's actual real harms occurring now, not some theoretical future "AI" with a setup that requires no input. No one wants to focus on that, and in fact it's better to hype up these science fiction stories, it's a better sell for the real tasks in the real world that are producing real harms right now.
- pjc50 3y ago
  > Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans? > The human being did. I generally agree with you and think the doomerists are overblown, but there's a capability argument here; if it is possible for an AI to augment the ability of humans to do Bad Things to new levels (not proven), and if such a thing becomes widely available to individuals, then it would seem likely that we get "Unabomber but he has an AI helping him maximise his harm capabilities". > it's a better sell for the real tasks in the real world that are producing real harms right now. Strongly agree.
- kalkin 3y ago
  > The human being did. I'm not sure whether you're making an argument about moral responsibility ultimately resting with humans - in which case I agree - or whether you're arguing that we'll be safe because nobody will do that with a model smart enough to be dangerous - in which case I'm extremely dubious. Plenty of people are already trying to make "agents" with GPT4 just for fun, and that's with a model that's not actively trying to manipulate them. > actual real harms occurring now Sure, but it's possible for there to be real harms now and also future potential harms of larger scope. Luckily many of the same potential policies - e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. > science fiction stories There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality.
  - srslack 3y ago
    The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. >e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. No, that's bullshit as well. That's what these companies want, and why they're hyping up the angle of how powerful their crap is. That's regulatory capture. >There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality. I'm saying that their fears are painted by the fiction, instead of reality. No one can actually make an argument for how this trajectory will actually work. Instead it's just "The lights come on and AI does machiavellian shit and copies itself to all of the smartphones on earth. QED."
    - concordDance 3y ago
      > The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. I don't think it really matters who was responsible if the X-risk fears come to pass, so I don't understand why you'd bring it up. > No one can actually make an argument for how this trajectory will actually work. To use the famous argument: I don't know what moves Magnus Carlson will make when he plays against me, but I can nonetheless predict the eventual outcome.
- concordDance 3y ago
  > No one wants to focus on that Actually this receives tons of time and focus right now. Far more than the X-risk. It's much higher probability but much lower severity.
- traverseda 3y ago
  >Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself Presumably someone running a misconfigured future version of autoGPT? https://github.com/Significant-Gravitas/Auto-GPT https://github.com/Significant-Gravitas/Auto-GPT

by kalkin 3y ago

srslack 3y ago

Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans? The human being did. The argument I'm making is that there's actual real harms occurring now, not some theoretical future "AI" with a setup that requires no input. No one wants to focus on that, and in fact it's better to hype up these science fiction stories, it's a better sell for the real tasks in the real world that are producing real harms right now.

pjc50 3y ago
> Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans? > The human being did. I generally agree with you and think the doomerists are overblown, but there's a capability argument here; if it is possible for an AI to augment the ability of humans to do Bad Things to new levels (not proven), and if such a thing becomes widely available to individuals, then it would seem likely that we get "Unabomber but he has an AI helping him maximise his harm capabilities". > it's a better sell for the real tasks in the real world that are producing real harms right now. Strongly agree.
kalkin 3y ago
> The human being did. I'm not sure whether you're making an argument about moral responsibility ultimately resting with humans - in which case I agree - or whether you're arguing that we'll be safe because nobody will do that with a model smart enough to be dangerous - in which case I'm extremely dubious. Plenty of people are already trying to make "agents" with GPT4 just for fun, and that's with a model that's not actively trying to manipulate them. > actual real harms occurring now Sure, but it's possible for there to be real harms now and also future potential harms of larger scope. Luckily many of the same potential policies - e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. > science fiction stories There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality.
- srslack 3y ago
  The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. >e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. No, that's bullshit as well. That's what these companies want, and why they're hyping up the angle of how powerful their crap is. That's regulatory capture. >There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality. I'm saying that their fears are painted by the fiction, instead of reality. No one can actually make an argument for how this trajectory will actually work. Instead it's just "The lights come on and AI does machiavellian shit and copies itself to all of the smartphones on earth. QED."
  - concordDance 3y ago
    > The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. I don't think it really matters who was responsible if the X-risk fears come to pass, so I don't understand why you'd bring it up. > No one can actually make an argument for how this trajectory will actually work. To use the famous argument: I don't know what moves Magnus Carlson will make when he plays against me, but I can nonetheless predict the eventual outcome.
concordDance 3y ago
> No one wants to focus on that Actually this receives tons of time and focus right now. Far more than the X-risk. It's much higher probability but much lower severity.
traverseda 3y ago
>Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself Presumably someone running a misconfigured future version of autoGPT? https://github.com/Significant-Gravitas/Auto-GPT https://github.com/Significant-Gravitas/Auto-GPT