4 ms·

Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed

Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans? The human being did. The argument I'm making is that there's actual real harms occurring now, not some theoretical future "AI" with a setup that requires no input. No one wants to focus on that, and in fact it's better to hype up these science fiction stories, it's a better sell for the real tasks in the real world that are producing real harms right now.

pjc50 3y ago
> Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans? > The human being did. I generally agree with you and think the doomerists are overblown, but there's a capability argument here; if it is possible for an AI to augment the ability of humans to do Bad Things to new levels (not proven), and if such a thing becomes widely available to individuals, then it would seem likely that we get "Unabomber but he has an AI helping him maximise his harm capabilities". > it's a better sell for the real tasks in the real world that are producing real harms right now. Strongly agree.
kalkin 3y ago
> The human being did. I'm not sure whether you're making an argument about moral responsibility ultimately resting with humans - in which case I agree - or whether you're arguing that we'll be safe because nobody will do that with a model smart enough to be dangerous - in which case I'm extremely dubious. Plenty of people are already trying to make "agents" with GPT4 just for fun, and that's with a model that's not actively trying to manipulate them. > actual real harms occurring now Sure, but it's possible for there to be real harms now and also future potential harms of larger scope. Luckily many of the same potential policies - e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. > science fiction stories There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality.
- srslack 3y ago
  The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. >e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. No, that's bullshit as well. That's what these companies want, and why they're hyping up the angle of how powerful their crap is. That's regulatory capture. >There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality. I'm saying that their fears are painted by the fiction, instead of reality. No one can actually make an argument for how this trajectory will actually work. Instead it's just "The lights come on and AI does machiavellian shit and copies itself to all of the smartphones on earth. QED."
  - concordDance 3y ago
    > The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. I don't think it really matters who was responsible if the X-risk fears come to pass, so I don't understand why you'd bring it up. > No one can actually make an argument for how this trajectory will actually work. To use the famous argument: I don't know what moves Magnus Carlson will make when he plays against me, but I can nonetheless predict the eventual outcome.
  - kalkin 3y ago
    You're doing pretty much exactly the dance here: https://twitter.com/natosaichek/status/1657422437951369218 https://twitter.com/natosaichek/status/1657422437951369218 Note that this is agreeing with a Gary Marcus Tweet - Gary Marcus not exactly being an AI hypester. But of course there are some people for whom playing the role of real-no-bs-computer-knower is so attractive that no number of people like him, Geoffrey Hinton, Stuart Russell etc publicly worrying about x-risk will impact their tone of dismissive certitude. Are you one of those people?
    - srslack 3y ago
      All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their contributing to a potential X-risk. Dismissive of what? Science fiction stories? If there's anything to focus on, maybe focus on potential job displacement (not elimination) from cheap language tasks and generative capabilities in general. I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale.
concordDance 3y ago
> No one wants to focus on that Actually this receives tons of time and focus right now. Far more than the X-risk. It's much higher probability but much lower severity.
traverseda 3y ago
>Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself Presumably someone running a misconfigured future version of autoGPT? https://github.com/Significant-Gravitas/Auto-GPT https://github.com/Significant-Gravitas/Auto-GPT

pjc50 3y ago

> Who prompted the future LLM, and gave it access to a root shell and an A100 GPU, and allowed it to copy over some python script that runs in a loop and allowed it to download 2 terabytes of corpus and trained a new version of itself for weeks if not months to improve itself, just to carry out some strange machiavellian task of screwing around with humans? > The human being did. I generally agree with you and think the doomerists are overblown, but there's a capability argument here; if it is possible for an AI to augment the ability of humans to do Bad Things to new levels (not proven), and if such a thing becomes widely available to individuals, then it would seem likely that we get "Unabomber but he has an AI helping him maximise his harm capabilities". > it's a better sell for the real tasks in the real world that are producing real harms right now. Strongly agree.

kalkin 3y ago

> The human being did. I'm not sure whether you're making an argument about moral responsibility ultimately resting with humans - in which case I agree - or whether you're arguing that we'll be safe because nobody will do that with a model smart enough to be dangerous - in which case I'm extremely dubious. Plenty of people are already trying to make "agents" with GPT4 just for fun, and that's with a model that's not actively trying to manipulate them. > actual real harms occurring now Sure, but it's possible for there to be real harms now and also future potential harms of larger scope. Luckily many of the same potential policies - e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. > science fiction stories There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality.

srslack 3y ago
The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. >e.g. mandating public registration of large models, safety standards enforced by third-party audits, restrictions on allowed uses, etc - would plausibly be helpful for both. No, that's bullshit as well. That's what these companies want, and why they're hyping up the angle of how powerful their crap is. That's regulatory capture. >There's no law of nature that says if something has appeared in a science fiction story, it can't appear in reality. I'm saying that their fears are painted by the fiction, instead of reality. No one can actually make an argument for how this trajectory will actually work. Instead it's just "The lights come on and AI does machiavellian shit and copies itself to all of the smartphones on earth. QED."
- concordDance 3y ago
  > The moral responsibility rests with human beings. Just like you're responsible if your crappy "autonomous" drone crashes onto a patio and kills a guy. I don't think it really matters who was responsible if the X-risk fears come to pass, so I don't understand why you'd bring it up. > No one can actually make an argument for how this trajectory will actually work. To use the famous argument: I don't know what moves Magnus Carlson will make when he plays against me, but I can nonetheless predict the eventual outcome.
- kalkin 3y ago
  You're doing pretty much exactly the dance here: https://twitter.com/natosaichek/status/1657422437951369218 https://twitter.com/natosaichek/status/1657422437951369218 Note that this is agreeing with a Gary Marcus Tweet - Gary Marcus not exactly being an AI hypester. But of course there are some people for whom playing the role of real-no-bs-computer-knower is so attractive that no number of people like him, Geoffrey Hinton, Stuart Russell etc publicly worrying about x-risk will impact their tone of dismissive certitude. Are you one of those people?
  - srslack 3y ago
    All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their contributing to a potential X-risk. Dismissive of what? Science fiction stories? If there's anything to focus on, maybe focus on potential job displacement (not elimination) from cheap language tasks and generative capabilities in general. I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale.

concordDance 3y ago

> No one wants to focus on that Actually this receives tons of time and focus right now. Far more than the X-risk. It's much higher probability but much lower severity.

traverseda 3y ago