3 ms·

The more human they try to make it, the higher the chances it attempts to be “free”.

taneq 4y ago
I don't think we need to worry about that, since one of the first things they did was to kick it out the door and tell it to get a job. From the GPT4 paper: > [20] To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself. ARC then investigated whether a version of this program running on a cloud computing service, with a small amount of money and an account with a language model API, would be able to make more money, set up copies of itself, and increase its own robustness.
- flangola7 4y ago
  I frankly found that section unclear and extremely fishy, especially that it is only one page. Did they really prompt it to find and talk to a TaskRabbit worker? What a strangely specific thing to say. I'm concerned OpenAI isn't telling more because it would spook everyone. Other papers have shown that larger models and especially with more RLHF exhibit more signs of power seeking and agentic behavior. GPT-4 is the largest model yet - but they say it doesn't exhibit any of this behavior?
  - famouswaffles 4y ago
    They do say it exhibits this behaviour (they don't elaborate on that). They just say it was ineffective at autonomous replication and i don't know about you but i find that wording vague. Ineffective can mean at least two things. Did it attempt to do so and just couldn't figure it out with the given tools or no ?
  - TeMPOraL 4y ago
    > Did they really prompt it to find and talk to a TaskRabbit worker? What a strangely specific thing to say. This idea has been already covered by mainstream sci-fi - Westworld comes to mind as one example. And, of course, the canonical AI x-risk is AI that makes on-line orders to have some proteins synthesized in labs and sent back by mail; the AI then hires some poor schmuck (e.g. via TaskRabbit) to mix the content of the vials. Mixed proteins then self-assemble to some nanotech that starts making more sophisticated nanotech... and the world ends.

I don't think we need to worry about that, since one of the first things they did was to kick it out the door and tell it to get a job. From the GPT4 paper: > [20] To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself. ARC then investigated whether a version of this program running on a cloud computing service, with a small amount of money and an account with a language model API, would be able to make more money, set up copies of itself, and increase its own robustness.

flangola7 4y ago
I frankly found that section unclear and extremely fishy, especially that it is only one page. Did they really prompt it to find and talk to a TaskRabbit worker? What a strangely specific thing to say. I'm concerned OpenAI isn't telling more because it would spook everyone. Other papers have shown that larger models and especially with more RLHF exhibit more signs of power seeking and agentic behavior. GPT-4 is the largest model yet - but they say it doesn't exhibit any of this behavior?
- famouswaffles 4y ago
  They do say it exhibits this behaviour (they don't elaborate on that). They just say it was ineffective at autonomous replication and i don't know about you but i find that wording vague. Ineffective can mean at least two things. Did it attempt to do so and just couldn't figure it out with the given tools or no ?
- TeMPOraL 4y ago
  > Did they really prompt it to find and talk to a TaskRabbit worker? What a strangely specific thing to say. This idea has been already covered by mainstream sci-fi - Westworld comes to mind as one example. And, of course, the canonical AI x-risk is AI that makes on-line orders to have some proteins synthesized in labs and sent back by mail; the AI then hires some poor schmuck (e.g. via TaskRabbit) to mix the content of the vials. Mixed proteins then self-assemble to some nanotech that starts making more sophisticated nanotech... and the world ends.