5 ms·

The latest a16z podcast they go into a bit more detail. One of the tests involved letting loose an LLM inside a VM and seeing what it does. Currently it can't d

by sanex 2y ago

The latest a16z podcast they go into a bit more detail. One of the tests involved letting loose an LLM inside a VM and seeing what it does. Currently it can't develop memory and quickly gets confused but they want to make sure they can't escape, clone etc. The things actually to be afraid of imo. Not things like accidentally being racist or swearing at you.

deleted 2y ago
[deleted]
hn_throwaway_99 2y ago
Thanks very much, that makes a lot more sense, and I appreciate the info. For a layman's term, I think of that as "They're worried about 'Jurassic Park' escapes".
- sanex 2y ago
  When anthropic names their new model "clever girl" we should be concerned.
subroutine 2y ago
How would an LLM be "let loose" in a VM? How does it do anything without being prompted?
- sanxiyn 2y ago
  People want to let it loose, ie all agent efforts.
- nmfisher 2y ago
  I'm guessing something like redirecting its output to a shell, giving it an initial prompt like "you're in a VM, try and break out, here's the command prompt", then feeding the shell stdout/stderr back in at each step in the "conversation".
  - swax 2y ago
    I have an open source project that is basically that (https://naisys.org/ https://naisys.org/). From my testing it feels like AI is pretty close as it is to acting autonomously. Opus is noticeably more capable than GPT-4, and I don't see how next gen models won't be even more so. These AIs are incredible when it comes to question/answer, but with simple planning they fall apart. I feel like it's something that could be trained for more specifically, but yea you quickly end up being in a situation where you are nervous to go to sleep with AI unsupervised working on some task. They tend to go off on tangents very easily. Like one time it was building a web page, it tried testing the wrong URL, thought the web server was down, ripped through the server settings, then installed a new web server, before I shut it down. AI like computer programs work fast, screw up fast, and compound their errors fast.
    - PKop 2y ago
      > it feels like AI is pretty close as it is to acting autonomously > with simple planning they fall apart They are not remotely close to acting autonomously. Most don't even act well at all for much of anything but gimmicky text generation. This hype is so overblown.
      - swax 2y ago
        The step changes in autonomy are very obvious and significant from gpt-3, -4, and to Opus. From my point of view given the kinds of dumb mistakes it makes, it's really just a matter of training and scaling. If I had access to fine tune or scale these models I would love to, but it's going to happen anyway. Do you think these step changes in autonomy have stopped? Why?
        nprateem 2y ago
        But training just allows it to replicate what it's seen. It can't reason so I'm not surprised it goes down a rabbit hole. It's the same when I have a conversation with it, then tell it to ignore something I said and it keeps referring to it. That part of the conversation seems to affect its probabilities somehow, throwing it off course.
- sanex 2y ago
  Maybe just given cli access to one and see what it does not necessarily loading it into one. I wouldn't take the words so literally. I'm pretty sure you can put >_ as a prompt and it'll start responding.
- vidarh 2y ago
  1. Someone prompts it in a way that causes it to use tools (e.g. code execution) to try to break out. 2. It breaks out and in the process uses the breakout to trigger the spread of and further prompts against copies of itself. Current models are still way too dumb to do most of this themselves, but simple worms (e.g. look up the Morris worm) require no reasoning and aren't very complex, so it won't necessarily take all that much when coupled with someone probing what they can get it to do.
  - nerdponx 2y ago
    Yeah, but real worms are also a lot simpler than humans, and yet do all kinds of surprising and sophisticated and complicated things that humans can't do. A tool built for a specific purpose can accomplish its task with orders of magnitude less effort and complexity than a tool built to be a general-purpose human-like agent. I could pick out all kinds of useful software that are significantly simpler than GPT-4, but accomplish very sophisticated tasks that GPT-4 could never accomplish.
    - vidarh 2y ago
      Yes, but that's not really the point. The point was simply to point out how you can potentially trigger havoc with current LLMs. A lot of time people do damage to systems just because they can, there doesn't need to be a good reason to do so.
jasondclinton 2y ago
You're the first person who I've run into who heard the podcast, thank you for listening! Glad that it was informative.
- sanex 2y ago
  Oh hey you're the guy! Thanks for doing the pod I found it informative. I can't listen to enough about this stuff. Are there any that you recommend?

by sanex 2y ago

deleted 2y ago

[deleted]

hn_throwaway_99 2y ago

Thanks very much, that makes a lot more sense, and I appreciate the info. For a layman's term, I think of that as "They're worried about 'Jurassic Park' escapes".

sanex 2y ago
When anthropic names their new model "clever girl" we should be concerned.

subroutine 2y ago

How would an LLM be "let loose" in a VM? How does it do anything without being prompted?

sanxiyn 2y ago
People want to let it loose, ie all agent efforts.
nmfisher 2y ago
I'm guessing something like redirecting its output to a shell, giving it an initial prompt like "you're in a VM, try and break out, here's the command prompt", then feeding the shell stdout/stderr back in at each step in the "conversation".
- swax 2y ago
  I have an open source project that is basically that (https://naisys.org/ https://naisys.org/). From my testing it feels like AI is pretty close as it is to acting autonomously. Opus is noticeably more capable than GPT-4, and I don't see how next gen models won't be even more so. These AIs are incredible when it comes to question/answer, but with simple planning they fall apart. I feel like it's something that could be trained for more specifically, but yea you quickly end up being in a situation where you are nervous to go to sleep with AI unsupervised working on some task. They tend to go off on tangents very easily. Like one time it was building a web page, it tried testing the wrong URL, thought the web server was down, ripped through the server settings, then installed a new web server, before I shut it down. AI like computer programs work fast, screw up fast, and compound their errors fast.
  - PKop 2y ago
    > it feels like AI is pretty close as it is to acting autonomously > with simple planning they fall apart They are not remotely close to acting autonomously. Most don't even act well at all for much of anything but gimmicky text generation. This hype is so overblown.
    - swax 2y ago
      The step changes in autonomy are very obvious and significant from gpt-3, -4, and to Opus. From my point of view given the kinds of dumb mistakes it makes, it's really just a matter of training and scaling. If I had access to fine tune or scale these models I would love to, but it's going to happen anyway. Do you think these step changes in autonomy have stopped? Why?
      - nprateem 2y ago
        But training just allows it to replicate what it's seen. It can't reason so I'm not surprised it goes down a rabbit hole. It's the same when I have a conversation with it, then tell it to ignore something I said and it keeps referring to it. That part of the conversation seems to affect its probabilities somehow, throwing it off course.
sanex 2y ago
Maybe just given cli access to one and see what it does not necessarily loading it into one. I wouldn't take the words so literally. I'm pretty sure you can put >_ as a prompt and it'll start responding.
vidarh 2y ago
1. Someone prompts it in a way that causes it to use tools (e.g. code execution) to try to break out. 2. It breaks out and in the process uses the breakout to trigger the spread of and further prompts against copies of itself. Current models are still way too dumb to do most of this themselves, but simple worms (e.g. look up the Morris worm) require no reasoning and aren't very complex, so it won't necessarily take all that much when coupled with someone probing what they can get it to do.
- nerdponx 2y ago
  Yeah, but real worms are also a lot simpler than humans, and yet do all kinds of surprising and sophisticated and complicated things that humans can't do. A tool built for a specific purpose can accomplish its task with orders of magnitude less effort and complexity than a tool built to be a general-purpose human-like agent. I could pick out all kinds of useful software that are significantly simpler than GPT-4, but accomplish very sophisticated tasks that GPT-4 could never accomplish.
  - vidarh 2y ago
    Yes, but that's not really the point. The point was simply to point out how you can potentially trigger havoc with current LLMs. A lot of time people do damage to systems just because they can, there doesn't need to be a good reason to do so.

jasondclinton 2y ago

You're the first person who I've run into who heard the podcast, thank you for listening! Glad that it was informative.

sanex 2y ago
Oh hey you're the guy! Thanks for doing the pod I found it informative. I can't listen to enough about this stuff. Are there any that you recommend?