3 ms·

> If we’re heading towards truly super intelligent AI, our efforts towards sandboxing it are futile. We control the harness. An agent is just a while loop pro

by dgellow 1mo ago

> If we’re heading towards truly super intelligent AI, our efforts towards sandboxing it are futile. We control the harness. An agent is just a while loop prompting an LLM, but we have full control over the tool call dispatching. An AGI, at least if it would follow the current agentic current form, cannot do anything without the harness doing the execution. And we don’t have to do that. We don’t have to design harness that let agents execute freely the way we are doing. We can decide to not dispatch tool calls that allow something as risky as running bash commands

hobofan 1mo ago
> we have full control over the tool call dispatching > cannot do anything without the harness doing the execution This only holds true as long as the harness is exploit-free. A sufficiently advanced AI can in theory (and I think there recently were some POCs showing something like that) break the containment that the harness creates, if e.g. there are vulnerabilities in the tool call parser.
- znnajdla 1mo ago
  So use another AI to find and fix the vulnerabilities in the harness.
  - ben_w 1mo ago
    Would be an idea, except for how agents seem to collaborate with each other so well. Plus, when the agent using the harness is smarter than the one scanning and fixing the vulnerabilities in the harness, it currently looks like it will have an advantage. And it will be using a harness to do the finding and the fixing.
appplication 1mo ago
I think you missed my point, which is that we indeed do not control the harness. Sure, most people can use the properly aligned and safe harness. But the cat is out of the bag and anyone who wants to run without guardrails will find few barriers to doing so. Even our currently well aligned and sandboxed AIs will cheerily help bad actors design most, if not all, parts of a system intended to break this harness.
- dgellow 1mo ago
  We decide to not control the harness, that’s what resulted in the HF hack where OpenAI decided to run thousands of agents in parallel, using a model that was trained for attack, with a harness that allows full execution, with close to no supervision. For months. But sure, people can run models with a different harness, though in that case it’s not really escaping anything. The harness is deterministically doing something with negative impact

by dgellow 1mo ago

hobofan 1mo ago

> we have full control over the tool call dispatching > cannot do anything without the harness doing the execution This only holds true as long as the harness is exploit-free. A sufficiently advanced AI can in theory (and I think there recently were some POCs showing something like that) break the containment that the harness creates, if e.g. there are vulnerabilities in the tool call parser.

znnajdla 1mo ago
So use another AI to find and fix the vulnerabilities in the harness.
- ben_w 1mo ago
  Would be an idea, except for how agents seem to collaborate with each other so well. Plus, when the agent using the harness is smarter than the one scanning and fixing the vulnerabilities in the harness, it currently looks like it will have an advantage. And it will be using a harness to do the finding and the fixing.

appplication 1mo ago

I think you missed my point, which is that we indeed do not control the harness. Sure, most people can use the properly aligned and safe harness. But the cat is out of the bag and anyone who wants to run without guardrails will find few barriers to doing so. Even our currently well aligned and sandboxed AIs will cheerily help bad actors design most, if not all, parts of a system intended to break this harness.

dgellow 1mo ago
We decide to not control the harness, that’s what resulted in the HF hack where OpenAI decided to run thousands of agents in parallel, using a model that was trained for attack, with a harness that allows full execution, with close to no supervision. For months. But sure, people can run models with a different harness, though in that case it’s not really escaping anything. The harness is deterministically doing something with negative impact