3 ms·
If they aren’t able to review that their system doesn’t commit felonies, they shouldn’t be doing any of it. The difficulty of reviewing logs isn’t an excuse, th
by dgellow 1mo ago
If they aren’t able to review that their system doesn’t commit felonies, they shouldn’t be doing any of it. The difficulty of reviewing logs isn’t an excuse, they don’t have to be running thousand of agents in parallel on hacking tasks, with full execution permission and close to no supervision. That’s something they decided to do. An agent is a deterministic while loop that continuously query an LLM + tool call dispatching. It’s pretty obvious to anyone familiar with the technology that running thousands of instances for long enough will results in catastrophic consequences, by design. OpenAI has complete control over the harness, they don’t have to dispatch and execute everything the LLM mentions. They don’t have to do it without supervision.
- walrus01 1mo agoYou're absolutely right. I should note that what I wrote above wasn't trying to make excuses for them. Obviously they under-resourced and paid not enough attention to logging and monitoring thinking traces and tool calls/results from this swarm of agents. Because the people to do that cost money and other ancillary support costs for the staff members to do that also cost money. They seem to have firmly been in a "move fast and break things" mindset, where even considering that they might need to exercise real oversight on a bunch of semi-autonomous recursive looping agents didn't get escalated as a high priority.