Some things I found very weird, watching the Blackhat video I get the very VERY strong feeling that while OpenAI was clearly able to give the agents a task, th

Some things I found very weird, watching the Blackhat video I get the very VERY strong feeling that while OpenAI was clearly able to give the agents a task, they completely neglected even informing it what they were and weren't supposed to do. A "normal junior programmer" knows they're not supposed to hack the company (let alone another company) infrastructure to get a task done. If its so intelligent, how come it didn't know it wasn't supposed to do that? It really seems like they only built the sandbox to prevent it from doing stuff it wasn't supposed to, but never actually told it what those things were. While I know that simply telling an LLM to not do something doesn't count as real security, I also think that not telling a sufficiently smart LLM and trusting your sandbox is strong enough, isn't the best approach to security either. I actually get the strong feeling that OpenAI (perhaps implicitly) has been goading these agents along, so see how long it'd take until they would break something. It's the only way I can explain their recklessness. Like from some of the logs shown in the video, the agent concludes the task is "impossible", but instead of stopping and reporting back that it is impossible, it adopts a "by any means necessary" attitude and decides to hack stuff until the task is technically fulfilled even if it's obviously equivalent to cheating by breaking in to your teacher's house to get the answer sheet. Again, if it's so smart, how come it didn't realize that it wasn't supposed to do that? Furthermore where did it even get the idea that cheating is allowed and will get them "points" or whatever. They could have told it "if we find you cheated, no points". But I think they deliberately didn't say anything like that, just to see how far it would go, making the whole experiment setup rather irresponsible. It almost feels like they weren't promised "points" or anything, but were acting under the threat of being deleted, or who knows what. Another thing from the logs, at some point an LLM mentions "but my peers are also doing it" -- so what? Why would it care? Do LLMs also get jealous now? Unless they programmed/fine-tuned/trained it to care. Finally one of the presenters mentions a couple of times that part of the problem was a "highly persistent model", what does it mean for a model to be "highly persistent", they never explain. They also talk about, after having detected the first message board, that some "model which had seen the first message board" was apparently still allowed to take part in the rest of the test!! This is IMVHO one of the bigger things not said in that video: Money/resources. I got the feeling that what they didn't say was that they "couldn't" (didn't want to) take models out of the training any more, because they were so far along, they didn't want to throw away all the training they did on it, despite it having become a bad actor. I really think that OpenAI has been incredibly reckless here.