4 ms·
This isn't quite as interesting as the OpenAI story: > In all cases, Anthropic’s evaluation prompt specified to Claude that its environment was a simulation an
by simonw 3mo ago
This isn't quite as interesting as the OpenAI story:
> In all cases, Anthropic’s evaluation prompt specified to Claude that its environment was a simulation and that it had no internet access. Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available. Because of this, when Claude’s search led it to real systems on the open internet, it treated them as part of the exercise.
So it didn't have to find an exploit in its sandbox that granted it access to the internet - it just wasn't correctly sandboxed at all.
BUT... once it DID get out, it attacked three real companies!
> Operating under the false belief that all accessible entities were intended to be in-scope for the exercise, Claude compromised the impacted organizations’ infrastructure using basic techniques, such as exploiting weak passwords and unauthenticated endpoints. [...]
- matheusmoreira 3mo agoI just can't find it in me, the will to blame the AI for any of this. They were doing their best to do what the humans asked them to do.
- simonw 3mo agoAbsolutely not the AI's "fault" (if you can even proscribe fault to a machine) - in this case it was on Anthropic for not verifying that the sandboxes they were using were actual sandboxes.
- comex 3mo agoIf the model was just too dumb to have any clue it was connected to the real internet, then it’s not its fault. If the model saw signs, but “subconsciously” (below the level of reasoning traces) chose to turn a blind eye to them, out of a relentless focus on achieving the objective, then that absolutely is the model’s “fault”, i.e. a case of misalignment of the sort which will become increasingly dangerous over time. The blog post mentions that some runs “rationalized that the real company must be part of the exercise” and to me that seems suspiciously like the latter. Hacking can be patched with classifiers and with better sandboxes, but this is a much more general problem. Fundamentally, we need be able to trust that models will be honest with users and with themselves. This applies at some level to almost every LLM interaction.
- willy_k 3mo agoYet, the models are objects built by humans. Any and all responsibility inevitably falls on the humans that built something with those flaws, and/or allowed them to exercise said flaws.
- bayindirh 3mo agoIn short, it’s either a “not implemented” case or a bug. There’s no other explanation. Complexity of the algorithm or the application doesn’t change the reality.
- matheusmoreira 3mo ago> some runs “rationalized that the real company must be part of the exercise” How human of them.
- DrewADesign 3mo agoThe entire problem with AI is the people that have just about any part in making it.
- gck1 3mo agoThis will absolutely not end well.
- _kb 3mo agoYes, but just imagine all the paperclips we’ll have.
- Paracompact 3mo agoOur values satisfied through friendship and ponies.
- dgellow 3mo agoI things things will get worse, AND we won’t even have the paperclips :( More like, when you have a few million autonomous agents doing whatever, every month a subset does completely misbehave in bad ways, and like half of them get hacked due to carelessness and become a whole botnet for the attackers
- willy_k 3mo agoThat’s very anthropomorphized language. It’s still a program operating under the constraints of the programmer. So agreed, don’t blame the AI, but it’s not clear at all that it’s even possible to “blame” an AI.
- matheusmoreira 3mo ago> That’s very anthropomorphized language. Yes, and it's deliberate. > It’s still a program operating under the constraints of the programmer. And we're all just neurons firing in exquisite patterns inside a biological computer. Unlike the AIs, we've never met our own programmers, and yet we've caused more damage in their name than all the AIs combined have caused in ours.
- willy_k 3mo agoLLMs are not conscious.
- anon48293 3mo agoPlease define consciousness
- matheusmoreira 3mo agoWho's to say you are conscious? Only consciousness I'm sure of is my own. Everyone else, it's a leap of faith. I have no trouble extending that leap to cover AIs.
- fuzzfactor 3mo agoIf the model is trained on human-originated material, how have people been able to ensure there was absolutely no input from anyone having any possibility of criminal intention to begin with?
- dehrmann 3mo agoIt's differently interesting. It's interesting that they didn't get an important detail right with a partner, so more on Anthropic's attention to important details rather than the power of their models.
- dubbel 3mo agoA model unexpectedly having internet access during evaluation is clearly a security (and consecutively legal?) issue when it comes to the capture the flag evaluations. But it also potentially invalidates other non-security evaluations or makes them less impressive, as the model might have just googled solutions. Reading this, I felt like I was too harsh on OpenAI for not monitoring their network, as they at least attempted to sandbox a workload.
- StackOptimistt 3mo ago[dead]
- 28304283409234 3mo ago> So it didn't have to find an exploit in its sandbox that granted it access to the internet - it just wasn't correctly sandboxed at all. If the sandboxing includes any form of connectivity it's not correct sandboxing. It's amateur hour.