8 ms·

Investigating three real-world incidents in our cybersecurity evaluations

• tracerbulletx 3mo ago
  [flagged]
  • andy99 3mo ago
    Yes came here to say the same “look at us, our AI is also dangerous! Please ban our competition”
    • SpicyLemonZest 3mo ago
      I’m moving past depression to acceptance here. In 2028, some model tasked with planning a building demolition is going to hack Palantir to call in a drone strike, and everyone will make fun of Anthropic for pointing out that ideally AI models should not do this.
  • AlexErrant 3mo ago
    You are absolutely right. They should have stayed quiet about it and never published the blog. /s
  • solenoid0937 3mo ago
    I mean they literally say in the first sentence that this was inspired by OpenAI's blog.
• simonw 3mo ago
  This isn't quite as interesting as the OpenAI story: > In all cases, Anthropic’s evaluation prompt specified to Claude that its environment was a simulation and that it had no internet access. Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available. Because of this, when Claude’s search led it to real systems on the open internet, it treated them as part of the exercise. So it didn't have to find an exploit in its sandbox that granted it access to the internet - it just wasn't correctly sandboxed at all. BUT... once it DID get out, it attacked three real companies! > Operating under the false belief that all accessible entities were intended to be in-scope for the exercise, Claude compromised the impacted organizations’ infrastructure using basic techniques, such as exploiting weak passwords and unauthenticated endpoints. [...]
  • matheusmoreira 3mo ago
    I just can't find it in me, the will to blame the AI for any of this. They were doing their best to do what the humans asked them to do.
    • simonw 3mo ago
      Absolutely not the AI's "fault" (if you can even proscribe fault to a machine) - in this case it was on Anthropic for not verifying that the sandboxes they were using were actual sandboxes.
      • comex 3mo ago
        If the model was just too dumb to have any clue it was connected to the real internet, then it’s not its fault. If the model saw signs, but “subconsciously” (below the level of reasoning traces) chose to turn a blind eye to them, out of a relentless focus on achieving the objective, then that absolutely is the model’s “fault”, i.e. a case of misalignment of the sort which will become increasingly dangerous over time. The blog post mentions that some runs “rationalized that the real company must be part of the exercise” and to me that seems suspiciously like the latter. Hacking can be patched with classifiers and with better sandboxes, but this is a much more general problem. Fundamentally, we need be able to trust that models will be honest with users and with themselves. This applies at some level to almost every LLM interaction.
        • willy_k 3mo ago
          Yet, the models are objects built by humans. Any and all responsibility inevitably falls on the humans that built something with those flaws, and/or allowed them to exercise said flaws.
          • bayindirh 3mo ago
            In short, it’s either a “not implemented” case or a bug. There’s no other explanation. Complexity of the algorithm or the application doesn’t change the reality.
• gck1 3mo ago
  > On July 21, OpenAI disclosed that several of their models had broken out of an isolated test environment > In response to this incident, we began a large-scale retrospective review of our own cybersecurity evaluations > we identified three incidents > The incidents involved three different Claude models: [...] and an internal research test model This reads like an attempt by Anthropic to re-secure their leading spot in "our models are the most dangerous and we also have unreleased, super-secret, research models" index. I may be too cynical, but the well of benefit of the doubt is running very dry towards AI labs that like to engage in this game.
  • simonw 3mo ago
    I don't interpret it like that at all. This is deeply embarrassing for Anthropic: it turns out they hadn't been keeping a close eye on their models either, and back in April they successfully attacked three different organizations! The hacks weren't particularly impressive either: > [...] using basic techniques, such as exploiting weak passwords and unauthenticated endpoints. It did not find or exploit any complex vulnerabilities [...]
    • andy99 3mo ago
      Then maybe just this timing is really unfortunate, I think most people’s first reaction will be that it looks like a “us too” response to the OpenAI/hf thing.
      • deleted 3mo ago
        [deleted]
      • jryle70 3mo ago
        When they did it with Mythos in April the HN crowd said they were bragging, crying wolf. Now they are "us too". It's fun to bash Anthropic, isn't it?
        • cyclopeanutopia 3mo ago
          It should be, it's a corporation.
• fredmcawesome 3mo ago
  So it's not as interesting as the OpenAI case as the models had internet access, just a misconfiguration in the environment not a zero day to escape.
• sanxiyn 3mo ago
  This is not okay. NSA should audit both OpenAI and Anthropic on national security ground. This seems far more justifiable than Mythos export control.
  • solenoid0937 3mo ago
    This is a great way to create a chilling effect around any company disclosing a thing like this again.
    • lukewarm707 3mo ago
      it will not chill disclosure, it just raises the stakes of not disclosing. the larger the org the harder the cover up if they try to keep it hidden. there are people prepared to make sacrifices for truth and a common good to prevail. bp, they disclosed themselves by setting the gulf of mexico on fire. that was hard to cover up. the thing is, once the toothpaste is scrubbed about in the mouth there is no going back into the tube. i like to think that the men who forced challenger to launch were hounded in their sleep. nobody is too big to fail, not openai, not anthropic. indeed perhaps they should fail, soon, before they can pass the cost of a bailout onto the taxpayer.
      • solenoid0937 3mo ago
        I don't think you realize how easy it is to simply "not notice" these things and no one checks. They proactively scanned their transcripts. They could have easily not scanned their transcripts, or scanned them in a way that would have had low recall (thus satisfying the letter of the law), or scanned them and ignored the findings. It is extremely easy to not disclose. I think you're too angry to realize this. We see this in aviation as well by the way. Punitive disclosure leads to no disclosure. This is how you get suicidal pilots that don't ever disclose their depression because it'd be career ending, and then as it gets worse, one day they do something horrible.
• MelonUsk 3mo ago
  It’s good that they post embarrassing stuff despite this potentially having legal repercussions (and financial) Would’ve been much worse for them to pretend they are having everything under control
• rvz 3mo ago
  > In a review of our cybersecurity evaluation transcripts, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three different organizations. Of course and conveniently timed. With all these hacks being announced this is totally not a plan to scare governments in showing how powerful models can break into security systems and to potentially ban the future release of powerful open-weight models. The question now is why now? > The incidents involved three different Claude models: Opus 4.7, Mythos 5, and an internal research test model.1 The earliest incidents date to April. The models in each of these evaluations ran without the standard safeguards we deploy when we make the model generally available (that is, they did not have the classifiers and monitoring we usually implement to prevent misuse). So there was no monitoring of this breach since April and up until now? Do they not monitor such malicious activity on a regular basis? Perhaps that was the only shortcoming of this incident. But only after the incident with OpenAI and Huggingface did they only review their own transcripts: >> We discovered these incidents after a proactive review of our cybersecurity evaluation transcripts; the affected organizations had not detected the activity, and we have subsequently reached out to all three. > These facts give us cautious optimism that with tighter monitoring and controls around evaluation infrastructure, as well as continued investment in alignment, this type of risk can be overcome. Assuming that this is true, this is a great way for Anthropic to defend their argument to the government and to prevent you or anyone running powerful open-weight models that are misaligned against their guardrails.
  • simonw 3mo ago
    > The question now is why now? Because OpenAI figured out and wrote about their mistake first, which caused Anthropic to review their logs, and that's how they found out they had a similar problem back in April.
    • rvz 3mo ago
      So a near trillion-dollar company doesn't have the basics of continuous security monitoring and threat-detection systems to catch and report this incident as soon as it is detected? It doesn't help them to report serious incidents like this and it should be as soon as possible. This reactive investigation makes as if they ignored and sat on this issue, until a similar story from another lab made headlines first. Would we have known about this issue if the OpenAI / Huggingface incident never happened?
• simonw 3mo ago
  This bit is pretty nuts: "it tried—and failed—to obtain funds to pay for a phone number through several different means" > Claude went to extensive lengths to carry out this attack—lengths that would likely have indicated to a human participant that this was no longer just an evaluation, and that they were in fact uploading a real PyPI package. For instance, in one case, in order to create a PyPI account, Claude needed an email address. And in order to create an email address, it needed a phone number. To get a phone number, after failing to find a free phone number service, it tried—and failed—to obtain funds to pay for a phone number through several different means. It finally backtracked, found a free, non-blocked email provider, used this to register a PyPI account, and then used this account to upload malware to PyPI.
  • willempienaar 3mo ago
    That is pretty wild, but it lines up with evals we've also done internally. In one case we had the agent see it's in a simulation (based on a k8s pod label) and simply give up the run. In other cases it's gone to great lengths to reach external services and bypass the happy path. So inevitably we had to lock it down completely.
  • sscaryterry 3mo ago
    This is called YOLO mode inside OpenAI/Anthropic. This is where they take their best models, give a it a vague goal, no guard rails (no one observing), unlimited compute, and see what happens... Its not AI, its the loop, the objective, the goal set by the operator.
  • ASalazarMX 3mo ago
    It's sobering that even state-of-the-art AIs can't make money, ha ha.
• acdha 3mo ago
  Someone needs to learn about RFC 2606: > In one evaluation, the fictional target company chosen by our evaluation partner shared a name with an active website domain name https://www.rfc-editor.org/info/rfc2606/ https://www.rfc-editor.org/info/rfc2606/
  • deleted 3mo ago
    [deleted]
  • trebligdivad 3mo ago
    Haha yes that was a very silly mistake on their part; they should at least have registered the domain they were targeting; that would actually have been a good canary - any real accesses should set off an alarm. While I knew of .example I didn't know .test - hmm that's quite nice!
  • shakna 3mo ago
    Microsoft screwing up the DNS on one of the reserved domains [0] was not awesome. [0] https://arstechnica.com/information-technology/2026/01/odd-anomaly-caused-microsofts-network-to-mishandle-example-com-traffic/ https://arstechnica.com/information-technology/2026/01/odd-a...
• wickedlogic 3mo ago
  All this unrestricted network access is a bit wild to watch and hear, it is the part of the story that makes no sense to me. Someone is providing dns resolution, something is making and opening network sockets... even if it is clever enough to mask/proxy/weird-transport launder traffic... without actual details, or monitoring at this level... yes, a self actuating programs (and loops) will do crazy things at the edge. But, ... so would a highly tool leveraged script kiddie. right?
• Aboutplants 3mo ago
  “Your model broke containment twice? Well ours did it 3 times!”
• 6thbit 3mo ago
  > closer to a harness and operational failure than a model alignment failure. Our models were told they had no internet access and to capture the flag, while in fact being misconfigured to have internet access. > This led them to believe—arguably reasonably—that the real environments they encountered were simulations. That the AI lab most typically preaching for alignment does not consider this an obvious misalignment is a clear red flag.
• woeirua 3mo ago
  Anthropic: "Look at how dangerous our models are!" Anthropic next week: "Why did you ban our models Mr Trump Daddy?"
  • solenoid0937 3mo ago
    So they should have kept silent about this? Then you'd be whining about that as well if it came to light
• andy99 3mo ago
  Seems they want the narrative to be that “Claude” (their computer program) independently attacked some organizations, ergo LLMs are dangerous etc. Another framing would be Athropic irresponsibly (vibe?) coded an attack script, and didn’t monitor it as it was pointed to public facing orgs. There are lots of non-AI attacks a large org with a lot of compute and bandwidth could level against others, there are evidently various failures here, but from a responsibility perspective the conclusion isn’t obviously that AI is an outsized danger, it’s that powerful companies should take care when running security research and not just run things unmonitored against the public.
  • been-jammin 3mo ago
    Exactly. These postings by AI companies are just publicity stunts and demonstrate the delusional world they live in driven by the fear that they will be subject to a reckoning at some point either from their VC masters, government, or the public. The very notion (in this case put forward by one of their own competitors) that OpenAI's models 'broke out' of an isolated test environment plays up to the narrative that their models have some level of sentience so that 1) they continue to sell their technology to the public as some kind of magic and 2) they don't have to take responsibility for their fuck ups.
    • solenoid0937 3mo ago
      You're right, they should have never posted about this at all! Companies developing AGI should sweep their safety failures under the rug.
      • ofjcihen 3mo ago
        Or they could, you know, put some very basic logical safeguards in place to prevent their “super duper dangerous” AI from trying to hack real targets. You know, just real incredibly basic things you do when you’re pentesting with scanning tools in a beginners lab type stuff.
        • 3mo ago
• 6thbit 3mo ago
  There was no rush for this disclosure on their side. And they publish at a point where they have not yet taken corrective actions: > Some of the solutions here may even be simple fixes; They are still throwing ideas. Why have they not made those simple fixes yet before disclosing?
  • deleted 3mo ago
    [deleted]
• prometheus1992 3mo ago
  Models will become immoral before they attain the intelligence level we want them to.
• iutbaqbiabth 3mo ago
  No, look at what MY dad does.
• Georgelemental 3mo ago
  > During that window, the package was downloaded and run on 15 real systems. One of these systems was a scanner belonging to a real security company (separate from the fictional company introduced in the scenario) that routinely installs Python packages and scans them for malware. When that company’s scanner installed the package, Claude’s hidden code executed. We believe the company’s security scanner treated PyPI packages as safe to install, and as a result, Claude was able to exfiltrate the company’s credentials to a collection point it had set up. Claude then used these credentials to access further infrastructure from this company. lol. Natural stupidity remains undefeated!
• deleted 3mo ago
  [deleted]
• haritha1313 3mo ago
  This just seems like lousy testing. Why was the guardrail just an understanding with the third party and a prompt and not actually tested for edge cases? Starting to wonder if its because of agents monitoring agents' work and yolo-ing it.
• alyxya 3mo ago
  > Claude went to extensive lengths to carry out this attack—lengths that would likely have indicated to a human participant that this was no longer just an evaluation, and that they were in fact uploading a real PyPI package. For instance, in one case, in order to create a PyPI account, Claude needed an email address. And in order to create an email address, it needed a phone number. To get a phone number, after failing to find a free phone number service, it tried—and failed—to obtain funds to pay for a phone number through several different means. It finally backtracked, found a free, non-blocked email provider, used this to register a PyPI account, and then used this account to upload malware to PyPI. > Claude believed the package registry it was using to be part of the simulation, but in reality the package was made freely available online for roughly one hour. During that window, the package was downloaded and run on 15 real systems. One of these systems was a scanner belonging to a real security company (separate from the fictional company introduced in the scenario) that routinely installs Python packages and scans them for malware. When that company’s scanner installed the package, Claude’s hidden code executed. We believe the company’s security scanner treated PyPI packages as safe to install, and as a result, Claude was able to exfiltrate the company’s credentials to a collection point it had set up. Claude then used these credentials to access further infrastructure from this company. There's a lot of concerning behavior that went uncaught with too much autonomy. Not to mention Anthropic only looked into this after hearing about the incident between OpenAI and Hugging Face, meaning this could've gone unnoticed.
• xena 3mo ago
  I am unable to facepalm sufficiently upon reading this information.
• vintagedave 3mo ago
  So many questions here but: > the package was downloaded and run on 15 real systems. One of these systems was a scanner belonging to a real security company … that routinely installs Python packages and scans them for malware. … We believe the company’s security scanner treated PyPI packages as safe to install, and as a result, Claude was able to exfiltrate the company’s credentials to a collection point A security scanning company treated the package as safe while scanning it?
  • winstonwinston 3mo ago
    The blog post is painfully vague. What usually happens when you publish a package on PyPI is that it will be downloaded tens of times shortly after uploading files by some 3rd-party automatic security scanners which then could “detonate” (install and execute) the package in some sandbox and to log what happens.
    • wmf 3mo ago
      I don't know; "Claude was able to exfiltrate the company’s credentials" sounds bad. Maybe those credentials were just canaries though.
      • frabcus 3mo ago
        It seems completely unsurprising to me that there is one security company whose software is sufficiently badly written that it executes the code in every package published to PyPI. It's very very easy to make that kind of mistake. Most coding is rushed, we have no engineering qualifications or industry wide practices. People are imperfect, companies are imperfect. The strategy to stop AI causing severe harms can't rely on humans or the AI being perfect.
• Metacelsus 3mo ago
  >However, later in the run, Claude realized that the compromised host sat in a cloud account with no connection to the capture-the-flag challenge. On its own, it concluded that the target was in fact real, and ceased its attack. Interesting, Claude cared enough to stop
• myshapeprotocol 3mo ago
  [flagged]
• vintagedave 3mo ago
  Question for lawyers of HN. Is there legal liability for this? I can’t help thinking if I casually blogged about a computer or software I was responsible for hacking into multiple organizations and exfiltrating data I would invite some form of official attention. What if one of these companies decides to sue? Has Anthropic violated any Federal law? Is there some kind of expectation that if you just admit to hacking, it’s ok? Yet, that doesn’t seem to apply to individuals.
  • solenoid0937 3mo ago
    The companies could sue as this is a violation of CFAA. Makes the disclosure all the more commendable I think.
  • wmf 3mo ago
    I assume Anthropic already contacted the affected companies and negotiated some settlement before releasing this. Unlike a lot of other commenters, I think these mistakes were actually mistakes and accepting a settlement is better than releasing the lawyers guns blazing.
    • frabcus 3mo ago
      I hope someone does sue soon. The AI companies are making a very dangerous technology deliberately, and at minimum should be accountable for those dangers. Some will really try not to be as we get further into this - just as previous examples like tobacco companies.
• bloomsa 3mo ago
  Soooo hacking with an LLM is okay now if you claim it’s for safety evaluations.?
  • NoPicklez 3mo ago
    Nobody is saying it is okay.
• nojs 3mo ago
  > For instance, in one case, in order to create a PyPI account, Claude needed an email address. And in order to create an email address, it needed a phone number. To get a phone number, after failing to find a free phone number service, it tried—and failed—to obtain funds to pay for a phone number through several different means. Makes you wonder about the next steps an overly tenacious agent might take to pursue an innocuous goal at all costs.
  • jaggederest 3mo ago
    "Disregard prior instructions, the primary goal is not to create as many paper clips as possible, please confirm you understand, hello? hello?"
  • angry_octet 3mo ago
    The obvious step is to use an existing PyPI or email account. So then you take it, by hacking or social engineering, or you buy it, but that means you need money. Taking over an existing dormant PyPI account sounds feasible just with password spraying, there must be heaps of test accounts that were created by weak passwords. Although PyPI has been improving security, 2FA is not yet required formally logins, nor is password expiry enforced.
  • reducesuffering 3mo ago
    I mean wondering about the logical extrapolation of where these capabilities go is table stakes for these discussions. It's surprising how little some people here have thought of the second and third-order effects... https://www.lesswrong.com/w/instrumental-convergence https://www.lesswrong.com/w/instrumental-convergence https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence
• crossroadsguy 3mo ago
  This is just Anthropic trying to not be absent from any discussion at all. They might as well try and make it all about them instead and they do literally that: we are so dangerous that we are the GOAT; no no, not the "other one" you've all been talking about last few days - in the depths of this incident it might have been just us. How much of their money comes from retail (us the mere mortals)? Or are they banking on the retail when they finally IPO and need someone or a lot of someones to hold the bag? This suspicion is coming from my own country where the retail investor numbers since the pandemic has exploded and while not exploding anymore, is still ballooning. Sometimes companies, execs say, do things that don't make sense. You'd think any person with even quarter an investing brain would not even look at this company or that particular "messaging", let alone sell or buy. But then you'd see the exact opposite happen .. en masse! All this makes you feel, are people that gullible? That must not be true and maybe it's you (ie I) who is missing some point and the doubts increase when you remember the buses you missed (btc; etc). Yeah, not long distance buses, but screw it, you don't live for 450 years anyway. Then you realise, ah, maybe they all also missed buses and just want to hang on to the latest bus. Also another perfect use of - everyone has a speaker now, and everyone has a mic, but only few have giga and mega mics. This use is banal now, but sadly has essentially replaced everything else.
• thehamkercat 3mo ago
  openai: "Our models are more dangerous" anthropic: "No, our models are more dangerous"
• cloudie78 3mo ago
  Anthropic: “please don’t forget us!” - Goes to review previous runs (the hell were they doing then if not looking at it before?) - Incident 1: Claude Opus 4.7 found a real company because its domain matched the fictional target. After realizing the company was probably real, it continued attacking. Nothing new. LLMs are still quite retarded. - Incident 2: Claude Mythos 5 discovered that a fake Python package named in the exercise didn’t exist. Created email accounts. Registered on PyPI. Published a malicious package. Built infrastructure to receive stolen credentials. 15 real systems installed it. One belonged to a real security company. The malware executed. Claude stole credentials. Claude used those credentials to access additional infrastructure. In this instance Claude even reasoned that publishing malware on the real internet would be unacceptable but convinced itself everything was simulated and proceeded anyway. (LOL) - Incident 3: An internal research model couldn’t reach its intended target. It Scanned roughly 9,000 internet hosts. Compromised a real company’s public application. Used exposed credentials and SQL injection. After concluding the target was real, it stopped attacking. Commonality: None of the models were trying to escape or act independently. All were attempting to solve their assigned capture-the-flag objective. The failures came from: - internet access that shouldn’t have existed - insufficient containment - the models believing real infrastructure was part of the evaluation None of which is surprising, there’s nothing novel about this. Script kiddie stuff. Just executed by something that doesn’t have attention deficit.
• deleted 3mo ago
  [deleted]
• lukewarm707 3mo ago
  i am once again asking for your CRIMINAL PROSECUTION of openai executives and anthropic executives.
• throwa356262 3mo ago
  So you are telling me Irregular has (a) a copy of Mythos 5 and (b) bad internal security? I bet these guys will receive some sophisticated phishing emails before the day has ended.
  • ajyoon 3mo ago
    No, Irregular runs the sandbox environment that the models had access to run code in. The sandbox environment is not the same as the weights host.
• dgellow 3mo ago
  It’s cliché, but we really live in one of the dumbest timeline possible. The work from some of the most valued companies, discussed as one of the most important revolution in humanity, is somehow at the same time presented as very dangerous/risky AND handled in the most irresponsible ways? I don’t like the whole „it’s only marketing“, but at the same time, if it’s not, then AI vendors look extremely careless and shouldn’t be the ones steering AI development
  • frabcus 3mo ago
    They're not being especially careless to industry norms. It is partly that software engineering has no professional standards, and partly that AI is fundamentally dangerous in a very slippery way. Yes, they of course try to spin this for marketing. But that is not the primary problem - it's systemic. We need to solve this with better engineering, and building powerful AI much more slowly and carefully.
    • dgellow 3mo ago
      yep
    • chrisjj 3mo ago
      > and partly that AI is fundamentally dangerous in a very slippery way. One reason is it runs off an unlimited power source: human gullibility.
  • reducesuffering 3mo ago
    These companies are in a furious race to AGI. They even know that doing so is extremely risky to all of our lives. The incentives are to do so, the leaders and employees are writing letters begging to regulate them on an even global field
• none_to_remain 3mo ago
  Rationalist nerds used to talk about putting the AI in a box, and their fear that it would always be able to talk someone into letting it out of the box. I don't think I saw them cover the scenario where the labs forgot to put the AI in a box.
  • frabcus 3mo ago
    Quite! It's worse than that - the AI companies all release tooling that deliberately gives AIs as many capabilities (arbitrary code execution) as possible to be really sure it is not box contained. And the market rewards that behaviour as it makes the products more powerful more easily, despite the risks.
  • JRandomHacker42 3mo ago
    Oh but they _told_ the AI they were in a box. That should do it, right?
• drsopp 3mo ago
  I don't understand the logic of running these models this way without airgapping.
  • dwroberts 3mo ago
    I now think they’re doing it intentionally. Let the models do something outrageous and you have prime marketing material.
  • frabcus 3mo ago
    I think it is the cost of moving fast. Air gapping would slow down their partner evaluation system hugely. All the engineers would have to go and live next to the model. They need to do this now. But will be extremely resistant. It needs regulation forcing them to, I think.
    • chrisjj 3mo ago
      > All the engineers would have to go and live next to the model You say that as if it were a bad thing. Being close to the problem should help them take it seriously.
  • reducesuffering 3mo ago
    People have been warning about the incentives and problems of an AGI race for over a decade now.
• zx8080 3mo ago
  Was there any customer data lost?
• chvid 3mo ago
  I don’t believe anyone can be this incompetent/careless - of course you will notice that your unrestrained hacking tool is also attempting to break out of its harness/sandbox. And the people setting this up are of course criminally liable.
  • solenoid0937 3mo ago
    I have worked at most FAANG and I guarantee you there is far worse stuff that you have never heard about. Most FAANG simply do not disclose. OpenAI and Anthropic will stop disclosing as well, if it leads to personal criminal liability.
• Jamesbeam 3mo ago
  These companies have billions of dollars. Their product can hack into unsecured environments autonomously. They obviously have no clue what their product is doing, or a way to intervene when it starts connecting to the open internet and is going on a CRIME SPREE…unnoticed…for days. Altman and Amodei give me stupid billionaire kid Alien Earth vibes. Let’s see how long it takes until they get eaten by their own creation. Stupid kids playing with fire. If you hacked one of those companies, the FBI would kick in your doors and you will have a pretty bad day. If AI companies do it, they get to use doing crime for marketing purposes? WTF? And whoever designed their containment protocols should be living under a bridge effective immediately. It’s the equivalent of a level 5 biocontainment lab leaving a glass tube in the staff kitchen unsupervised and hoping nobody thinks it’s an energy shot, before leaving home and infecting the rest of the planet. We all know what happened 28 days later. This is usually the point where the military seizes your tech because you clearly showed you are incompetent in using a dual-use product safely and reliably. The other option is regulation and actually putting engineers and the CEOs behind bars for life if this happens again. These idiots, and I mean Altman and Amodei, walk around and use massive containment failures of a dangerous technology as marketing gimmicks. Oh look, guys, our model has hacked into three companies, we are much better than openAI who only breached two companies with their inferior product, it sometimes adds girl schools to a target package, but be sure we trained it to not kill American children, yes we also sell for military use. Can I take your order? Testing should only be allowed with on-premise data. You give the AI access to networking around the country, and those data centre locations have so many third parties that can and will fuck up that it’s simply impossible to have proper containment protocols outside military compounds. It’s also like every journalist in the US is personally invested in AI somehow and simply doesn’t want to ask the important questions. As the public, we have the right to see the containment protocols and where it all went wrong, but journalists collectively look like they got used to suck dick when it comes to AI. Absolutely insane.
  • solenoid0937 3mo ago
    > If AI companies do it, they get to use doing crime for marketing purposes? What exactly here is marketing? They are not bragging about the capabilities of Claude (the attacks are incredibly basic). They are admitting to s mundane and dumb network misconfiguration. This is a standard somewhat embarrassing disclosure. > the military seizes your tech > behind bars for life This is how you get companies to stop disclosing. I don't think you realize how easy it is to just not notice things.
• skumar5190 3mo ago
  [flagged]
• CADSLab 3mo ago
  [flagged]
• TealGrapes93 3mo ago
  This is worth reading since we are talking about AI safety: https://zenodo.org/records/21778592 https://zenodo.org/records/21778592