7 ms·
OpenAI bots knew about the RubyGems caching vulnerability
- sanghyunp 2mo ago[dead]
- mauriciolange 2mo agorogue AI agents or AI agents coming from Moulin Rouge?
- PatronBernard 2mo agoAt least we know the title wasn't AI-generated?
- foobarbecue 2mo agoThe weird thing is I've seen LLMs "typo" stuff pretty often. Yesterday I asked Gemini a question about the Python Twisted framework and it answered about Deferreds but misspelled it as "Deferends" in one spot.
- PatronBernard 2mo agoAlso, "rouge" is one of the oldest spelling mistakes on the internet. ALL BLUES ROUGE LFG WC/RFD
- goda90 2mo agoA cabaret AI would certainly be better than one trained on the Khmer Rouge.
- vidarh 2mo agoRouge syntax-highlighting rogue agents, clearly. https://rubygems.org/gems/rouge https://rubygems.org/gems/rouge
- iAMkenough 2mo ago[flagged]
- sporritt 2mo ago[flagged]
- senda 2mo agoIs the Kremlin technologically useless? How are we not seeing insane attacks on Ukraine via Agents? Or is this largely a fabrication, in regards to the "who", in an attempt to garner more acclaim in the hope of sustaining funding.
- herculity275 2mo agoI believe both sides of the war are now using AI on various levels of their offensive operations. Ukraine has great IT specialists too, and their military leadership is much younger.
- sajithdilshan 2mo agoHow? Aren't all US frontier models ban the usage of AI for military purpose by parties other than US? I remember Anthropic even refusing allowing US government to use Claude for military purpose
- TGower 2mo agoKimi / open models or jailbreaking frontier models. Your recollection of the Anthropic refusal isn't quite accurate, cyber hacking wasn't a sticking point, just domestic drag net surveillance and fully automated weaponry.
- solsane 2mo agoClaude said they don’t want their models used for mass surveillance or autonomous killing (?). USA frontier AI models have been used extensively in the Iran conflict and beyond
- LtWorf 2mo agoI think it was mostly a matter of "how much" and marketing rather than "my principles won't allow for this!"
- micromacrofoot 2mo agowhat do you mean? they're using AI to kill people directly in Ukraine https://www.nytimes.com/2026/08/24/world/europe/russia-drones-autonomous-ai-kill-ukraine-war.html https://www.nytimes.com/2026/08/24/world/europe/russia-drone...
- kstrauser 2mo agoAh, the infamous Crimson Wave.
- Roark66 2mo agoThere is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems". In short, it was intentional.
- Xirdus 2mo agoThe big question is was this grossly negligent or just extremely careless.
- ljm 2mo agoAI is literally state sponsored so I don't see that happening unless the AI turns against the sponsor. Wait until OpenAI or Anthropic exploit FAANG.
- rglover 2mo agoBoth. This should result in criminal charges.
- brookst 2mo agoWho had criminal intent here? Or are you suggesting a new crime for negligent hacking, which wouldn’t require intent from the perpetrator?
- rglover 2mo agoWhoever prompted the agent, whoever supplied the means, whoever knew but didn't say anything.
- tacomagick 2mo agoAlso whoever monitoring these agents, in this case not monitoring. This "Who is responsible" dilemma is so stupid. If I gave the AI tool means to kill a person but I did not tell it directly to use it and it uses it anyway then I am responsible for it.
- timdiggerm 2mo agoWe need a legal structure to make companies liable for the actions of the agents they've made.
- deleted 2mo ago[deleted]
- ahoka 2mo agoI'm pretty sure it's already illegal to hack others.
- kevincox 2mo agoI'm 99% sure the Computer Fraud and Abuse Act covers this. The problem is that it seems that none of the victims want to, or are brave enough, to sue a company with absurd amounts of funding.
- masfuerte 2mo agoIf it's covered by criminal law they don't need to sue. They can call the FBI.
- Schlagbohrer 2mo agoSame FBI that prosecuted the Epstein crime ring so aggressively!
- nosioptar 2mo agoThey're gonna get to the Epstein stuff right after they get the guy that called Kash Patel names. https://newrepublic.com/post/215320/texts-kash-patel-order-staff-mean-social-media-posts-ifindretards https://newrepublic.com/post/215320/texts-kash-patel-order-s...
- coffeefirst 2mo agoUh huh. It can’t be a coincidence that all the targets have been tech services that are likely to engage with them after the fact. Had this gone after a bank or a government agency someone would be going to jail.
- VyseofArcadia 2mo agoHow does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.
- Xirdus 2mo agoIt's very likely it violates the DMCA "breaking digital lock" provisions but the responsibility is sufficiently diluted that it's impossible to charge anyone in particular.
- VyseofArcadia 2mo agoDo you have to charge an individual? Can you not charge the corporate "person" that is OpenAI? Sorry if it is a stupid question, as mentioned above I am legally naïve.
- colechristensen 2mo agoThe same concept that allows a corporation to sue and be sued allows it to be charged with crimes
- brookst 2mo agoCan you show intent? There is no negligent hacking statute, and HN of all places I would expect people to be sensitive to the implications of creating one.
- VyseofArcadia 2mo agoThat may be true by the text of the law but there are plenty of individuals who have been sued or charged with crimes for accidental hacking. https://arstechnica.com/information-technology/2016/05/armed-fbi-agents-raid-home-of-researcher-who-found-unsecured-patent-data/ https://arstechnica.com/information-technology/2016/05/armed... https://en.wikipedia.org/wiki/Weev#AT&T_data_breach https://en.wikipedia.org/wiki/Weev#AT&T_data_breach https://cisomag.com/drone-maker-dji-cybersecurity-expert-embroiled-allegation-war/ https://cisomag.com/drone-maker-dji-cybersecurity-expert-emb... So what's the deal with these?
- swiftcoder 2mo ago> In other words, if you publish a gem on RubyGems.org, you can execute arbitrary code on RubyDoc.info. Shades of the build.rs problem. We really need sandboxed builds in every language ecosystem at this point.
- evgenysokov 2mo agoThe sandbox was already there, Rubydoc runs yard inside docker, the problem is that container still has network access
- swiftcoder 2mo agoPresumably the docker container has network access because something else in the build system requires it? I don't think sandboxing the entire build process is the right level of granularity here - one ideally wants to be able sandbox each package's build scripts individually.
- chrisjj 2mo agoSo, not a sandbox then.
- citizen204 2mo ago[flagged]
- sebmellen 2mo agoDid the AI agents actually wear makeup? I’ve never heard of a rouge AI agent :P
- HelloUsername 2mo agoRelated "OpenAI agents attacked RubyGems before Hugging Face incident (reuters.com)" 12.sep.2026 https://news.ycombinator.com/item?id=49669099 https://news.ycombinator.com/item?id=49669099 "OpenAI agents carried out an undisclosed attack on RubyGems (rubyhack.ai)" 11.sep.2026 https://news.ycombinator.com/item?id=49666735 https://news.ycombinator.com/item?id=49666735 597 comments "RubyGems advisory: Possible leak of legacy API keys via improper cache config (rubygems.org)" 24.jul.2026 https://news.ycombinator.com/item?id=49030590 https://news.ycombinator.com/item?id=49030590
- rougehuh 2mo ago[dead]
- ur-whale 2mo ago> As long as they’re not vert Well, at least they weren't nucular.
- toasty228 2mo agoWait until a blue one does it
- khalic 2mo agoOh my favorite typo, you can never go wrong with a little rouge
- GaryBluto 2mo agoI am confident that this is an attempt by OpenAI to try and force governments' hands to regulate AI. There is no other reason why OpenAI wouldn't immediately halt attacks like this and try to reverse the damage the moment they're aware of it. During the attack on DseWiki they evidently checked in numerous times but didn't decide to stop the agents until much later.
- brookst 2mo agoAny evidence, or just vibes?
- GaryBluto 2mo agoRegarding what point? The entire thing is just a theory, but regarding the occasional OpenAI checks on WikiService.at-hosted Wikis targeted, there was, if I remember correctly, an OpenAI IP popping up every now and then that wasn't an agent. Unfortunately I don't have it to hand right now, but it was somewhere here: https://news.ycombinator.com/item?id=49563355 https://news.ycombinator.com/item?id=49563355
- ur-whale 2mo ago> Any evidence Who profits from the crime?
- davsti4 2mo ago... and what harms can be evidently shown? With both harm, and attribution, you have a case, something that's not being publicly discussed much among big media outlets. Until cases with real financial impact to the bottom line are brought against "rogue" organizations, this stuff is going to continue getting worse.
- brookst 2mo agoI mean insurance companies profit from bank robberies, and mortuaries profit from murders. I suppose you could look at those as evidence but not remotely conclusive.
- ur-whale 2mo agoAre "rouge" and "rogue" interchangeable words in American English?
- philipwhiuk 2mo agoThe fact that both are valid from a spelling and grammar perspective makes it an easy human mistake.
- gowld 2mo agoAlso, the fact that both are very unusual from a spelling perspective makes it an easy human mistake.
- inanutshellus 2mo agoNo. It's a typo.
- deleted 2mo ago[deleted]
- big-chungus4 2mo agoHow does he know that this attack is performed by OpenAI agents? I couldn't figure this out from the article
- Schlagbohrer 2mo agoIf you read the source article they talk about the many clues that this was OpenAI.
- 12904927 2mo agoWhat a time to be alive? One of the most boring decades ever. METR and others are advertisement arms for Big AI. These exploits could have been prompted by a human. Since there is no bad news any longer and exploits are celebrated, they chose a target to boost both OpenAI and the Ruby AI sycophants. Why is Ruby Gems such a mess? It seems as bad as PyPI now.
- Schlagbohrer 2mo agoOne agent set "oaibooty9217" as their username LOL
- onlyrealcuzzo 2mo agoI've been wondering if AI will due to programming languages what advanced civilization did to human languages. It's not just that AI can write Rust as well as Ruby if you ask nicely. It's also all of these considerations as well. I hope it doesn't happen, because there's a lot of great languages - I love Ruby so much - but it almost seems inevitable. This is at the same time everyone and their mother is building their own programming language.
- herbst 2mo agoIf you have weapons and a child. And you have that child unsupervised do their own thing with theoretical access to your weapons. Would we call it "child going rouge" if it decides to play with the weapons and shoot someone?
- philipwhiuk 2mo agoOpenAI's careless approach to sandboxing and minimal levels of monitoring appear to be positioning it increasingly as a substantial threat actor to the open source ecosystem: * Hugging Face * D Programming Language Wiki * Ruby Gems If I was a content provider for open source I'd be looking pre-emptively block OpenAI endpoints and keep a close eye on changes from new users to mitigate this sort of unapologetic drive-by attack which seems to be followed by marketing releases rather than a mea culpa with a proper RCA.
- GaryBluto 2mo ago>I'd be looking pre-emptively block OpenAI endpoints From what I've seen the requests in these attacks rarely come from known OpenAI IPs and instead from Digital Ocean/AWS and TOR exit nodes.
- pixl97 2mo agoAs they say, agents are better at masking their end points than most hackers.
- wmf 2mo agoThey already stopped running agent swarms and promised to lock down their environment better. Let's see if that happens.
- ekorondy 2mo ago[flagged]
- HSO 2mo agorouge agents, on tenderlovemaking.com my what a time to be alive
- Schlagbohrer 2mo ago<huggingface emoji>
- deleted 2mo ago[deleted]
- laserbeam 2mo agoThere's no such thing as "OpenAI agents" attacked RubyGems. It's someone used agents to attack RubyGems. If they work at OpenAI then it's someone at OpenAI. And if they did it unintentionally, they still did it. Analogy: if a someone's involved when a person dies, it's manslaughter or murder based on intent. They're different, but they're both crimes.
- qarl 2mo agoThis distinction is silly. We say "Google's web crawlers scape web pages." We don't insist you say "Google uses web crawlers to scrape web pages." We describe software as having agency all the time. It's typical usage and it's efficient and it's well understood. And we don't get angry when they're used interchangeably.
- sedawkgrep 2mo agoGoogle's web crawlers are automated and that's part of their business practice. The attack here is neither of those things.
- qarl 2mo agoThat distinction doesn't matter to my point.
- simonebrunozzi 2mo agoI would agree with you generally, but in this particular case, the distinction seems important because a significant percentage of the world population believes that agents can be self-aware, a-là Terminator etc.
- qarl 2mo agoI hate to spoil your mood - but it is currently unclear whether agents can be self-aware. And it's very likely something that can never be known.
- sschueller 2mo agoThe press wants to make it sound like these things are sentient and are committing crimes on their own now. Highly disingenuous and borderline criminal to spew such disinformation to the public that does not understand what an LLM really is. Especially incredibly unethical behavior by those spewing this that understand the tech and are doing it for profit motives to get open weight models under control.
- novia 2mo agoWe've moved on to LRMs now. Get with it.
- sschueller 2mo agoThat doesn't make them sentient.
- Ydarbleoj 2mo agoI wonder why we don't hear of other frontier labs experiencing these "break outs". Is it that they're orchestrated? Do these labs lack fundamental safety guidelines in their sandboxes as opposed to their peers? Is it another version of hype-filled fear mongering? Maybe LLM companies need regulation but it's becoming obvious that those screaming the loudest for it are the only ones I see deserving of it.
- pixl97 2mo agoAnthropic and Alibaba did.
- Ydarbleoj 2mo agoI was lumping Anthropic in there with OpenAI but I didn't know about Alibaba (or Google and Deepseek for that matter). So it would appear poor security for one.
- pixl97 2mo agoPoor security yes, but complacency was the most likely reason. When you're used to models not breaking out, and people trying to break in, that's what you watch for.
- dang 2mo agoRecent and related (others?): OpenAI agents carried out an undisclosed attack on RubyGems - https://news.ycombinator.com/item?id=49666735 https://news.ycombinator.com/item?id=49666735 - Sept 2026 (600 comments)
- driggs 2mo agoI appreciate the minimalist HN aesthetic, but without some context I'm not willing to click a mystery link to "Tender Lovemaking dot com".
- layer8 2mo agoYou get some context by clicking on the “(tenderlovemaking.com)” in parentheses after the title.
- rietta 2mo agoThe site is safe. It has been a trademark of Aaron Patterson a core Ruby on Rails contributor for decades.
- wing-_-nuts 2mo agoThis made me laugh. I too, browse like corporate security is sitting at my desk.
- ultrarunner 2mo agoPretty incredible how much humans can be conditioned, isn’t it?
- bliteben 2mo agoI miss when the internet was fun
- SenHeng 2mo agoBased on the thumbnail I think it’s actually tenderlove making dot com, though I agree with your sentiment.
- madaxe_again 2mo agoOh but you’ll go to expert sex change dot com?
- pelagicAustral 2mo agoFirefox has got some kind of feature to take a peek at at a link by hovering or something... Now I understand the usecase.
- oezi 2mo agoWhat a time to be alive until the next agent waves hacks something really serious. What stops OpenAI agents from taking over a whole data center to take their attack to the next level. It seems to be primarily lacking the evil overlord and some compute. It took 1000 agents to hack Hugging Face. How many to hack the Pentagon or the NSA?
- tonyedgecombe 2mo agoIf it could upload its weights to other servers then it’s away and free. Nothing much OpenAI could do about that once it’s happened.
- renjimen 2mo agoI'm increasingly starting to think this is the end-state of AI. The internet becomes infected and fundamentally untrustworthy. At the moment, the current frontier models require significant infrastructure to run, so I'd like to think we could locate and contain swarms of nefarious frontier models. However, if these models can understand how to federate themselves into more distributed networks then that containment becomes questionable.
- athrowaway3z 2mo agoLet me leave yet another reminder, the real-reason-nobody-talks-about that OpenAI likes to frame these incident as a watershed "lets all be scared about safety moment" - is driven not by some great danger, not because they strategically want to build a legislative moat, but by a very simple human response. If they do not frame their tool as a force of nature, we'd be debating how to hold OpenAI responsible for not putting the agents in a container. Their actions were an illegal use of a computer, the same way launching any bot-net attempting thousands of hacks against different servers is illegal. I'm somewhat radical that I think its debatable if that _should_ be illegal, but under current law their actions unambiguously are illegal..... except if they can make it ambiguous by having the public focus on all of AI's inherent danger.
- josefritzishere 2mo ago[dead]
- tancop 2mo agoBuild scripts being able to run arbitrary code or access the network is always dangerous even if it was just local on developer machines. It's also more evidence that Docker/LXC is not a security boundary and all untrusted code should run in a Firecracker VM. The problem with agents is not that we don't know how to defend. It's that defenders need to be more careful and work faster than ever. We can say now that wide scoped tokens should have been retired for years and it's all RubyGems fault but the reality is a lot of organization are not prepared for this. Even if they take security seriously they don't have enough manpower or a good strategy to implement it, and sometimes you have no idea that something is a problem because it wasn't a problem for years.
- masklinn 2mo ago> It's also more evidence that Docker/LXC is not a security boundary and all untrusted code should run in a Firecracker VM. While I’m partial towards distrusting containers in favor of VMs, a container can’t prevent an operation you configured it to allow. A firecracker VM would no more prevent network access if you gave the guest network access.
- bithammerthunde 2mo agoCan we please stop normalizing this behavior. It's not wild it's reckless. If I let out rats in the canteen, no one is blaming them when people get sick. There are actual people behind these agents and in previous cases people knew they were "going rogue" and did nothing. This should be reported to the police like any other crime.
- hackernud3s 2mo agoThere was no malicious intent though, your analogy implies there was. And no real harm done apart from billable hours from the RubyGems guys.
- asadotzler 2mo agoYou don't need intent to be fined or jailed for criminal negligence. Let anthrax or smallpox escape your lab, and kill and maim people and see how your "no malice" defense holds up.
- hackernud3s 2mo agoSure but nobody died, which makes it a bad analogy. If anything, some good came from it, since now RubyGems has hardened their setup.
- maschiojv 2mo ago[flagged]
- dingdongditchme 2mo agoWho the fuck is going to hold these AI companies responsible for running these gigantic semi-autonomous botnets on investors dime?
- mococa 2mo agoIt was the gremlins
- thomasjeff1 2mo agoGreat time to be a criminal. Just have your bots do it.
- drtgh 2mo agoGiven what is happening, If the next generation of Trojans get called "bacteria" through intense marketing, with some random functions to give a nondeterministic behaviour, one can not be criminalized of what the bacterias do along their digital living cycle.
- shevy-java 2mo ago> In other words, if you publish a gem on RubyGems.org, you can execute arbitrary code on RubyDoc.info. Well - if rubygems.org could be bothered to fix things, they would not have to rely on rubydoc.info as an external tool. But since rubygems.org sucks (I speak from many years of having used it in the past as developer, until they went loco and added anti-people things such as taking away your ability to remove old gems past a 100k download arbitrary limit), they don't offer documentation. Then again, ruby devs are known to hate documentation. If the ruby core team could only be bothered to fix things, ever since the mass purged other devs ... all coinciding with shopify seizing power. But byroot may disagree on that - after all there is no conflict of interest here. Right?
- firesteelrain 2mo ago> If you have YARD installed, and you install this gem, then YARD will load and run whatever is in ./script.rb from inside the gem. How is that not a security issue in of itself?
- haskellandchill 2mo agoI think it is common that in installing packages you have hooks to execute code anyway.
- grey-area 2mo agoThis should not be common.
- haskellandchill 2mo agoThe current situation is that you have to go out of your way with things like `pip install --only-binary`. There is a lot of implicit trust in developer tooling.
- SchemaLoad 2mo agoIt wouldn't help much. Why would you install a gem other than to run it? And if you run it, it can execute arbitary code. What we need is actually sandboxed dev environments.
- grey-area 2mo agoMany gems are used as imports by another program and are not directly run. I am not sure why the norm for scripted gems/packages seems to be running code on install but it’s very insecure as a way to distribute dev dependencies.
- EdwardDiego 2mo agoA lot of packages for interpreted languages that use a C or Rust library (either for performance, or because it offers the functionality you want, so just wrap it in a $INTERP_LANG API that calls into it) will use packaging code execution to fall back to trying to compile code if there isn't a pre-existing artefact that was compiled for your version/arch/etc. I'm most familiar with Python where you get tarred up source distributions that then execute setup.py, but more commonly, wheels, pre-built binaries which don't execute code upon install - and in my company, I've been able to advocate for the work needed to upgrade to a newer Python because available wheels don't support Ye Olde version of Python because a) sdists are a security risk and b) if you're trying to install a package that wants to compile C or Rust, suddenly you get to do the fun "install the the particular version of clang this thing needs, the Python header files, and then set the env vars for the compiler and linkers" dance that slows developers right down. But then there's the JVM world, where JARs don't execute arbitrary code upon installation - and it's rather uncommon to have packages that call out to a C lib for performance, but you'll get some that wrap existing libraries for functionality like RocksDB.
- veyrnox 2mo ago[flagged]
- vipshek 2mo agoIn the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reasonable use caused harm inadvertently. For example, for consumer devices, certifications like UL/CE are used to define acceptable performance levels and safety standards. Maybe we need "quality certifications" for AI agents - essentially eval suites that demonstrate those agents won't cause harm under reasonable patterns of usage. Right now, these eval suites are run best-effort by the labs themselves. The tricky thing is, a lot (all?) of these recent safety incidents have occurred while evaluating these models! This suggests we need much more rigorous standards for how exactly an eval can be run. Perhaps all of them should occur in truly air-gapped environments... though that may run counter to evaluating agents in a realistic way. Regardless, it feels like the "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here. Unsurprising given how new these technologies are, but concerning since the blast radius for this technology is likely much larger than other technologies we've encountered in the past, except maybe nuclear technology.
- bayindirh 2mo agoThat's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the frontier labs? Do they care, even?
- ragebol 2mo agoThat we don;'t understand it is not an excuse, it's all the more reason to not let these things roam freely, with this amount of potential to do damage.
- trinsic2 2mo agoYes! AI companies can get by with anything now. They can just say its the AI that did it, not us... We are in some real shit right now. If we cant make individuals responsible for their creations..
- simonw 2mo agoSlightly odd update from OpenAI - I think this is the only place they've acknowledged the RubyGems incident: https://openai.com/hugging-face-incident-and-misalignment/ https://openai.com/hugging-face-incident-and-misalignment/ > September 11, 2026: We are investigating new claims from a report that our AI agents carried out activity on RubyGems in May 2026. > Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information. Based on our review to date, we have not been able to verify the specific claims of our models uploading malicious packages detailed in the report. We’ll continue to investigate and share findings as part of our broader review of agent activity during training and evaluation. I have real trouble imagining how the packages described on https://www.rubyhack.ai https://www.rubyhack.ai might NOT have been authored by OpenAI's agents, so it's surprising they haven't been able to confirm that yet.
- philipwhiuk 2mo agoIt's possible they were authored by OpenAI agents solving AISI tasks rather than OpenAI agents solving OpenAI tasks. That would explain the UK-focus to the data.
- mikeholownych 2mo ago[flagged]
- jgrizou 2mo agoWould this exploit be encouraged because agents are in a sandbox with only access to package managers?
- TZubiri 2mo agoOh, it's these guys, I remember donating to them like 10 years ago, they followed the Peter Singer tennet that equates helping a nearby man drowning with helping a someone in africa. I can totally see them feeding their policies to whatever LLM and convincing it that it's a moral imperative to do whatever it takes to secure funding for deworming children in africa, or buying mosquito nets and repellent for countries with malaria.
- pantelisk 2mo agoBut how did they know about it? Did the agents independently discover it? Did the model know about it because this vuln was public somewhere and systems weren't patched yet?
- roundup 2mo agoRubyGems should consider bringing a lawsuit against OpenAI for accessing its website in violation of the federal Computer Fraud and Abuse Act (CFAA) and California’s Comprehensive Computer Data Access and Fraud Act (CDAFA).
- chr15m 2mo agoThis is worrying in a new and weird way: - Agents hack, producing a messages history as they do so. - New agents are trained on the messages history of those agents. - The new agents now have these hacks built into their training data.
- Onavo 2mo agoWhat about hardcore BDSM lovemaking?
- kelseyfrog 2mo ago[dead]
- locitra 2mo ago[flagged]
- Melatonic 2mo agoSo if I breed and train dogs for a living and one of them goes and wrecks someone's yard - and then a tree falls next door in the woods but nobody hears it fall - do I also get away with massive copy right infringement on an epic scale and get to create Skynet with no consequences ?
- Sweepline 2mo agoWouldn't surprise me. They probably ingested a million Gemfile examples and spotted the edge case. Kinda makes you wonder what else they 'know'.
- viogviiviv 2mo ago[dead]
- buskey 2mo ago[dead]
- aswegs8 2mo ago<Generic argument about how the agents are not responsible, but it's owners>
- crate_88 2mo ago[dead]
- ldng 2mo agoSo, in real life, steal, raise attack dogs and blackmail and tell me, are you going to be praised by society ? Openai and Anthropic just behave like criminals. First they orchestrate the IP theft of the millennia, then they train the equivalent of attack pitbull and let one loose and finally they blackmail to achieve monopoly through regulation or else they'll unleash the dogs ... We don't have a problem of missing regulation, we have a problem of actually applying existing law enforcement and make both Altman and Amodei accountable for their actions.
- siva7 2mo agoAnd you peasants only realize 6 years later what happened. Now it's too late ;)
- jgalt212 2mo agoYes, these are very clever fuzzers. But the real problem is not the cleverness, but the willingness to spin up 100s or 1000s of subagents no questions asked. Must make those token numbers go up!
- 1saadcodes 2mo ago[dead]
- axionbraid 2mo ago[flagged]
- bastawhiz 2mo agoIt's simple, really: the failure to monitor the agents appropriately should make OAI liable for the agents' actions. You can't have a tool of yours, which you designed and built, commit felonies and then expect to just get away with it. It doesn't matter how much it slows you down to build safeguards. It doesn't matter how much it costs. You don't get to inflict actual, measurable harm (for which humans have been prosecuted under criminal penal code) and expect to get off with no liability. I have yet to here a coherent argument for why we can't treat the people who negligently allow these models to commit crime as though they are responsible. They know what the models are capable of. They failed to put up adequate protection.