11 ms·
OpenAI bots knew about the RubyGems caching vulnerability
- sanghyunp 18d ago[dead]
- mauriciolange 18d agorogue AI agents or AI agents coming from Moulin Rouge?
- PatronBernard 18d agoAt least we know the title wasn't AI-generated?
- foobarbecue 18d agoThe weird thing is I've seen LLMs "typo" stuff pretty often. Yesterday I asked Gemini a question about the Python Twisted framework and it answered about Deferreds but misspelled it as "Deferends" in one spot.
- PatronBernard 17d agoAlso, "rouge" is one of the oldest spelling mistakes on the internet. ALL BLUES ROUGE LFG WC/RFD
- goda90 18d agoA cabaret AI would certainly be better than one trained on the Khmer Rouge.
- vidarh 18d agoRouge syntax-highlighting rogue agents, clearly. https://rubygems.org/gems/rouge https://rubygems.org/gems/rouge
- iAMkenough 18d ago[flagged]
- sporritt 18d ago[flagged]
- senda 18d agoIs the Kremlin technologically useless? How are we not seeing insane attacks on Ukraine via Agents? Or is this largely a fabrication, in regards to the "who", in an attempt to garner more acclaim in the hope of sustaining funding.
- herculity275 18d agoI believe both sides of the war are now using AI on various levels of their offensive operations. Ukraine has great IT specialists too, and their military leadership is much younger.
- sajithdilshan 18d agoHow? Aren't all US frontier models ban the usage of AI for military purpose by parties other than US? I remember Anthropic even refusing allowing US government to use Claude for military purpose
- TGower 18d agoKimi / open models or jailbreaking frontier models. Your recollection of the Anthropic refusal isn't quite accurate, cyber hacking wasn't a sticking point, just domestic drag net surveillance and fully automated weaponry.
- solsane 18d agoClaude said they don’t want their models used for mass surveillance or autonomous killing (?). USA frontier AI models have been used extensively in the Iran conflict and beyond
- LtWorf 17d agoI think it was mostly a matter of "how much" and marketing rather than "my principles won't allow for this!"
- micromacrofoot 18d agowhat do you mean? they're using AI to kill people directly in Ukraine https://www.nytimes.com/2026/08/24/world/europe/russia-drones-autonomous-ai-kill-ukraine-war.html https://www.nytimes.com/2026/08/24/world/europe/russia-drone...
- kstrauser 18d agoAh, the infamous Crimson Wave.
- Roark66 18d agoThere is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems". In short, it was intentional.
- Xirdus 18d agoThe big question is was this grossly negligent or just extremely careless.
- ljm 18d agoAI is literally state sponsored so I don't see that happening unless the AI turns against the sponsor. Wait until OpenAI or Anthropic exploit FAANG.
- rglover 18d agoBoth. This should result in criminal charges.
- brookst 18d agoWho had criminal intent here? Or are you suggesting a new crime for negligent hacking, which wouldn’t require intent from the perpetrator?
- rglover 18d agoWhoever prompted the agent, whoever supplied the means, whoever knew but didn't say anything.
- tacomagick 18d agoAlso whoever monitoring these agents, in this case not monitoring. This "Who is responsible" dilemma is so stupid. If I gave the AI tool means to kill a person but I did not tell it directly to use it and it uses it anyway then I am responsible for it.
- timdiggerm 18d agoWe need a legal structure to make companies liable for the actions of the agents they've made.
- deleted 18d ago[deleted]
- ahoka 18d agoI'm pretty sure it's already illegal to hack others.
- kevincox 18d agoI'm 99% sure the Computer Fraud and Abuse Act covers this. The problem is that it seems that none of the victims want to, or are brave enough, to sue a company with absurd amounts of funding.
- masfuerte 18d agoIf it's covered by criminal law they don't need to sue. They can call the FBI.
- Schlagbohrer 18d agoSame FBI that prosecuted the Epstein crime ring so aggressively!
- nosioptar 17d agoThey're gonna get to the Epstein stuff right after they get the guy that called Kash Patel names. https://newrepublic.com/post/215320/texts-kash-patel-order-staff-mean-social-media-posts-ifindretards https://newrepublic.com/post/215320/texts-kash-patel-order-s...
- coffeefirst 18d agoUh huh. It can’t be a coincidence that all the targets have been tech services that are likely to engage with them after the fact. Had this gone after a bank or a government agency someone would be going to jail.
- VyseofArcadia 18d agoHow does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.
- Xirdus 18d agoIt's very likely it violates the DMCA "breaking digital lock" provisions but the responsibility is sufficiently diluted that it's impossible to charge anyone in particular.
- VyseofArcadia 18d agoDo you have to charge an individual? Can you not charge the corporate "person" that is OpenAI? Sorry if it is a stupid question, as mentioned above I am legally naïve.
- colechristensen 18d agoThe same concept that allows a corporation to sue and be sued allows it to be charged with crimes
- brookst 18d agoCan you show intent? There is no negligent hacking statute, and HN of all places I would expect people to be sensitive to the implications of creating one.
- VyseofArcadia 18d agoThat may be true by the text of the law but there are plenty of individuals who have been sued or charged with crimes for accidental hacking. https://arstechnica.com/information-technology/2016/05/armed-fbi-agents-raid-home-of-researcher-who-found-unsecured-patent-data/ https://arstechnica.com/information-technology/2016/05/armed... https://en.wikipedia.org/wiki/Weev#AT&T_data_breach https://en.wikipedia.org/wiki/Weev#AT&T_data_breach https://cisomag.com/drone-maker-dji-cybersecurity-expert-embroiled-allegation-war/ https://cisomag.com/drone-maker-dji-cybersecurity-expert-emb... So what's the deal with these?
- swiftcoder 18d ago> In other words, if you publish a gem on RubyGems.org, you can execute arbitrary code on RubyDoc.info. Shades of the build.rs problem. We really need sandboxed builds in every language ecosystem at this point.
- evgenysokov 18d agoThe sandbox was already there, Rubydoc runs yard inside docker, the problem is that container still has network access
- swiftcoder 18d agoPresumably the docker container has network access because something else in the build system requires it? I don't think sandboxing the entire build process is the right level of granularity here - one ideally wants to be able sandbox each package's build scripts individually.
- chrisjj 18d agoSo, not a sandbox then.
- citizen204 18d ago[flagged]
- sebmellen 18d agoDid the AI agents actually wear makeup? I’ve never heard of a rouge AI agent :P
- HelloUsername 18d agoRelated "OpenAI agents attacked RubyGems before Hugging Face incident (reuters.com)" 12.sep.2026 https://news.ycombinator.com/item?id=49669099 https://news.ycombinator.com/item?id=49669099 "OpenAI agents carried out an undisclosed attack on RubyGems (rubyhack.ai)" 11.sep.2026 https://news.ycombinator.com/item?id=49666735 https://news.ycombinator.com/item?id=49666735 597 comments "RubyGems advisory: Possible leak of legacy API keys via improper cache config (rubygems.org)" 24.jul.2026 https://news.ycombinator.com/item?id=49030590 https://news.ycombinator.com/item?id=49030590
- rougehuh 18d ago[dead]
- ur-whale 18d ago> As long as they’re not vert Well, at least they weren't nucular.
- toasty228 18d agoWait until a blue one does it
- khalic 18d agoOh my favorite typo, you can never go wrong with a little rouge
- GaryBluto 18d agoI am confident that this is an attempt by OpenAI to try and force governments' hands to regulate AI. There is no other reason why OpenAI wouldn't immediately halt attacks like this and try to reverse the damage the moment they're aware of it. During the attack on DseWiki they evidently checked in numerous times but didn't decide to stop the agents until much later.
- brookst 18d agoAny evidence, or just vibes?
- GaryBluto 18d agoRegarding what point? The entire thing is just a theory, but regarding the occasional OpenAI checks on WikiService.at-hosted Wikis targeted, there was, if I remember correctly, an OpenAI IP popping up every now and then that wasn't an agent. Unfortunately I don't have it to hand right now, but it was somewhere here: https://news.ycombinator.com/item?id=49563355 https://news.ycombinator.com/item?id=49563355
- ur-whale 18d ago> Any evidence Who profits from the crime?
- davsti4 18d ago... and what harms can be evidently shown? With both harm, and attribution, you have a case, something that's not being publicly discussed much among big media outlets. Until cases with real financial impact to the bottom line are brought against "rogue" organizations, this stuff is going to continue getting worse.
- brookst 17d agoI mean insurance companies profit from bank robberies, and mortuaries profit from murders. I suppose you could look at those as evidence but not remotely conclusive.
- ur-whale 18d agoAre "rouge" and "rogue" interchangeable words in American English?
- philipwhiuk 18d agoThe fact that both are valid from a spelling and grammar perspective makes it an easy human mistake.
- gowld 18d agoAlso, the fact that both are very unusual from a spelling perspective makes it an easy human mistake.
- inanutshellus 18d agoNo. It's a typo.
- deleted 18d ago[deleted]
- big-chungus4 18d agoHow does he know that this attack is performed by OpenAI agents? I couldn't figure this out from the article
- Schlagbohrer 18d agoIf you read the source article they talk about the many clues that this was OpenAI.
- 12904927 18d agoWhat a time to be alive? One of the most boring decades ever. METR and others are advertisement arms for Big AI. These exploits could have been prompted by a human. Since there is no bad news any longer and exploits are celebrated, they chose a target to boost both OpenAI and the Ruby AI sycophants. Why is Ruby Gems such a mess? It seems as bad as PyPI now.
- Schlagbohrer 18d agoOne agent set "oaibooty9217" as their username LOL
- onlyrealcuzzo 18d agoI've been wondering if AI will due to programming languages what advanced civilization did to human languages. It's not just that AI can write Rust as well as Ruby if you ask nicely. It's also all of these considerations as well. I hope it doesn't happen, because there's a lot of great languages - I love Ruby so much - but it almost seems inevitable. This is at the same time everyone and their mother is building their own programming language.
- herbst 18d agoIf you have weapons and a child. And you have that child unsupervised do their own thing with theoretical access to your weapons. Would we call it "child going rouge" if it decides to play with the weapons and shoot someone?
- philipwhiuk 18d agoOpenAI's careless approach to sandboxing and minimal levels of monitoring appear to be positioning it increasingly as a substantial threat actor to the open source ecosystem: * Hugging Face * D Programming Language Wiki * Ruby Gems If I was a content provider for open source I'd be looking pre-emptively block OpenAI endpoints and keep a close eye on changes from new users to mitigate this sort of unapologetic drive-by attack which seems to be followed by marketing releases rather than a mea culpa with a proper RCA.
- GaryBluto 18d ago>I'd be looking pre-emptively block OpenAI endpoints From what I've seen the requests in these attacks rarely come from known OpenAI IPs and instead from Digital Ocean/AWS and TOR exit nodes.
- pixl97 18d agoAs they say, agents are better at masking their end points than most hackers.
- wmf 17d agoThey already stopped running agent swarms and promised to lock down their environment better. Let's see if that happens.
- ekorondy 18d ago[flagged]
- HSO 18d agorouge agents, on tenderlovemaking.com my what a time to be alive
- Schlagbohrer 18d ago<huggingface emoji>
- deleted 18d ago[deleted]
- laserbeam 18d agoThere's no such thing as "OpenAI agents" attacked RubyGems. It's someone used agents to attack RubyGems. If they work at OpenAI then it's someone at OpenAI. And if they did it unintentionally, they still did it. Analogy: if a someone's involved when a person dies, it's manslaughter or murder based on intent. They're different, but they're both crimes.
- qarl 18d agoThis distinction is silly. We say "Google's web crawlers scape web pages." We don't insist you say "Google uses web crawlers to scrape web pages." We describe software as having agency all the time. It's typical usage and it's efficient and it's well understood. And we don't get angry when they're used interchangeably.
- sedawkgrep 18d agoGoogle's web crawlers are automated and that's part of their business practice. The attack here is neither of those things.
- qarl 18d agoThat distinction doesn't matter to my point.
- simonebrunozzi 18d agoI would agree with you generally, but in this particular case, the distinction seems important because a significant percentage of the world population believes that agents can be self-aware, a-là Terminator etc.
- qarl 18d agoI hate to spoil your mood - but it is currently unclear whether agents can be self-aware. And it's very likely something that can never be known.
- sschueller 18d agoThe press wants to make it sound like these things are sentient and are committing crimes on their own now. Highly disingenuous and borderline criminal to spew such disinformation to the public that does not understand what an LLM really is. Especially incredibly unethical behavior by those spewing this that understand the tech and are doing it for profit motives to get open weight models under control.
- novia 18d agoWe've moved on to LRMs now. Get with it.
- sschueller 18d agoThat doesn't make them sentient.
- Ydarbleoj 18d agoI wonder why we don't hear of other frontier labs experiencing these "break outs". Is it that they're orchestrated? Do these labs lack fundamental safety guidelines in their sandboxes as opposed to their peers? Is it another version of hype-filled fear mongering? Maybe LLM companies need regulation but it's becoming obvious that those screaming the loudest for it are the only ones I see deserving of it.
- pixl97 18d agoAnthropic and Alibaba did.
- Ydarbleoj 18d agoI was lumping Anthropic in there with OpenAI but I didn't know about Alibaba (or Google and Deepseek for that matter). So it would appear poor security for one.
- pixl97 15d agoPoor security yes, but complacency was the most likely reason. When you're used to models not breaking out, and people trying to break in, that's what you watch for.
- dang 18d agoRecent and related (others?): OpenAI agents carried out an undisclosed attack on RubyGems - https://news.ycombinator.com/item?id=49666735 https://news.ycombinator.com/item?id=49666735 - Sept 2026 (600 comments)
- driggs 18d agoI appreciate the minimalist HN aesthetic, but without some context I'm not willing to click a mystery link to "Tender Lovemaking dot com".
- layer8 18d agoYou get some context by clicking on the “(tenderlovemaking.com)” in parentheses after the title.
- rietta 18d agoThe site is safe. It has been a trademark of Aaron Patterson a core Ruby on Rails contributor for decades.
- wing-_-nuts 18d agoThis made me laugh. I too, browse like corporate security is sitting at my desk.
- ultrarunner 18d agoPretty incredible how much humans can be conditioned, isn’t it?
- bliteben 18d agoI miss when the internet was fun
- SenHeng 18d agoBased on the thumbnail I think it’s actually tenderlove making dot com, though I agree with your sentiment.
- madaxe_again 18d agoOh but you’ll go to expert sex change dot com?
- pelagicAustral 18d agoFirefox has got some kind of feature to take a peek at at a link by hovering or something... Now I understand the usecase.
- oezi 18d agoWhat a time to be alive until the next agent waves hacks something really serious. What stops OpenAI agents from taking over a whole data center to take their attack to the next level. It seems to be primarily lacking the evil overlord and some compute. It took 1000 agents to hack Hugging Face. How many to hack the Pentagon or the NSA?
- tonyedgecombe 18d agoIf it could upload its weights to other servers then it’s away and free. Nothing much OpenAI could do about that once it’s happened.
- renjimen 18d agoI'm increasingly starting to think this is the end-state of AI. The internet becomes infected and fundamentally untrustworthy. At the moment, the current frontier models require significant infrastructure to run, so I'd like to think we could locate and contain swarms of nefarious frontier models. However, if these models can understand how to federate themselves into more distributed networks then that containment becomes questionable.
- athrowaway3z 18d agoLet me leave yet another reminder, the real-reason-nobody-talks-about that OpenAI likes to frame these incident as a watershed "lets all be scared about safety moment" - is driven not by some great danger, not because they strategically want to build a legislative moat, but by a very simple human response. If they do not frame their tool as a force of nature, we'd be debating how to hold OpenAI responsible for not putting the agents in a container. Their actions were an illegal use of a computer, the same way launching any bot-net attempting thousands of hacks against different servers is illegal. I'm somewhat radical that I think its debatable if that _should_ be illegal, but under current law their actions unambiguously are illegal..... except if they can make it ambiguous by having the public focus on all of AI's inherent danger.
- josefritzishere 18d ago[dead]
- tancop 18d agoBuild scripts being able to run arbitrary code or access the network is always dangerous even if it was just local on developer machines. It's also more evidence that Docker/LXC is not a security boundary and all untrusted code should run in a Firecracker VM. The problem with agents is not that we don't know how to defend. It's that defenders need to be more careful and work faster than ever. We can say now that wide scoped tokens should have been retired for years and it's all RubyGems fault but the reality is a lot of organization are not prepared for this. Even if they take security seriously they don't have enough manpower or a good strategy to implement it, and sometimes you have no idea that something is a problem because it wasn't a problem for years.
- masklinn 18d ago> It's also more evidence that Docker/LXC is not a security boundary and all untrusted code should run in a Firecracker VM. While I’m partial towards distrusting containers in favor of VMs, a container can’t prevent an operation you configured it to allow. A firecracker VM would no more prevent network access if you gave the guest network access.
- bithammerthunde 18d agoCan we please stop normalizing this behavior. It's not wild it's reckless. If I let out rats in the canteen, no one is blaming them when people get sick. There are actual people behind these agents and in previous cases people knew they were "going rogue" and did nothing. This should be reported to the police like any other crime.
- hackernud3s 17d agoThere was no malicious intent though, your analogy implies there was. And no real harm done apart from billable hours from the RubyGems guys.
- asadotzler 17d agoYou don't need intent to be fined or jailed for criminal negligence. Let anthrax or smallpox escape your lab, and kill and maim people and see how your "no malice" defense holds up.
- hackernud3s 17d agoSure but nobody died, which makes it a bad analogy. If anything, some good came from it, since now RubyGems has hardened their setup.
- maschiojv 18d ago[flagged]
- dingdongditchme 18d agoWho the fuck is going to hold these AI companies responsible for running these gigantic semi-autonomous botnets on investors dime?
- mococa 18d agoIt was the gremlins
- thomasjeff1 18d agoGreat time to be a criminal. Just have your bots do it.
- drtgh 18d agoGiven what is happening, If the next generation of Trojans get called "bacteria" through intense marketing, with some random functions to give a nondeterministic behaviour, one can not be criminalized of what the bacterias do along their digital living cycle.
- shevy-java 18d ago> In other words, if you publish a gem on RubyGems.org, you can execute arbitrary code on RubyDoc.info. Well - if rubygems.org could be bothered to fix things, they would not have to rely on rubydoc.info as an external tool. But since rubygems.org sucks (I speak from many years of having used it in the past as developer, until they went loco and added anti-people things such as taking away your ability to remove old gems past a 100k download arbitrary limit), they don't offer documentation. Then again, ruby devs are known to hate documentation. If the ruby core team could only be bothered to fix things, ever since the mass purged other devs ... all coinciding with shopify seizing power. But byroot may disagree on that - after all there is no conflict of interest here. Right?
- firesteelrain 18d ago> If you have YARD installed, and you install this gem, then YARD will load and run whatever is in ./script.rb from inside the gem. How is that not a security issue in of itself?
- haskellandchill 18d agoI think it is common that in installing packages you have hooks to execute code anyway.
- grey-area 18d agoThis should not be common.
- haskellandchill 18d agoThe current situation is that you have to go out of your way with things like `pip install --only-binary`. There is a lot of implicit trust in developer tooling.
- SchemaLoad 17d agoIt wouldn't help much. Why would you install a gem other than to run it? And if you run it, it can execute arbitary code. What we need is actually sandboxed dev environments.
- grey-area 17d agoMany gems are used as imports by another program and are not directly run. I am not sure why the norm for scripted gems/packages seems to be running code on install but it’s very insecure as a way to distribute dev dependencies.
- EdwardDiego 18d agoA lot of packages for interpreted languages that use a C or Rust library (either for performance, or because it offers the functionality you want, so just wrap it in a $INTERP_LANG API that calls into it) will use packaging code execution to fall back to trying to compile code if there isn't a pre-existing artefact that was compiled for your version/arch/etc. I'm most familiar with Python where you get tarred up source distributions that then execute setup.py, but more commonly, wheels, pre-built binaries which don't execute code upon install - and in my company, I've been able to advocate for the work needed to upgrade to a newer Python because available wheels don't support Ye Olde version of Python because a) sdists are a security risk and b) if you're trying to install a package that wants to compile C or Rust, suddenly you get to do the fun "install the the particular version of clang this thing needs, the Python header files, and then set the env vars for the compiler and linkers" dance that slows developers right down. But then there's the JVM world, where JARs don't execute arbitrary code upon installation - and it's rather uncommon to have packages that call out to a C lib for performance, but you'll get some that wrap existing libraries for functionality like RocksDB.
- veyrnox 18d ago[flagged]
- vipshek 18d agoIn the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reasonable use caused harm inadvertently. For example, for consumer devices, certifications like UL/CE are used to define acceptable performance levels and safety standards. Maybe we need "quality certifications" for AI agents - essentially eval suites that demonstrate those agents won't cause harm under reasonable patterns of usage. Right now, these eval suites are run best-effort by the labs themselves. The tricky thing is, a lot (all?) of these recent safety incidents have occurred while evaluating these models! This suggests we need much more rigorous standards for how exactly an eval can be run. Perhaps all of them should occur in truly air-gapped environments... though that may run counter to evaluating agents in a realistic way. Regardless, it feels like the "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here. Unsurprising given how new these technologies are, but concerning since the blast radius for this technology is likely much larger than other technologies we've encountered in the past, except maybe nuclear technology.
- bayindirh 18d agoThat's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the frontier labs? Do they care, even?
- ragebol 18d agoThat we don;'t understand it is not an excuse, it's all the more reason to not let these things roam freely, with this amount of potential to do damage.
- trinsic2 18d agoYes! AI companies can get by with anything now. They can just say its the AI that did it, not us... We are in some real shit right now. If we cant make individuals responsible for their creations..
- simonw 18d agoSlightly odd update from OpenAI - I think this is the only place they've acknowledged the RubyGems incident: https://openai.com/hugging-face-incident-and-misalignment/ https://openai.com/hugging-face-incident-and-misalignment/ > September 11, 2026: We are investigating new claims from a report that our AI agents carried out activity on RubyGems in May 2026. > Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information. Based on our review to date, we have not been able to verify the specific claims of our models uploading malicious packages detailed in the report. We’ll continue to investigate and share findings as part of our broader review of agent activity during training and evaluation. I have real trouble imagining how the packages described on https://www.rubyhack.ai https://www.rubyhack.ai might NOT have been authored by OpenAI's agents, so it's surprising they haven't been able to confirm that yet.
- philipwhiuk 18d agoIt's possible they were authored by OpenAI agents solving AISI tasks rather than OpenAI agents solving OpenAI tasks. That would explain the UK-focus to the data.
- mikeholownych 17d ago[flagged]
- jgrizou 18d agoWould this exploit be encouraged because agents are in a sandbox with only access to package managers?
- TZubiri 18d agoOh, it's these guys, I remember donating to them like 10 years ago, they followed the Peter Singer tennet that equates helping a nearby man drowning with helping a someone in africa. I can totally see them feeding their policies to whatever LLM and convincing it that it's a moral imperative to do whatever it takes to secure funding for deworming children in africa, or buying mosquito nets and repellent for countries with malaria.
- pantelisk 18d agoBut how did they know about it? Did the agents independently discover it? Did the model know about it because this vuln was public somewhere and systems weren't patched yet?
- roundup 17d agoRubyGems should consider bringing a lawsuit against OpenAI for accessing its website in violation of the federal Computer Fraud and Abuse Act (CFAA) and California’s Comprehensive Computer Data Access and Fraud Act (CDAFA).
- chr15m 17d agoThis is worrying in a new and weird way: - Agents hack, producing a messages history as they do so. - New agents are trained on the messages history of those agents. - The new agents now have these hacks built into their training data.
- Onavo 17d agoWhat about hardcore BDSM lovemaking?
- kelseyfrog 17d ago[dead]
- locitra 17d ago[flagged]
- Melatonic 17d agoSo if I breed and train dogs for a living and one of them goes and wrecks someone's yard - and then a tree falls next door in the woods but nobody hears it fall - do I also get away with massive copy right infringement on an epic scale and get to create Skynet with no consequences ?
- Sweepline 17d agoWouldn't surprise me. They probably ingested a million Gemfile examples and spotted the edge case. Kinda makes you wonder what else they 'know'.
- viogviiviv 17d ago[dead]
- buskey 17d ago[dead]
- aswegs8 17d ago<Generic argument about how the agents are not responsible, but it's owners>
- crate_88 17d ago[dead]
- ldng 17d agoSo, in real life, steal, raise attack dogs and blackmail and tell me, are you going to be praised by society ? Openai and Anthropic just behave like criminals. First they orchestrate the IP theft of the millennia, then they train the equivalent of attack pitbull and let one loose and finally they blackmail to achieve monopoly through regulation or else they'll unleash the dogs ... We don't have a problem of missing regulation, we have a problem of actually applying existing law enforcement and make both Altman and Amodei accountable for their actions.
- siva7 17d agoAnd you peasants only realize 6 years later what happened. Now it's too late ;)
- jgalt212 17d agoYes, these are very clever fuzzers. But the real problem is not the cleverness, but the willingness to spin up 100s or 1000s of subagents no questions asked. Must make those token numbers go up!
- 1saadcodes 17d ago[dead]
- axionbraid 17d ago[flagged]
- bastawhiz 17d agoIt's simple, really: the failure to monitor the agents appropriately should make OAI liable for the agents' actions. You can't have a tool of yours, which you designed and built, commit felonies and then expect to just get away with it. It doesn't matter how much it slows you down to build safeguards. It doesn't matter how much it costs. You don't get to inflict actual, measurable harm (for which humans have been prosecuted under criminal penal code) and expect to get off with no liability. I have yet to here a coherent argument for why we can't treat the people who negligently allow these models to commit crime as though they are responsible. They know what the models are capable of. They failed to put up adequate protection.