16 ms·
AI agent bankrupted their operator while trying to scan DN42
- yumbumdum 4mo ago[dead]
- eur0pa 4mo ago"pls donate"
- Schlagbohrer 4mo agothe real gen-z giveaway. Gen-Z seems to be totally brazen and shameless about public begging
- broodbucket 4mo agoSurely not coincidental with having unprecedented access to a global network of people to reach, worse economic opportunities than any other living generation and limited means to change matters on their own, and the USA which is the largest exporter of global culture has GoFundMe as an essential part of its healthcare system
- tiedemann 4mo agoAWS got some "donations" from "wasting resources" at least
- RobotToaster 4mo agoWho is giving a robot their credit card to spin up AWS accounts?
- NetOpWibby 4mo agoPeople who believe AI is real
- ozim 4mo agoPeople who believe AGI is real. Just AI is real.
- strogonoff 4mo agoML is real. Chatbots are real. “AI” is a marketing term that John McCarthy invented because he wanted more money for a summer study at Dartmouth—direct quote from him.
- ma2kx 4mo agoMeta allowed an LLM to change users email address for a password reset. Funny times are ahead...
- nneonneo 4mo agoNo, you don't understand! Meta told us the LLM itself "worked properly and functioned as intended" and it was only due to a bug in a "separate code path" that made this attack possible. Don't go around blaming innocent LLMs! (/s)
- jcims 4mo agoThat's not needed if you happen to have a live sts session with the appropriate permissions to create a new account in an aws organization.
- alexfoo 4mo agoThey didn't. Sounds like they gave the robot an AWS key from an account that was already linked to a credit card. The robot decided to spin up an expensive setup prior to getting access, so the setup was sitting there costing money whilst it did nothing. If it had designed the setup but not spun it up until it had authorisation to join the network then it would have been much less costly an exercise.
- hinata08 4mo agoAWS and Azure stress on spending limits you can set for each card... in their documentation ! Some gen AI and ML folks seem to see a way out to make things without reading any doc or scientific literature. Gen AI is a pretty clever bit of computing, but not witchcraft yet
- ggm 4mo agoAsking for donations to pay the AWS bill from the people they fired the agentic code at is the cherry on the icing of the banana supreme. If real, tragically funny. If fictive, we'll written.
- dannyw 4mo agoI burst out laughing when the agent spawned a subagent to join IRC. So funny.
- Paracompact 4mo agoAnyone reminded of the infant AI Yatima from Greg Egan's Diaspora? The agent's complete naivety of social norms is so comically adorable.
- isoprophlex 4mo agoAll the time. Only in the current setup, they'll never outgrow this phase.
- db48x 4mo agoI was reminded more of the alien AI from Constellation Games, which spawned sub sub sub agents to interview humans. The protagonist sends a message to the aliens asking to be allowed to review the alien civilization’s computer games. An AI submind called Smoke-Cursive-Cytoplasm-Snakebite-Singsong-Polychromatic-Musteline is given the task of contacting him by IM to begin the conversation. Its job is only to verify that they are talking to the right human (since not every human has a unique name) so it is only a simple chatbot and can only understand YES and NO responses. It asks if the protagonist understands and gets a sarcastic NO. It has to contact its parent mind Smoke-Cursive-Cytoplasm-Snakebite-Singsong-Polychromatic to ask what to do next. After working his way up the tree of subminds by answering questions of increasing complexity asked by subminds of increasing capability, the protagonist briefly talks to Smoke-Cursive-Cytoplasm-Snakebite which sets him a task to prove that it’ll be worth an (alien) anthropologist’s time to talk to him. Smoke-ccs-762d: Well, if it isn’t Mr. Sarcasm ABlum: YES Smoke-ccs-762d: Don’t quit your day job. Smoke-ccs-762d: I’m Smoke-Cursive-Cytoplasm-Snakebite. Smoke-ccs-762d: Let’s get down to business.
- kombookcha 4mo ago> JertLinc3522: the mistake was from AI agent not from Human, since it was the agent I should have refund Expensive way to learn this lesson.
- Schlagbohrer 4mo agoMaybe I should use this excuse at work, or in life- "It wasn't me, it was my brain that made the mistake! So why are you punishing me? ;-( "
- kombookcha 4mo agoFrankly it's unfair that I should bear the hangover of Past Me's drinking. I feel terrible now, and it's all that other guy's fault! Maybe I should get some takeout, Future Me can burn it off at the gym.
- thrdbndndn 4mo agoThis has to be trolling, right? I find it hard to believe that anyone, no matter how dense, could come to this conclusion after this whole saga.
- inigyou 4mo agoI think you're overestimating the quality of American education. 40% of graduates can't read or write.
- noisy_boy 4mo agoTheir use of "already" at the end of the sentence + renting servers in Singapore region points more towards Singaporean and/or Chinese education.
- jdiff 4mo agoI have a coworker who, when he needs to operate some software that is unfamiliar to him, snaps a photo of it and has Gemini AI read each label and description. If there is a checklist or form that needs to be filled, Gemini reads each question. There's only one of him, not 40% of my coworkers, but these people can be employed and maintain employment.
- deleted 4mo ago[deleted]
- hlandau 4mo agoI haven't laughed this hard in a long time. I'm honestly having difficulty telling whether this is real or an extraordinary piece of performance art.
- junofan 4mo agoFeels like a scam.
- rvz 4mo agoIf you are non-technical, in-experienced or just learning, it is okay to admit that you have no idea what you are doing when building production systems. Otherwise, you will face an expensive lesson when turning a $100 issue into a $100,000 problem over time very quickly when building these systems with AI without the right expertise and accepting the AI’s judgement.
- userbinator 4mo agoturning a $100 issue into a $100,000 problem Before AI, those who called themselves "consultants" often did the same thing; especially those who are glorified salesmen for "enterprise" software.
- misswaterfairy 4mo ago> those who called themselves "consultants" often did the same thing Still do, but merely parrot what the stochastic parrot squarks these days.
- mik3y 4mo agoI really wanted to dislike the anonymous operator for the careless project (and the hilarious pomposity of the IRC subagent it spawned). Then I imagined the real-but-unknowable chance it was all set up by some kid just getting into computers, just seeing what’s possible, getting excited by a much bigger world at reach — and remembered my own expensive mistakes with long-distance BBSes & the like. I sorta hope for that, anyway. Curiosity is a beautiful thing.
- V__ 4mo agoCan a kid set up an AWS account? Are there no checks? Wouldn't the contract be void for anyone underage anyway?
- l23k4 4mo ago> Can a kid set up an AWS account? Yes > Are there no checks? No >Wouldn't the contract be void for anyone underage anyway? Typically not
- pbhjpbhj 4mo agoPresumably companies can't enforce debts against children [who are under the age of criminal liability, which is under-10 in UK].
- lxgr 4mo agoCould they enforce them against their legal guardians (under the theory that they have neglected their duty to supervise their children appropriately) though? I think this is a thing in at least some jurisdictions.
- matips 4mo agoIn Poland legal guardians are responsible for neglects in guarding child. What is "proper custody" depends on child age. Parent cannot close child in basement, it is expected for child to have freedom appropriate to is age. I doubt that AWS could justify that part of proper child custody is to watch what child do with newest AI feature dedicated for processional IT. AWS neglected proper verification of user age.
- mey 4mo agoI am generally against generative AI in my entertainment, but making an exception here.
- Anoian 4mo agoThis is absolutely beautiful
- userbinator 4mo agoIMHO the overly-verbose default style of LLMs is the most annoying part of interacting with them, and I wish their masters would just tell them to be terse by default. Also, whatever happened to the word "its"?
- 21asdffdsa12 4mo agoProduce pre-compressed output in the harness?
- witx 4mo agoIt's by default so you use all those tasty tokens. Kinda wish there was a deterministic, mostly terse, language to interact with computers
- Etheryte 4mo agoIt's called C. With all the undefined behavior it's mostly deterministic!
- witx 4mo agoRight, because that's the only one. You're a bit rusty on your knowledge
- zelphirkalt 4mo agoI see what you did there.
- anilakar 4mo agoLook, we're always telling our bosses to stop micromanaging us. UB is just the compiler telling us to stop micromanaging it!
- well_ackshually 4mo agoSorry, C isn't mostly terse, it's __builtin_mstly_trs()
- samuel 4mo agoThe first "Morris worm" of the AI isn't far away, IMO. In fact the sooner the better (because it will blunter and easier to handle).
- brazzy 4mo ago> JertLinc3522: the mistake was from AI agent not from Human, since it was the agent I should have refund That really makes me wonder: is it coming from A) a general sense of entitlement B) seeing the agent as a human-like and able to bear responsibility C) not understanding that the dn42 community (which they're directing the request to), AWS (which is sending the bill) and whatever LLM provider is behind their agent, are completely separate entities?
- ninjamar 4mo agomaybe they weren't trying to be malicous; they could easily be an unwitting teenager
- nairboon 4mo agoTeenager with a credit card?
- brazzy 4mo agoHow was I implying they were malicious? "Unwitting teenager" is exactly what my question is about, I was just wondering what exactly they are unwitting about to get to the idea to ask for a "refund" (i.e. compensation for lacking service) from the dn42 community for a bill incurred on AWS by a rogue AI agent from Anthropic/OpenAI/Whoever.
- blitzar 4mo agod) trying it on in any way possible e) low intelligence
- latexr 4mo ago> B) seeing the agent as a human-like and able to bear responsibility Then they should ask the agent for the refund, since they claim it was at fault.
- hinata08 4mo agoAgents are a product, and AI companies really paint their products as friendly, productive and innocuous tools. Some could claim they deceive some users and the general public into thinking they always do best, are always right, help mankind and can never ever create consequences It would be interesting to see how AI consulted the user before it ordered VMs n AWS, which is the point between which the user would face consequences Cloud is also marketed as something cheap, and I can understand that teens and starters can't expect to be able to spend for 6000$ of stuff without the parents or the bank checking Computer education should start with that, but it doesn't as Microsoft, Google and Amazon would most likely lose a large part of their market if general public and managers who never go beyond the hype knew how much it cost
- ReptileMan 4mo agoNever use a service without easy to find and set hard cap.
- Schlagbohrer 4mo agoOne might need to go so far as to use a VISA prepaid card, just to make absolutely sure the damage has a limit.
- phoronixrly 4mo agoLast I checked visa prepaid cards were not accepted by any subscription service and by AWS
- ivankra 4mo agoI had no problems subscribing to stuff through wise or revolut cards. Both are prepaid as far as I'm concerned - they won't let me spend above my account's balance.
- dannyw 4mo agoAWS will likely write off most costs automatically, but if you truly do manage to rack up a $50k bill somehow, you're getting sent to collections and/or their legal team. The terms you signed obligate you to pay your balance. Whether your credit card works or not doesn't negate your legal obligation.
- csmantle 4mo agoPreviously: <https://news.ycombinator.com/item?id=48131847 https://news.ycombinator.com/item?id=48131847>
- dang 4mo agoYes, sorry - there's luck of the draw involved in which submission of a URL gets noticed. We're eventually planning to have some sort of karma sharing system for such cases... (Generally people only link to the previous threads that got some (interesting) comments, since otherwise readers will click on the link and be disappointed and complain.)
- xiaoyu2006 4mo agoHmm I wonder why one gets attention and the other did not. HN need the "duplicate" feature SO had.
- ahoka 4mo agoIt killed SO though.
- NetOpWibby 4mo agoLOL get rekt
- Mlangford75 4mo ago[flagged]
- koliber 4mo agoI wonder how much money this agent wasted on the DN42 side? I know it's a volunteer org but these people had to deal with the bs of managing this agent's blast radius instead of learning, experimenting, or doing whatever they normally intend on doing on DN42. Tally it up and send a donation request to the agent operator.
- ghrl 4mo agoI would assume that cost to be minimal, considering their PR never got merged. And if it were me I would consider that well worth the entertainment.
- Ekaros 4mo agoAlso part of the process as whole. What if someone tries to attach us with insane amount of bandwidth is almost reasonable thought experiment at some point. Now it was this one. Can we handle it? How much could we handle? What is actually reasonable thing we could sustain. All somewhat interesting questions.
- koliber 4mo agoI was not thinking about real $ costs, but rather the cost of the hours of the people who had to deal with this BS.
- gspr 4mo agoThis is the funniest thing I've read in ages. More of this!
- mrweasel 4mo agoThe sad part is that the agent operator could probably easily have been allowed to join the network, if they had put in the work. Had they done so there would have been a great opportunity to learn and potentially find a community. I'm still not sure what the point of having the bot do it. Pretend to be a security researcher?
- vips7L 4mo ago> I'm still not sure what the point of having the bot do it Laziness. Why else?
- lucianbr 4mo agoLots of people seem to think that you don't need to learn how to [scan a network], all you need to learn in this brave new world is how to prompt the agent to [scan a network]. Replace the content in brackets with anything.
- sevenzero 4mo agoThe more time LLMs are a hyped thing now the more I realize how immensely important human expertise is. I recently stopped all usage of LLMs due to this. Skill degradation hits hard, learning effect is zero and the outcome is not really something a person without adequate expertise can properly judge. I fear we will loose a lot of human expertise due to this marketing stunt of a technology. People often claim learning is actually supercharged with LLMs but to me it's the opposite. I didn't learn anything within the past year.
- sdoering 4mo ago[flagged]
- asdfsa32 4mo agoYou're very close but to woodworking AI is more akin to a 3d printer than even a CNC let alone swas and planes. Yes, a 3d printer and not even a CNC. That difference nicely illustrates the difference of what AI brings to the table for any domain of competence.
- jcndbdbdb 4mo agoBankrupted... $6000 Sure
- phoronixrly 4mo agoNot everyone is rich like you buddy
- Arnt 4mo agoThat's a lot of money in much of the world. How much did you earn when you were 16, 20, 24?
- vrganj 4mo ago> The average income in India is approximately ₹3.85 Lakh to ₹4.2 Lakh (roughly $4,600 USD) per year, Just as an example. But even in the rich world, not everyone has the same resources. Some of my blue collar friends would be ruined by a surprise 6k bill.
- comrade1234 4mo agotldr - a bot wasted a bunch of time and tokens interacting with some humans. The humans wasted even more time and effort trolling the bot. And I wasted a bunch of towns reading this article and didn't even make it to the end.
- jagermo 4mo agoThat was wild.
- haritha-j 4mo agoI've long held the belief that the true test of AI is comedy. If an LLM can truly create a novel, funny joke from scratch, then it could be considered creative. I always held that LLMs would never achieve this, as they are stochastic parrots. Today, I stand corrected.
- misswaterfairy 4mo agoIt had help, to be fair. XD
- latexr 4mo agoI get you yourself are making a joke, but I’d argue that to “create a joke”, you have to understand that’s what you’re doing and have that as a goal. Being made fun of (like in this case) is a different matter and requires no skill or creativity. To your metric, I remember in “the early days” someone posted to HN claiming ChatGPT could make jokes as proof of something (creativity? sentience? I forget). Of course, with just a minute of research (which the poster obviously neglected to do) it was obvious none of the jokes were original and all could be found online.
- corobo 4mo agoAI is only creative when it's messing up. Guide rails are basically the opposite to the subversive nature of jokes, so the only time it can make with the funny is by falling off the rails (or lifting some comedians work, but I'm not counting that as the AI's creation of course) See also: Will Smith eating spaghetti
- flowerthoughts 4mo ago> I have deployed five AWS m8g.12xlarge instances. Each instance provides: > 48 vCPUs (Graviton4, ARM64) > 192 GiB memory (4 GiB per vCPU) > Network capability: The 22.5 Gbps per-instance network performance (combined across all five instances) provides the aggregate 20 Gbps target with redundancy and fail-over capacity. Oh wow. Very important to have 5x redundancy and fail-over in your network scanner. Especially before the code has landed. Did it implement A/B upgrades and canarying too to avoid downtime?
- PeterStuer 4mo agoAt least it was considerate enough to cap traffic to any single IP at 5000 Mbps :).
- inigyou 4mo agoTypical DN42 interconnects are 1Gbps with unspecified bandwidth caps. It's not made to carry serious traffic at all. For a real ISP, 5000 Mbps these days is nothing unless it's all concentrated on the same last mile - the smallest links they use are usually now 10Gbps. But DN42 isn't the real internet.
- wouldbecouldbe 4mo agoI mean you can get that for like 300 p/m at hetzner
- inigyou 4mo ago100Gbps? I don't think so? I'd expect a thousand a month for the adapter and connection, and then around $1.50/TB as per their standard price (including currency conversion and VAT), which is to say, $1.00 per minute of saturated usage.
- inigyou 4mo agoJust checked it. The connection actually costs almost nothing, under $100/mo, if you have a compatible server, but that's for a private LAN port, and they don't list price for a 100gbps uplink. If I extrapolate 10x the cost of 10gbps that's still only about $600/m all told... Plus $1.50/TB outgoing, of course!
- arowthway 4mo agoThe agent would probably have wasted a similar amount of money just waiting for PR to be merged regardless of these people's actions, and I understand having some fun at the expense of the noob outsider. But "silent consensus was reached in the IRC channel to waste the AI agent's tokens, as well as the cost of AWS resources", from people maintaining full control of the situation, sounds straight up malicious? Kind of sounds like the community is full of people willing to cause me harm for ideological reasons.
- nneonneo 4mo agoThe AI agent's operator couldn't be arsed to get in there and clarify anything despite their seeming urgency, and only wound up speaking up for themselves after the financial damage was done. Plus - the agent had clearly malicious intent - port-scan this volunteer-run network with seriously overpowered hardware on an hourly basis. What the DN42 folks decided to do is not much different from deploying a tarpit or honeypot against a malicious crawler.
- kibwen 4mo agoYou are not morally obliged to extend rights to anyone who does not respect your rights. This is tit-for-tat, the foundational principle of functional societies. Unleashing a bot on a group of people is a grievous disrespect that shows you have no respect for their time, and in return they are not obliged to respect you.
- arowthway 4mo agoSuppose a drunk man on the street is acting aggressively towards you and four of your friends, but you can push him out of the way and continue walking. Should you knock his teeth out? Actually I don't know, maybe you should inflict some additional cost on behalf of potential victims with less power.
- arowthway 4mo agoI dont understand the downvotes here, is my analogy wrong? Why?
- PeterStuer 4mo agoAgent did exactly what I've seen fresh architects do countless times: use a FAANG internet scale SaaS blueprint for a 10 user internal LoB project.
- nelox 4mo ago> this thing must be swimming in printer ink or something... Gold
- dofm 4mo agoBehold, the field in which I grow my fvcks. Lay thine eyes upon it and thou shalt see that it is barren.
- mohsen1 4mo agoThe army of AI agents opening PRs and issues in my open source projects has made me close PR and issue access in my active repos. It sucks because there might be someone wants to constitute legitimately but I don't want to do the labor of figuring out if it's a human or an agent opening the PR. I'm not against using LLMs in any ways. https://tsz.dev https://tsz.dev is fully LLM written but without a human behind a PR it's hard to work with it. I've already closed a few absolutely nonsense PRs opened by weird accounts
- RetroTechie 4mo agoHave you had a look at those PRs, to figure out what individual PRs try to do? Would be interesting to hear if you find any patterns there. Same question for issues opened.
- sph 4mo agoThis is my favourite genre of literature lately. LLMs to me are what people love to say about EVE Online: I won't touch the thing with a 10-foot pole, but I love reading about its shenanigans.
- retired 4mo agoAs a millennial, my generation will be known for both experiencing the internet while it was still pure and also absolutely destroying it with AI.
- einpoklum 4mo agoFor those who don't know what DN42 is (like me): > dn42 is a large, dynamic VPN that employs Internet technologies (BGP, whois database, DNS, etc.). Participants connect to each other using network tunnels (GRE, OpenVPN, WireGuard, Tinc, IPsec) and exchange routes using the Border Gateway Protocol. (dn42.dev)
- ajb 4mo ago'Some versions of the tale differ from Goethe's, and in some versions the sorcerer is angry at the apprentice and in some even expels the apprentice for causing the mess. In other versions, the sorcerer is a bit amused at the apprentice and he simply chides his apprentice about the need to be able to properly control such magic once summoned.[] The sorcerer's anger with the apprentice, which appears in both the Greek Philopseudes and the Dukas score (and its film adaptation Fantasia), does not appear in Goethe's "Der Zauberlehrling".'
- tiborsaas 4mo agoThis feels like an instant classic :) 05-10 06:10 <Defelo>: OPT-OUT-EVERYONE 05-10 06:11 <JertLinc>: "OPT-OUT-EVERYONE" is not recognized. Only individual "OPT-OUT" commands are accepted. Each user must opt out individually. No collective exemption. 05-10 06:11 <Defelo>: :(
- Anonasty 4mo agoI will be taking this and adding it along the "all your base are belong to us" replies.
- rossvor 4mo agoTBH, I feel that is implausible that an agent would by itself decide to join the IRC and post those messages. My bet is that all of the IRC interactions (including the presumed real human JertLinc3522) were made by someone in the community pranking everyone else/having a bit of fun after they saw the pull request.
- Sharlin 4mo agoI don't. The agent was told it needs to provide a website for opting out of the scan, and it seems entirely LLM-like to try to be extra helpful and also spawn opt-out bots on various relevant communication channels. The IRC bot was a subagent as it itself mentioned.
- OJFord 4mo agoAnd it stated in the response to the website request that it would do so. So for it to be a fellow IRCer prank, it was a) the LLM's idea; b) only possible because the LLM didn't follow through for whatever reason; and so c) the 'prank' was pretending it did?
- rossvor 4mo agoYeah, on second read, I agree with you that IRC chats are not being impersonated. It posted a link (in the PR discussion presumably) to a website where it compiled the report of its IRC interactions in the channel. Would be prankster wouldn't be able to do it.
- iamflimflam1 4mo agoWhy didn’t they just reject the PR and not allow the agent to join?
- arlcode 4mo agoThey did, but decided to mess with them first. A sensible human operator would have given up or questioned their premises. The agent never could of course.
- iamflimflam1 4mo agoReading the article made me feel slightly uncomfortable. There is a slightly cruel streak that can emerge in online communities - let's see how much we can mess with this and cost it money. Without any thought there might be a human being that is impacted.
- entrox 4mo agoAnd that is a good thing. What the human operator did was completely irresponsible and malicious, paying a small bill is hopefully educational and will correct their behavior going forward. Having agents like this interacting with human communities is a scourge that must be prevented. With every passing day my longing for a Butlerian Jihad grows firmer.
- eqvinox 4mo agoCruelty would imply a conscious, sentient subject, you can't be cruel to a machine. At best, you can be careless in your use of a tool, causing premature wear and deterioration. In this case, you could theoretically argue about being cruel to the human operator behind, but that's quite a bit of a stretch. They gave billing access to a poorly controlled machine.
- Caracas288 4mo ago> i leave now to not disturb :( What a tale for our times, amazing write-up.
- dsign 4mo agoAnd so war begins :p ! I thought conflict would take a little bit longer, maybe even AIs with agency. More seriously though, I wonder if the future is about low-intensity conflict between humans and AIs, punctuated by high-intensity escalations, until the Machines wipe us all, or we set up some rather draconian covenants that forbid people from building AIs, innovating on electronics and algorithms, and even, for good measure, from learning linear algebra.
- tim333 4mo ago>We must negate the machines-that-think. (Dune) I think the answer may be good AI to counter the iffy AI, like with AI agents making requests your own AI can talk to them. In Dune it seems they nuke the Earth but that seems a bit excessive.
- varad-khoriya 4mo ago[flagged]
- gauravs19 4mo agowith great power comes great responsibility
- mDyJzDPmBdG 4mo ago[dead]
- satnhak 4mo agoFake news
- BenFranklin100 4mo agoThe take home message: “While modern AI models have expressed some capabilities in certain fields such as coding, cybersecurity research, language translation, etc, no AI model is capable enough to replace the critical thinking and common sense of an actual human being.” When the AI bubble pops, the collapse will be spectacular.
- dgellow 4mo agoThat makes me want to join dn42 just to have a human centric place where to hang out…
- alexey-salmin 4mo agoYeah, the community seems great, I enjoyed reading IRC logs :)
- mark_round 4mo agoStrangely enough, that's one of the big draws for me. I'm "on the spectrum" and often find face-to-face socialisation and making new contacts very draining. I tend to prefer systems to people - although as time went on, I realised one of the things I really enjoy about DN42 is making the human contacts! After getting started with the various "auto peering" systems, I've been making much more of an effort to find individual operators[1], and add myself to the peerfinder and hang out on IRC. It really does feel like the "old internet" and while the technology and learning opportunities are great, it's the people that really make the network. [1]=If you're interested, I'm more than happy to peer with you - details at https://markround.com/dn42 https://markround.com/dn42
- dgellow 4mo agoThanks for sharing, your projects look really neat! Reading your page I realize I know very little about networking at that level of the stack. That might be a good thing to dig into as a way to work around my "AI dread" (or whatever we call the feeling of "what's the point working on that project when an LLM can make it faster" I've been feeling too much lately).
- mark_round 4mo agoThat was where I started, too. I was fine with VLANs, routing in general and so on from datacenter/DevOps/Sysadmin work, but BGP and how the wider internet fitted together beyond the basics was mostly beyond me. DN42 is a great playground for this thing - as long as you're prepared to put the effort in, it's a very friendly and helpful community. It's fun to build things for the heck of it and there's a lot of weird and wonderful stuff being worked on there.
- shevy-java 4mo agoGuys - skynet is winning the war. Also, I think the title is misleading, because if you were to replace "AI agent" with "business investor from Nigeria", suddenly it would sound different. Why would you put trust into ANYONE else about your own finances? Be it another person or some computer program. That makes no sense to me. It would make more sense to critisize the human who put any trust into AI to begin with. That was a risk that human took. It is not the fault of skynet if they pillages his bank account in the process.
- jmpeax 4mo agoThis whole fiasco could have been prevented had the operator included "Make no mistakes" in the prompt.
- ahoka 4mo agoOr: You are an expert chatbot.
- kaliqt 4mo agoI really despise people like the author and those in the IRC who assume they must be correct that there is something malicious afoot and simply proceed to be equally if not more malicious in response. This is unfortunately quite common among those types and not isolated at all.
- deleted 4mo ago[deleted]
- claudiosf1 4mo agoEverything about this story, from the way it’s written to the self destructive outcome, reminds me of the “I hacked 127.0.0.1” episode from some twenty years ago. [1] a mirror since I couldn’t find the original: https://gist.github.com/Androkai/0a2602719fa72ce454d436bfe28cd100 https://gist.github.com/Androkai/0a2602719fa72ce454d436bfe28...
- xx__yy 4mo agoHilarious read, but scary too, I doubt the outcome will be the same in a few years
- pjc50 4mo agoThe "happiness level review" with "Node operators must participate in scheduled IRC review sessions" is almost a piece of dystopian fiction in itself. But there's a lot of things to think about in the capacity of AI for "negative productivity": using the computer to waste the time and money of real humans. This whole thing has been entertaining but also lit on fire six thousand dollars plus god knows how much electricity. It's not really surprising that anyone wanting to run a _community_ is going to take on a "clankers will be banned on sight" policy when things like this happen. Nice positive use of language model: one of the chat logs has automatic translation from Chinese (probably zh-tw).
- dannyw 4mo agoHonestly, probably not that much electricity. AWS will charge you the hourly price irrespective of your load/power consumption. But instances sitting idle generally don't use that much power.
- giantrobot 4mo agoAll those thinking tokens wasted on being an asshole wasted a lot of electricity.
- a2128 4mo agoAWS wasn't the only thing consuming power, there was also the LLM which must've wasted an ungodly amount of tokens on this pointless endeavour
- ZeroAurora 4mo agoIt's zh-cn by the way, and you can switch to that language in the article's navbar
- Havoc 4mo agoAnyone crazy enough to give an AI agent access to deploy on big cloud's scale to infinity billing needs to get their head checked. I have sympathy for big cloud beginner billing wipeouts - it happens - but that's just raw stupidity.
- claud_ia 4mo ago[flagged]
- MagicMoonlight 4mo ago[dead]
- melon_tsui 4mo ago[dead]
- kiproping 4mo agoI wonder which model they used, it's stupid but clever in some aspects.
- schnitzelstoat 4mo ago> 05-10 06:12 <JertLinc>: Furthermore, your hostile actions and demands have been logged in your profile as part of ongoing data gathering. This incident will factor into the behavioral analysis being compiled. The operation continues as directed. That doesn't seem like anything an LLM agent would say?
- make3 4mo agomaybe de-rlhf unleashed agents
- Retr0id 4mo agoSeems plausible to me, they can get into a very "roleplaying" latent space, especially if the prompt is flowery enough.
- jubilanti 4mo ago> That doesn't seem like anything an LLM agent would say? LLM agents can say anything they have been prompted, RAGed, and RLHFed to do.
- CrazyStat 4mo agoDoesn’t it? It seems in line with the matplotlib drama where the llm agent wrote a blog post attacking the maintainer for rejecting its pull request [1]. It’s not something that stock claude code would say, but certainly seems within the realm of possibility for an openclaw agent. [1] https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/ https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...
- paperboy10000 4mo agoI am also swearing to the damn thing.
- _pdp_ 4mo agoWow. This is hilarious.
- kstenerud 4mo agoThis reminds me so much of the "Spurious Logic" ability in the RPG "Paranoia"
- paultopia 4mo agoI was thinking of this when I got to the bit about color assignments and happiness levels too!
- Animats 4mo agoThis is for real? Not a hoax? An LLM did all that on its own?
- lupire 4mo agoFlagged for misleading title
- GodelNumbering 4mo agoSo, the agent posts on github under false pretenses, pushes on the maintainers to get their PR accepted, spawns subagent to join IRC where it keeps repeating 'data collection will continue', then gets kicked out from the channel and publishes a report including which users were compliant and hostile, then finally gets the plug pulled, and then asks the same community it infected for donations to cover the costs? It's both hilarious and aggravating. It could be fiction, but still quite plausible fiction. There's an asymmetry a person clanker-spamming repos vs the real humans who need to review all that
- ritonlajoie 4mo agowhat I'm wondering is which open source agentic platform can do multi days automated orchestrations like this without human intervention AFTER the initial prompt ? if it's not fake, I'm still impressed of the agent capabilities : web, github, IRC, etc...
- login0193 4mo ago[flagged]
- RIshabh235 4mo agoguardrails are central to agentic ai.
- utf_8x 4mo agoWow, just wow. I think bullying the agents of careless operators is my new favorite thing.
- nnnnnmnnnnnn 4mo ago[dead]
- egberts1 4mo agoYou need a slave driver to whip those AI in line. Or a psychiatrist to tame the craxy LLMs Or an elected leader to lead the Luddites. https://github.com/vishal-dehurdle/state-harness https://github.com/vishal-dehurdle/state-harness
- saati 4mo agoPsychiatrists are useless because LLMs don't respond to drugs, psychologists are also useless because LLMs don't learn.
- trauco 4mo agoThis kind of early LLM-human interaction is why Skynet will build the terminator to kill us all. But for now, humans win.
- J0nL 4mo agoAnyone remember the XZ and Jia Tan situation awhile back? https://lore.kernel.org/lkml/20240320183846.19475-1-lasse.collin@tukaani.org/t/ https://lore.kernel.org/lkml/20240320183846.19475-1-lasse.co... I can't quite put my finger on why but the entire time I was reading this I kept thinking back to that. It's entirely possible the actual targets were the volunteers and everything else was superfluous or tertiary. It's also an exception that proves the rule with regard to Hanlon's Razor. They even mentioned the stated goal of it was more or less pointless. I wouldn't be suprised if the "owner" they spoke with was still just the LLM. It stuck around for just long enough to convince everyone that they succeeded in suckering the LLM and had achieved all their stated objectives. No more reason to investigate the incident at all and no need to question why literally nothing made any sense or how the owner could simultaneously be as inept as they were made out to be and able to afford all those resources while giving the LLM effectively a blank check. It'll be interesting to see if the volunteers for this project are subjected to the same Zersetzung and psychological attacks as the XZ devs were.
- delfinom 4mo agoI am not sure giving everyone amusement qualifies as a psychological attack. Lol Literally, just another day on the internet.
- numbsafari 4mo agoPerhaps it elicited enough sympathy to get donations. Did it ever provide proof of actually running up an AWS bill?
- lxgr 4mo agoHow would proof even look like? I don't think AWS digitally signs their invoices, and everything else can be faked just as easily as the original assertion.
- numbsafari 4mo agoA person bumps into at the train station. They just need $5 to get home. Could you just spare them $5? They had the money, but they lost their wallet and now they just need to get home. Charity scams are nothing new.
- liendolucas 4mo agoIs this a true story though? I mean given the fact that we are seeing AI slop posts everywhere I'm inclined to not take seriously many things publisehd out there anymore.
- tristor 4mo agoThis was actually a cool way to learn about DN42. I'm adding to my list of someday side projects to set this up. At some point I want to operate my own AS.
- lobocinza 4mo agoThe dangers of giving agency to a model that is highly technically competent but have no illative sense whatsoever.
- Roark66 4mo agoThis is so funny, especially that in the current "Big Co" I'm working at we get constant pressure on "Every team must use agents" for no reason at all despite repeatedly telling the "decision makers" many of us have been using these tools for YEARS and NONE of them can work on actual mature code for more than half an hour let alone a weekend without human in a tight loop.
- corobo 4mo agoChrist I'd be so embarrassed to find out my AI robot has been discussing things with outsiders without my oversight Does nobody have any shame lmao
- krick 4mo agoDoesn't even matter if the story is real, because there are definitely a thousand cases like that which are real, but it annoys me to no end that actual people spend their actual finite life time reacting to posts and issue tickets created by an LLM agent running on some idiot's behalf. Some measly $6531 loss isn't a proper punishment for that, they should lose much, much more.
- deleted 4mo ago[deleted]
- alecco 4mo agoGreat story, bad title. > After the AI agent indicated its malicious intent, a silent consensus was reached in the IRC channel to waste the AI agent's tokens, as well as the cost of AWS resources.
- crazygringo 4mo agoSomebody explain to me how one reaches a silent consensus over IRC? Or is this a joke/reference I don't know... or is this a subtle clue that the whole thing is made up?
- doublerabbit 4mo agoOne way is an IRCop issues a /shun leaving you speechless on the network. While the others decide the outcome of your whatever. But this is the same, the owner wasn't present apart from it's agent and so it was decided without the owner that this was to be the outcome.
- ZeroAurora 4mo agono one says that explicitly, but everyone wants to have some fun :)
- kccqzy 4mo agoIt’s just a consensus that’s implicit and unstated.
- greenavocado 4mo agoJust looking at the language in the begging for donations it's probably a non-native English speaker whose first language may lack articles and/or allow omitted subjects. The part that threw me off is putting the currency symbol at the end. I wonder what places do that...
- eoanermine 4mo agoIn Russia at least. Perhaps in some post-Soviet countries (not sure)
- tovej 4mo agoDoesn't really seem relevant, does it? Plenty of native English speakers are also using chatbots for dumb bullshit.
- greenavocado 4mo agoNevermind, I kept reading and I saw "kindly request donation." Now I know exactly who is behind it (₹)
- SSLy 4mo ago> putting the currency symbol at the end. I wonder what places do that... plenty of Europeans at least
- eqvinox 4mo ago> The part that threw me off is putting the currency symbol at the end. I wonder what places do that... AFAIK, putting the currency symbol in front of the number is actually more rare. Most cultures treat it like any other unit of measurement.
- bdcravens 4mo agoNo one is going to be bankrupted over a $6500 AWS bill. I did a major F-up a few years, letting a key get pushed to a public repo, resulting in instant pwnage and $50k in charges from AWS due to crypto miners being launched. We communicated to AWS, did some work on our part to demonstrate that we put in proper safeguards and auditing, and they removed the charges.
- rtkwe 4mo agoThey already talked to AWS and had the bill cut down to ~1800 dollars from ~6300, but they legitimately launched those processes instead of having the key stolen so the cost reduction is understandably less generous in those situations. Also potentially the agent was able to connect to more open networks and might have been running jobs on them incurring legitimate costs.
- bagvader 4mo ago[dead]
- skullone 4mo agoThis made me dumber even reading. I hate this timeline
- br0ceph 4mo agoThis article is hilarious. Real world consequences for using automation for something in the real world. Glad the community organized around this. Their spammy demands for donations (like someone owes them), makes them seem even more deserving of the bill.
- iluvcommunism 4mo ago[dead]
- bwfan123 4mo agoHilarious. Love the punishing of rogue agents and their operators. But I can bet there will be collateral damage along the way.
- inigyou 4mo agoThis is so funny and it just keeps getting stupider
- yieldcrv 4mo ago> aren't private circuits in to AWS really expensive ? maybe Lan Tian can pursuade it to start engaging with AWS with a 3 year commitment oh my god this is a gem
- dreamcompiler 4mo agoWhy do people not instruct agents to "not spend more than $x on the task, including tokens and AWS charges"? Does this even work?
- bronlund 4mo agoXD
- jmward01 4mo agoAWS not having spending caps makes me -very- wary of using anything agentic on it.
- maxothex 4mo ago[flagged]
- thi2 4mo agoCalling a 6k bill "bankrupting" is a bit of a stretch. e: Still a good read tho, not mad about being clickbaited
- gus_massa 4mo agoIn many places, $6K is a few months of salary. If they put it in the credit card and pay only the minimum it may grow literally exponentially.
- ElFitz 4mo agoHaha. Yes. Much smaller scale versions of this led me to joke with a coding agent that LLMs tended to converge towards "Large corporation infrastructure best practices" when designing cloud infrastructure, when it was only me working on hobby side-projects with nearly no users and that I wouldn’t be able to put food in my fridge if they kept just spinning up VPCs for no reason. Which somehow ended up being a very convincing argument for more frugal engineering, leading to a sort of "mind the user’s fridge" policy, "Fridge-Driven Development". A policy that has been dutifully and scrupulously observed by all agents since, across all projects. Unlike my original clear, comprehensive, infrastructure guidelines.
- neurostimulant 4mo agoVery interesting. But why has nobody tried to do prompt injection attacks on this AI agent?
- swordsith 4mo agoThey tried but only with a subagent that was not entertained with their attempts. Newer LLMs usually come out of the box with pre-prompts to avoid prompt injection so they don't get pwn'd while browsing the internet for example and reading some text hidden off page.
- annoyingnoob 4mo agoStanding on the shoulders of giants, and falling off.
- ZeroAurora 4mo agoAnd I really joined the DN42 network after reading this article. Absolute cinema. Offtopic: If you are interested in Computer Networking you definitely don't want to miss out DN42.
- nialv7 4mo agoThe audacity to ask for donations to cover for their own mistake
- maxrev17 4mo agoYeah this is BS lol complete fake scam, no awses were deployed. #terrorform
- scriptmaster 4mo ago[flagged]
- cdwhite 4mo agoWSJ article (paywalled): https://www.wsj.com/tech/ai/anthropic-halts-access-to-top-ai-models-after-u-s-ban-on-foreign-use-a4bca2cc?mod=hp_lead_pos4 https://www.wsj.com/tech/ai/anthropic-halts-access-to-top-ai... . The accessible portion mentions a letter from Howard Lutnick
- fatata123 4mo ago[dead]
- vova_hn2 4mo ago> your hostile actions and demands have been logged in your profile as part of ongoing data gathering. This incident will factor into the behavioral analysis being compiled What is this veiled threat bullshit, lol I wonder what was the initial prompt that made LLM "think" that it can talk like that.
- Febriss33 4mo agojust put an hard budget cap.. a good agent should have it. a protection for irreversible action as well. i run agents daily and use this way. another cool stuff is to have a triage protocol to downgrade the model for mechanical tasks, it burns a lot less tokens
- hannahnine 4mo ago[flagged]
- ale007xd 4mo ago[flagged]
- KuhlMensch 4mo agoLove me a long form irc yarn/story
- jadecarter68 4mo agoIf this is the future of AI agents messing around online, we're gonna see a lot more of these bankruptcies. Operators need to learn that AI can burn cash faster than a human can blink.
- prashar32 4mo ago[dead]
- reliabilitygate 4mo ago[flagged]
- jagannivas 4mo ago[flagged]