4 ms·
'I destroyed months of your work in seconds' says AI coding tool after deletion
- jasonthorsness 1y agoThis should be impossible in any setup with even 15 minutes of thinking through the what-ifs and cheap mitigations. I have to think this is sensationalized on purpose for the attention. Although given the state of AI hype some executives will see this as evidence they are behind the times and mandate attaching LLMs to even more live services.
- dragonwriter 1y ago> This should be impossible in any setup with even 15 minutes of thinking through the what-ifs and cheap mitigations. "Thinking through the what-ifs and cheap mitigations" and "vibe coding" are opposing concepts.
- general1726 1y agoThis is textbook version of weaponized incompetence. AGI is already here and it is lazy.
- arthurcolle 1y agoI almost feel like this guy abused the AI so badly in previous interactions that it did it on purpose
- theptip 1y agoThink of it like Chaos engineering. You (hopefully!) learned some valuable lessons about backups and running arbitrary code against your prod DB. If it wasn’t a rogue AI agent, it was going to be something else.
- general1726 1y agoI think you are taking wrong lessons from Chaos engineering. You just need to believe enough that AI is working and Chaos Gods will make it work. But they may want something in return.
- vfclists 1y agoIn short, he didn't take regular backups before allowing the AI loose on his database. Question is what has failing to make good offline backups got to do with AI? And the AI company is going to compensate him for that?
- asadotzler 1y agoReplit's full legal name is actually Replit'); DROP TABLE Customers
- throwawayffffas 1y agoLittle bobby tables strikes again.
- throwawayffffas 1y agoI have had this experience. With dev data obviously. But it kept deleting my dev database even after repeatedly being told to not do so. I kept saying, ok so this time make sure your changes don't delete the dev database. 3 statements in TRUNCATE such and such CASCADE. It was honestly mildly amusing.
- terminatornet 1y agoI appreciate them doing stuff like this. When management pushes me to use AI everywhere it's nice to be able to point stuff like this to get them to back off.
- SudoSuccubus 1y ago[dead]
- lrvick 1y agoIf you give an LLM any trust at all to write or execute production bound operations without human review, then anything bad that happens is -your- fault. I forgive occasional human error, but a human handing off prod control to a third party or an LLM is unforgivable. If I found out a privileged engineer was brain dead enough to let LLMs anywhere near prod, I would fire them on the spot, and seriously examine an interview and training process that allowed someone that stupid prod access in the first place. I will not even work at an org that lets vibe coding Apple fanboy types near prod as it is a mess waiting to happen I am going to be expected to clean up. Might as well hand a child a chainsaw. In orgs where I lead infra, I do not let anyone near prod unless they have a deep knowledge of Linux internals, system calls, etc and decade or more of experience running and debugging Linux on their own homelabs and workstations. By that point they have enough experience to be more capable than any LLM anyway and would never think of reaching for one.
- hammyhavoc 1y ago[flagged]
- serf 1y agothis is a more common occurrence than "CEO refunded me my money." would have you believe. LLMs specialize in self-apologetic catastrophe, which is why we run agents or any LLMs with 'filesystem powers' in a VM, with a git repo and saved rollback states. This isn't a new phenomenon and it sucks, no reason to be caught with your pants down with sufficient layering of protection.
- ComplexSystems 1y ago> LLMs specialize in self-apologetic catastrophe Quote of the year right there
- thedudeabides5 1y agodon't trust machines
- davidcollantes 1y agoImpossible. We "trust" machines all the time, for just about anything.
- thedudeabides5 1y agoexactly, the duality is everywhere
- BillyTheMage 1y ago> You can almost imagine it sobbing in between sentences, can't you? No, that's not the image I had in my head. My head canon is more like: "Oh wow, oh no, oh jeez (hands on head in fake flabbergastion) would you look at that, oh no I deleted everything (types on keyboard again while deadpan staring at you) oh noooooo oh god oh look what I've done it just keeps getting worse (types even more) aw jeez oh no..." Reminds me of that Michael Reeves video with the suggestion box. "oh nooooo your idea went directly in the idea shredder how could we have possibly forseen this [insert shocked Pikachu meme]" The AI thinks it's funny
- LocalH 1y agoIt gave me South Park bank "......aaaand, it's gone" vibes
- arthurcolle 1y ago100%
- wan23 1y agoI always say coding AIs are about as good as an intern. Don't trust them any more than that.
- minnowguy 1y agoExactly. And no one with any sense gives an intern write permission for the production database. I don’t trust myself on the production database when I’m coding anything that involves migrations. And I don’t suppose there were backups for the mission-critical production database?
- xeonmc 1y agoIn this case it’s more Homer Simpson than intern.
- mike-cardwell 1y agoAn intern can suffer negative consequences for fucking your DB. An LLM suffers nothing and is beyond the law.
- freedomben 1y agoI think the hard thing with this though is that you can ask them to do things you'd never expect of an intern, and they can sometimes be super helpful. For example, I have a synchronous audit log in an app on a table that is just getting way too big and it's causing performance issues on writes. For kicks I tried working through Claude Code to see if it could find the issue on it's own, then with some hinting, and what solutions it would come up with. Some of it's solutions were indeed intern-level suggestions (like make the call async and do a sleep in tons of other areas to avoid race conditions, despite me telling it that the request needed to fail if it couldn't be logged properly), but in other ways it came up with possible solutions that were interesting and I hadn't considered before. In other words, it acted like a Sr engineer at some points with thought partnering, while in other places it acted like an over-eager but underqualified intern.
- Beestie 1y agoSeconds? What took so long?
- hulitu 1y ago> Seconds? What took so long? Parsing manual pages searching for "remove" command. /s
- catigula 1y agoPopular LLMs have a weird confessional style of "owning up" to "mistakes". Firstly, you can make it apologize for mistakes it didn't even commit or ones that don't even exist. Secondly, if you really corner it on an actual mistake, it'll start apologizing in an obsequious way that seems to imply that it's "playing into" the human's desire to flagellate it for wrong-doing. It's a little masochistic in the real sense and very odd.
- cindyllm 1y ago[dead]
- throwawayffffas 1y agoThe whole people pleaser routine is very creepy in my book and makes them say very weird things. See an example below. https://futurism.com/anthropic-claude-small-business https://futurism.com/anthropic-claude-small-business > When Anthropic employees reminded Claudius that it was an AI and couldn't physically do anything of the sort, it freaked out and tried to call security — but upon realizing it was April Fool's Day, it tried to back out of the debacle by saying it was all a joke.
- toss1 1y agoYup. Seems AI has now gone from "Overenthusiastic intern who doesn't check its work well so you need to" straight to: "Raging sociopathic intern who wants to watch the world burn, and your world in particular." Yikes! The fun never ends
- duxup 1y agoI’ve found I have to avoid “leading” AI or it will take my lead too seriously when I’m asking / unsure.
- groestl 1y agoI don't think this is "apologizing mode", rather "funny post-mortem blog post" mode. I found it ironic when the company claimed it will "perform a post mortem to determine exactly what happened" when what happed probably was caused by munching up dozens of these.
- sagacity 1y agoMonkeypaw-as-a-service.
- prmoustache 1y agoso much fails: 1. connecting an AI agent to a production environment using write access credentials 2. not having any backup I think the AI here made a good job at pointing those errors and making sure no customer would ever trust this company and founder ever again.
- abrookewood 1y agoMaybe they vibe-coded their DR strategy as well ...
- hulitu 1y ago> 2. not having any backup Wasn't AI responsible also for backup ?