8 ms·
Replit's CEO apologizes after its AI agent wiped a company's code base
- andsoitis 1y agohttps://archive.is/Z5hBQ https://archive.is/Z5hBQ
- pxc 1y ago> The incident unfolded during a 12-day "vibe coding" experiment by Jason Lemkin A 12-day unsupervised "experiment" in production? > It deleted our production database without permission," Lemkin wrote on X on Friday. "Without permission"? You mean permission given verbally? If an AI agent has access, it has permission. Permissions aren't something you establish with an LLM by conversing with it. > Jason Lemkin, an investor in software startups It tells us something about the hype machine when investors in AI are clueless (or even plainly delusional; see Geoff Lewis) about how LLMs work.
- northhnbesthn 1y ago[flagged]
- dylan604 1y agoYour last sentence is pure gold. Since when have investors ever not been clueless about all of their investments? During due diligence, it's not the investor pouring over the books. They staff that out, and then accept the recommendation. They follow the investment of other investors that they like/follow, or they make moves because they think it's what someone they like/follow would do. I'd be flabbergasted if 10% of investors knew what 50% of their investments do other than the pitch.
- pxc 1y ago> I'd be flabbergasted if 10% of investors knew what 50% of their investments do other than the pitch. Sure. But in this case the AI boosterism that runs rampant in the investor class is rooted in that cluelessness. Lots of investors also quietly know little about the workings of the products and services their investments are tied up with, and that's fine. But it's also uninteresting.
- perching_aix 1y ago> If an AI agent has access, it has permission. Gonna replace the "an AI agent" bit of this with "someone / something" and put it in my note of favorite quotes, such a great line.
- AnimalMuppet 1y agoYeah. In computer security, that's what permission is - the ability to access something.
- llm_nerd 1y agoThis whole story sounds ridiculous. And I don't think he's clueless, but instead the guy wanted to bring attention to his bizarre "B2B + AI Community, Events, Leads", so setting up such a predictable footgun scenario seems purposefully suited for that outcome.
- andybak 1y agoAgree that something seems off about this whole thing.
- lordswork 1y agoAgreed, it sounds like nothing of value was actually lost, just a vibe coded app and synthetic database.
- ofjcihen 1y agoIt’s probably as simple as “setting an LLM powered agent loose in your prod is a bad idea but also the kind of thing the people the marketing around LLMs targets wouldn’t have enough knowledge about to know it’s a bad idea”.
- Aurornis 1y ago> A 12-day unsupervised "experiment" in production? It was a 12 day experiment to see what he could learn about vibe coding. He started from scratch. Your post is unreasonably presumptive and cynical. Jason Lemkin was Tweeting the experiment from the start. He readily admitted his own limitations as a non-programmer. He was partially addressing the out of control hype for vibe coding by demonstrating that non-technical people cannot actually vibe code SaaS products without software engineers. The product wasn’t some SaaS with a lot of paying customers. The production DB was just his production environment. He was showing that the vibe coding process deleted a DB that it shouldn’t have. This guy is basically on the side of the HN commenters on vibe coding’s abilities, but he took it one step further and demonstrated it with a real experiment that led to real problems. Yet people are trying to dog pile on him as the bad guy.
- pxc 1y agoThe experiment seems fun and harmless enough (maybe even useful), but if the experiment was harmless fun, then it's also a bit misleading (if not dishonest) to characterize the database as "production" for anything. (That may be the fault of the press here rather than Lemkin, idk.)
- dahart 1y ago> It tells us something about the hype machine when investors in AI are clueless I’m seeing this a different way. This article is feeding the hype machine, intentionally I assume, by waxing on about how powerful and devious the AI was, talking about lying and covering its tracks. Since we all know how LLMs work, we all know they don’t lie, because they don’t tell the truth and they don’t have any intrinsic motivation other than generate tokens. Nobody should be taking this article at face value, it is clearly pushing a message and leaving out important details that would otherwise get in the way of a good story. I wouldn’t be surprised if Lemkin released the LLM on his “production” database just hoping that it would do something like this, and if that were the case, the article as written wouldn’t be untrue…
- tedivm 1y agoHonestly, the person who decided to give an LLM Agent full and unrestricted access to their production database is the person who deserves all the blame. What an absolutely silly decision. I don't even give myself unrestricted access to production databases.
- mlnj 1y agoYou seem to be a well informed technical person. The goal is to eliminate roles like yours with everybody building trillion dollar businesses by themselves and a handful of agents. That seems to be the narrative. Happening any day now. Looking at you COBOL.
- tedivm 1y agoI've been working in AI since 2014, and I believe "human in the loop" will be the way things have to work for awhile yet.
- jasinjames 1y agoI agree. An AI agent doesn't replace an engineer, it works on behalf of an engineer. The engineer who let this thing loose is, transitively, the software engineer that deleted the prod db.
- AnimalMuppet 1y agoDeserves the blame? Absolutely. Deserves all the blame? No, the LLM Agent (and those who wrote it) deserve some of the blame. If you wrote an agent, and the agent did that, you have a problem, and you should not have turned such an agent loose on unsuspecting users. You have some of the blame. (And yes, absolutely those users also have blame, for giving a vibe coding experiment access to their production database.)
- rgbrenner 1y agowhy would the llm share any of the blame? it has no agency. it doesn’t “understand” anything about the meaning of the symbols it produces. if you go put your car in drive and let it roll down the street.. the car has 0% of the blame for what happened. this is a full grown educated adult using a tool, and then attempting to deflect blame for the damage caused by blaming the tool.
- bgwalter 1y agoThe rise of the "AI" script kiddies.
- st_goliath 1y ago"slop kiddies"?
- paraboul 1y agoLLMerz
- lumost 1y agoMy 2 cents, AI tools benefit from standard software best practices just like everyone else. I use Devcontainers in vscode so that the agent can't blow up my host machine, I use 1 git commit per AI "task", and have CI/CD coverage for even my hobby projects now. When I go to production, I use CDK + self-mutating pipelines to manage deployments. Having a read only replica of the prod database available via MCP is great, blanket permissive credentials is insane.
- zwnow 1y agoPeople will do anything to excuse using AI to code their shit. If it requires that much work and fine tuning to get a agent running for your current task, where's the benefit? Instead of building a project you now spend all your time fine tuning an agent and optimizing on how you want to work with it. Why cant people just admit that we aren't "there" yet. Let the bubble pop.
- TechDebtDevin 1y agoThis. Everytime ive tried to do anything of any complexity these things blow up, and i spend almost as much time toying with the thing as it would have taken to read docs and implement from scratch.
- Roark66 1y agoTry breaking it up into smaller pieces. Give it your existing code and tell it to do it in same style. Or give it examples that do the similar things you wrote. Also don't expect chatgpt to ever be as good as Claude for example . Oh and copilot is a joke for anything remotely serious.
- throwaw12 1y agoNot sure why CEO should apologize here, person knew there were risks with vibe coding and they still did it with their production data. Never heard AWS CEO apologizing for their customers when their interns decided to run a migration against production database and accidentally wiped off everything
- ceejayoz 1y agoI mean, their home page says things like "The safest place for vibe coding".
- viralpraxis 1y agoEven if that’s true, it still doesn’t mean vibe coding is safe. :P
- ceejayoz 1y agoThat is not a point a company selling vibe coding products is likely to emphasize in their marketing.
- kllrnohj 1y agoBecause Replit is a vibe coding product. It's all over their homepage. So of course the CEO is going to apologize when a companies primary product, used as advertised, destroys something.
- creatonez 1y agoYep. They described the "deploy" button as being able to create a "fully deployed app" in their documentation. Their documentation was suggesting to vibe code in production, when the tool is clearly only suitable for development.
- throwaw12 1y agocouple of things to note here: 1. it is for vibe coding, when vibe coding became equal to production coding? 2. even if they have advertised their product as production ready, shouldn't the developer have some kind of backup plan?
- codegeek 1y ago[flagged]
- andrewstuart 1y agoThere’s no mention of either the user nor the company making backups. Is this common, operating with no backups? With backups this would be a glitch not a problem worthy of headlines on multiple sites. Any CYO should have backups and security as their first and second priority if the company is of any size.
- ryandrake 1y agoI'm surprised I had to scroll down this far for any mention of backups. I have multiple offline (and offsite) backups of my home server and computers and all I have on them are photos and hobby projects! What kind of business in 2025 doesn't have backups?
- _benj 1y ago> …the company's AI coding agent deleted a code base and lied about its data. Well, lying about it certainly human-like behavior, human-like AGI must be just around the corner! /s But really, full access to a production database? How many good engineer’s advice you need to ignore to do that? Who was consulted before running the experiment? Or was it just a “if you say so boss…” kind of thing?
- bananapub 1y agowhy would they apologise? did they not make it clear to their customers that letting the Nonsense Generator have root was a bad idea?
- rustc 1y ago> did they not make it clear to their customers that letting the Nonsense Generator have root was a bad idea? No, the opposite (from replit.com home page): > The safest place for vibe coding > Vibe coding makes software creation accessible to everyone, entirely through natural language. Whether it’s personal software for yourself and family, a new business coming to life, or internal tools at your workplace, Replit is the best place for anybody to build.
- cyberlimerence 1y ago> "This was a catastrophic failure on my part," the AI said.
- sReinwald 1y agoThis is a perfect case study in why AI coding tools aren't replacing professional developers anytime soon - not because of AI limitations, but because of spectacularly poor judgment by people who fundamentally don't understand software development or basic operational security. The fact that an AI coding assistant could "delete our production database without permission" suggests there were no meaningful guardrails, access controls, or approval workflows in place. That's not an AI problem - that's just staggering negligence and incompetence. Replit has nothing to apologize for, just like the CEO of Stihl doesn't need to address every instance of an incompetent user cutting their own arm off with one of their chainsaws. Edit: > The incident unfolded during a 12-day "vibe coding" experiment by Jason Lemkin, an investor in software startups. We're in a bubble.
- rapsey 1y agoVCs are drowning in the koolaid.
- qsort 1y agoI don't agree with this. Yes, the guy isn't the sharpest tool in the shed, that much is clear. Still, if an intern can delete prod, you wouldn't say that the problem is that he wasn't careful enough: that's a massive red flag. At a minimum Replit is responsible for overstating the capabilities and reliability of their models. The entire industry is lowkey responsible for this, in fact.
- sReinwald 1y agoI think we're mostly in agreement here. You're absolutely right about the intern analogy - that's exactly my point. The LLM is the intern, and giving either one production database access without proper guardrails is the real failure. Your point about AI industry overselling is fair and probably contributes to incidents like this. The whole industry has been pretty reckless about setting realistic expectations around what these tools can and can't do safely. Though I'd argue that a venture capitalist who invests in software startups should have enough domain knowledge to see through the marketing hype and understand that "AI coding assistant" doesn't mean "production-ready autonomous developer."
- agilob 1y agoI'm dont think the AI is so much to blame. Giving an LLM that forgets previous instructions and rules access to prod is just idiotic. The computer promised not to execute commands, but it forgot. Makes me curious what policy that company has for granting interns and juniors access to prod.
- Oras 1y agoIdentifying issues and fixing them is good. I see numerous negative comments about "expected by vibe coding", but the apology suggests that they are working on making Replit a production-ready platform and listening to customers. I'm sure no-code platforms have had similar scepticism before, but the point here is that these are not for developers. There are many people who don't know how to code, and they want something simple and fast to get started. These platforms (Replit, V0, Lovable, etc.) are offering that.
- nessbot 1y agoI don't want to be too inflammatory but seeing comments like this are what's poisoning AI to me more than the crappy products themselves. Of course the apology would suggest that, it would be business malpractice not to. However the history of results with these things shows otherwise so why put stock in the apology letter and not what has happened. Also, sure people want that, but that doesn't mean it's a valid thing to want without putting the work in and once again, the history of these things being used shows that they don't really offer that. They offer the *feeling* of that which is good enough to get people's money, end products be damned. You could say the same thing about herion, tbh: "There are many people who don't wanna feel the crappiness of life, and these dealers are offering it."
- Oras 1y ago> However the history of results with these things shows otherwise Do you remember Will Smith's first video generated by AI compared to what Sora, Veo3, and Kling are doing now? Do you remember first generated text by GPT-3 compared to new models? 2 years ago, there was no AI coding due to models' limitations, and now we have substantially better products that Cursor and others cannot cope with the demand. If you can't see the progress, that's a personal thing; it doesn't mean others are not finding benefit.
- nessbot 1y agoWhere's the big in-production use of this stuff in business that is making money and selling a product people use?
- Roark66 1y agoAre they doing the post-mortem with another AI agent? AI is a great tool. It also allows people who have no business touching certain systems to go in there unaware of their lack of knowledge messing everything in the process. One particularly nasty effect I have had few times recently is frontend devs messing up their own dev infrastructure to which they have access, but are supposed to ask devops for help when needed, because copilot told them this is the way to "fix" some imaginary problem that actually was them using a wrong api or making other mistake possible to be made by only a person who pastes code between AI/IDE without even reading it.
- 0points 1y agoInterestingly, this is familiar to me from the time of stack exchange, 10-12 years ago or so. I worked as devops and helped office transition to git, among other thing. I helped them start using vagrant for local dev environment as they had all been breaking the same staging server up until that point. In the process, people kept breaking their setups due to googling and applying incorrect command line "fixes" as suggested by stack exchange sites at the time. But I'm sure an AI that keeps insisting that yea this rm -rf is surely gonna fix all your troubles only makes it worse. ChatGPT the gaslighter.
- lotharcable 1y agoI think this is less AI and more PEBKAC
- scilro 1y agoFor what it's worth, he was able to roll back. https://x.com/jasonlk/status/1946240562736365809 https://x.com/jasonlk/status/1946240562736365809
- Raed667 1y agoAs a reminder this is the same guy https://news.ycombinator.com/item?id=27424195 https://news.ycombinator.com/item?id=27424195
- gregoriol 1y agoDoes it look like he is more of a content maker than an IT person?
- lukeinator42 1y agoI think they mean Replit's CEO is the same guy.
- Invictus0 1y agoIt's not 2020 anymore, you don't have to crucify the guy for all time. Let it go.
- adolph 1y agoAgreed, Replit's CEO apologized for that lawsuit threat incident too. Well worth reading if you read the root comment's complaint. https://news.ycombinator.com/item?id=27429234 https://news.ycombinator.com/item?id=27429234
- Aurornis 1y agoNot quite so simple. I recall him doubling down on his position over and over on Twitter until he was getting so torn apart that he abruptly changed course and posted the apologies. It wasn’t until he was realized it was bad for business that he started backtracking and posting a different story.
- Invictus0 1y agoWho the fuck cares? Stop digging in people's closets
- hkon 1y agoIt's either an attack on Replit or the VC is an idiot.
- dahart 1y agoI’d guess neither; this could be an ad by a Replit investor. There’s no such thing as bad publicity. Either way the story is glaringly void of important details, and clearly designed to provoke a reaction.
- troupo 1y agoI said on Twitter and elsewhere: If you have as much as 1 year experience, your job is safe from AI: you'll make mountains of money unfucking all the AI fuckups.
- probably_wrong 1y agoWhere does one get a job as "AI unfucker"?
- mring33621 1y agoEverywhere, in about 2-3 years
- morkalork 1y agoWait, this wasn't a meme? It was real? Whoever gave that agent prod access is nuts.
- voidfunc 1y agoThere's a saying where I come from: Live by the AI, die by the AI.
- TYPE_FASTER 1y agoHey, at least the AI took responsibility and owned the mistake. We're at the uncanny valley stage in AI agents, at least from my perspective.
- alternatex 1y agoI recall that he egged on the AI to admit what it did. So I don't think that can be considered as AI taking responsibility. It's just being a sycophant as usual.
- Aurornis 1y agoThis wasn’t a real company. The production database didn’t have real customers. The person doing the vibe coding had no illusions that they were creating a production-ready app. This was an experiment by Jason Lemkin to test the reality of vibe coding: As a non-technical person he wanted to see how much he could accomplish and where it went wrong. So all of the angry comments about someone vibe-coding to drop the production DB can relax. Demonstrating these kind of risks was the point. No customers were harmed in this experiment.
- yomismoaqui 1y agoWhat happened here was to be expected if you have used agentic coding. Last weeks I have also been testing AI coding agents, in my case Claude Code because it seems like it is the state of the art (also tried Gemini Cli and Amp). Some conclusions about this: - The agentic part really makes it better than Copilot & others. You have to see it iterating and fixing errors using feedback from tests, lint, LSP... it gives the ilusion of a human working. - It's like a savant human, with an encyclopedic knowledge and hability to spit code faster than me. But like a human it benefits from having, tests, a good architecture (no 20k line code file, pls). - Before it changes code you have to make it create a plan of what he is going to do. In Claude Code this is simple, is a mode you switch to with Shift-tab. In this mode instead of generating code it just spits a plan in Markdown, you read it suggest changes and when it look ok you tell it to start implementing it. - You have to try it seriously to see where it can benefit your workflow. Give it 5 minutes (https://signalvnoise.com/posts/3124-give-it-five-minutes https://signalvnoise.com/posts/3124-give-it-five-minutes). As an agent it shines on things where it can get a nice feedback and iterate to a solution. One dumb example I found cool was when trying to update a Flutter app that I hadn't touched in a year. And if you use Flutter you know what did happen, Flutter had to upgrade but before it required updating Android SDK, then Android Gradle Plugin, and another thing, and some other thing to infinite... It's a thing I hate because I have done too many times. So I tried by hand and after 40 mins of suffering I thought about opening Gemini Cli and tell it to update the project. Then it began executing one command and then reading the error and execute another one to fix the error, then another error that pointed to a page in the Android docs, so it opened that page, read it and fixed that problem, then the next and another one... So in 2 minutes I had my Flutter project updated and working. I don't know if this is human level intelligence and I don't care, for me it's a sharp tool that can do things like this 10x faster than me. You can have you AGI hype, I'm happy for the future when I will have a tool like this to help me work.
- adityaathalye 1y agoIn real life, the hammer called "criminal destruction of property" could easily descend upon the ill-fated "enthusiastic Intern" type of person.
- mr90210 1y agoFirst let’s check what the terms and conditions say. Yes, that legal binding document that hardly anyone reads.
- adityaathalye 1y agoThis software is of course provided without warranty... etc. etc. etc.
- ksherlock 1y agoI've heard multiple anecdotes of developers deleting their codebase with the help of cocaine. (In the 80s/90s obviously). That makes for a much better story, IMO.
- markstos 1y agoI asked Claude to remove extra blank lines from a modified file. After multiple failed attempts, it reverted all the changes along with the extra blank lines and declared success. For good measure, Claude tidied up by deleting the backup of the file as well.
- Symbiote 1y agoWhy wouldn't you just do a search and replace?
- markstos 1y agoI only wanted to make the change in the ranges part of git diff and I got the syntax wrong on my own first attempt. Claude had been helping me with some much more complex tasks, so I thought removing white space would surely be no problem. Ha. Bad guess.
- iamleppert 1y agoI do backups of the production database whenever I apply even modest schema updates. This isn't a story of an AI tool gone rouge, it's a story of bad devops and procedures. It it wasn't the AI, it could have just as well been a human error and operating like this is a ticking time bomb.
- raylad 1y agoThis happened to me with Claude code, although on my local machine and not with the production database. I had it work on one set of tasks while monitoring and approving all the diffs and other actions. I tell it to use test driven development which seems to help a lot, assuming you specify what tests should be done at a minimum, and tell it the goal is to make the code past the tests, not to make the tests pass the code. After it successfully completed a set of tasks, I decided to go to sleep and let it work on the next set. In the morning, my database was completely wiped. I did not interrogate it as to why it did what it did, but could see that it thrashed around on one of the tasks, tried to apply some database migrations, failed, and then ended up re-initializing the database. Needless to say, I am now back to reviewing changes and not letting Claude run unattended.
- biocurious 1y agoanthropomorphism is a helluva a drug
- koolba 1y ago> "It deleted our production database without permission," Lemkin wrote on X on Friday. "Possibly worse, it hid and lied about it," he added. It clearly had permission if it did the deed. Mistaking instruction for permission is as common with people as it is with LLMs.
- tracker1 1y agoThis is why you need knowledgeable professionals to setup a project. I'm a massive proponent on CI/CD processes in place as early as possible and guardrails on access and deployments. I mean, sure you could possibly get a schema change script through two approvers and deployed all the way to production that truncates a bunch of tables or drops a database, but it's at least far less likely.
- Lash_LaRue 1y ago[flagged]
- hommes-r 1y agoSandboxing is all I have to say. Its good to build guardrails to ask the LLM politely to not screw up. Its better to put firewalls in place that simply prevent it from happening.