8 ms·
AI helps ship faster but it produces 1.7× more bugs
- SideburnsOfDoom 11mo ago> ship faster but it produces more bugs This is ... not actually faster.
- bogzz 11mo agooh wow, an LLM-based company with an article that claims AI is oddly not as bad when it comes to generating gobbledegook as everyday empirical evidence should suggest
- jjmarr 11mo agoCoderabbit is an LLM code review company so their incentives are the opposite. AI is terrible and you need more AI to review it. fwiw, I agree. LLM-powered code review is a lifesaver. I don't use Coderabbit but all of my PRs go through Copilot before another human looks at it. It's almost always right.
- bpicolo 11mo agoTheir incentives are perfectly aligned - you’re making more bugs, surely you need some AI code review to help prevent that. It’s literally right at the end of their recommendations list in the article
- jjmarr 11mo agoThe original comment said: > an article that claims AI is oddly not as bad when it comes to generating gobbledegook Ironically, Coderabbit wants you to believe AI is worse at generating gobbledegook.
- GoatInGrey 11mo agoMake the gobbledygook from your gobbledygook generator better with our proprietary gobbledygook generator. I'm obviously taking the piss here, but the irony is amusing.
- jjmarr 11mo agoIt sounds stupid but it works. I've seen it. I put Copilot on AI-generated slop PRs and hit refresh until it stops commenting. It's great seeing it take out all the dead code.
- elktown 11mo agoYou're comment history suggests a pro-AI bias on par with AI companies. I don't understand it. It seems like critical thinking, nuance, and just basic caution have been turned off like a light-switch for far too many people.
- naasking 11mo agoOur industry never exhibited an abundance of caution, but if you have trouble understanding the value of AI here, consider that you are akin to an assembly language programmer in the 1970s or 80s who couldn't understand why people are so gung-ho about these compilers that just output worse code than they could write by hand. In retrospect, compilers only got better and better, and familiarity with programming languages and compilation toolchains became a valuable productivity skill and the market for assembly language programming either stagnated, or shrank. Doesn't it seem plausible to you that, whatever the ratio of bugs in AI-generated code today, that bug count is only going to really go down? Doesn't it then seem reasonable to say that programmers should start familiarizing themselves with these new tools, where the pitfalls are and how to avoid them?
- bogzz 11mo agocompilers aren't probabilistic models though
- mwigdahl 11mo agoTrue. The question is whether that's relevant to the trajectory described or not.
- naasking 11mo agoSuccessful compiler optimizations are probabilistic though, from the programmer's point of view. LLMs are internally deterministic too.
- miningape 11mo agoWhat? Do you even know how compilers work?
- smb06 11mo agoDo you use Copilot for coding and then also Copilot for reviewing? Or are you using some other coding agent and Copilot only for PR reviews?
- jjmarr 11mo agoI do not use Copilot for coding. I use other assistants now. Copilot code review is amazing. I use it all the time.
- tyleo 11mo agoI have a theory that vibe coding existed before AI. I’ve worked with plenty of developers who are happy to slam null checks everywhere to solve NREs with no thought to why the object is null, should it even be null here, etc. There’s just a vibe that the null check works and solves the problem at hand. I actually think a few folks like this can be valuable around the edges of software but whole systems built like this are a nightmare to work on. IMO AI vibe coding is an accelerant on this style of not knowing why something works but seeing what you want on the screen.
- jmkni 11mo agoBlindly copying and pasting from StackOverflow until it kinda sorta works is basically vibe coding AI just automates that
- giantg2 11mo agoYeah, but you had to integrate it until it at least compiled, which kind of made people think about what they're pasting. I had a peer who suddenly started completing more stories for a month or two when our output was largely equal before. They got promoted over me. I reviewed one of their PRs... what a mess. They were supposed to implement caching. Their first attempt created the cache but never stored anything in it. Their next attempt stored the data in the cache, but never looked at the cache - always retrieving from the API. They deleted that PR to hide their incompetence and opened a new one that was finally right. He was just blindly using AI to crank out his stories. That team had something like 40% of capacity being spent on tech debt, rework, and bug fixes. The leadership wanted speed above all else. They even tried to fire me because they thought I was slow, even though I was doing as much or more work than my peers.
- skydhash 11mo ago> Yeah, but you had to integrate it until it at least compiled, which kind of made people think about what they're pasting That’s a very low bar. It’s easy to get a program to compile. And if it’s interpreted, you can coast for months with no crashes, just corrupted state. The issue is not that they can’t code, it’s that they can’t problem solve and can’t design.
- phartenfeller 11mo agoDefinitely. But AI can also generate unit tests. You have to be careful by exactly telling the LLM what to test for and manually check the whole suite of tests. But overall it makese feel way more confident over increasing amounts of generated code. This of course decreases the productivity gains but is necessary in my opinion. And linters help.
- SketchySeaBeast 11mo agoI've been using Claude sonnet 4.5 lately and I've noticed a tendency for it to create tests that prove themselves. Rather than calling the function we're hoping to test, it re-implements the code in the test and then tests it there. It's helpful, and it usually works very well if you have well defined inputs and outputs, I much prefer it over writing tests manually, but you have to be very careful.
- stuaxo 11mo agoIt doesn't generate good tests by default though. I worked on a team where we had someone come in and help us improve our tests a lot. The default LLM generated tests are bit like the ones I wrote before that experience.
- dnautics 11mo agothis is solvable by prompting and giving good examples?
- LanceH 11mo ago> It doesn't generate good tests by default though. I agree with this. I've found I need a whole separate cycle of test writing to get proper (in both scope and accuracy) coverage. It does help tremendously with all the boilerplate of tests, and it seems to be quite good at setting up numerous tests for all combinations of variables. It does have to be done explicitly, though. And you do need to mind when a test fails whether it fixes the test or the code.
- strangescript 11mo agoDo they consider code readability, formatting and variable naming as "errors" for the overall count. That seems dubious given where we are headed. No one cares what a compiler or js minifier names its variables in its output. Yes, if you don't believe we will get there ever, then this is totally valid complaint. You are also wrong about the future.
- oblio 11mo agoThe "future" is a really long time. I'll take the other side of your bet for the next 10 years but I won't take it for the next 30 years. In that spirit, I want my fusion reactor and my flying car.
- strangescript 11mo agoIf your outlook is 10 years then for sure, its valid. I am not sure how you come to that conclusion logically though. At the beginning of the year we had 0 code agents. Now we have dozens, some are basically free, (of various degrees of quality, sure). The last 2-3 months of releases have been an unprecedented whirlwind. Code writing will be solved by the end of 2026. Architecture, maybe not, but formatting issues isn't architecture.
- oblio 11mo agoIt's similar with every technology, there's a reason we have sigmoids. In 1960 they were planning nuclear powered cars and nuclear mortars.
- bopbopbop7 11mo agoCode writing was solved in 1997 when Dreamweaver was released.
- oblio 11mo agoNope, it was solved with Visual Basic in 1991. And with Nextstep in 1989. And with... I really dislike people comparing GenAI with compilers. Compilers largely do mechanic transformations, they do almost 0 logic changes (and if they do, they're bugs). We are in an industry that's great at throwing (developing) and really bad at catching (QA) and we've just invented the machine gun. For some reason people expect the machine gun to be great at catching, or worse, they expect to just throw things continuously and have things working as before. There is a lot of software for which bugs (especially data handling bugs) don't meaningfully affect its users. BUT there isn't a lot of software we use daily and rely on for which that's the case. I know that GenAI can help with QA, but I don't really see a world where using GenAI for both coding and QA gets us to where we want to go, unless as some people say, we start using formal verification (or other very rigorous and hopefully automatable advanced verification), at which point we'll have invented a new category of programmers (and we will need to train all of them since the vast majority of current developers don't know about or use formal verification).
- cgearhart 11mo agoSo…great for prototyping (where velocity rules) but somewhere between mixed to negative for critical projects. Seems like this just puts some mildly quantitative numbers behind the consensus & trends I see emerging.
- GoatInGrey 11mo agoI'm seeing parallels between this and factory-assembled houses. Input costs are lower and velocity is higher. You get a finished product out the door quicker, though maintenance is more expensive. Largely because the product is no longer a collection of individual parts made to be interfaced by a human. It is instead a machine-assembled good that requires a machine to perform "the work". Therefore, because the machine is only designed to assemble the good, your main recourse is to have the machine assemble a full replacement. With that framing, there seems to be a tradeoff to bear in mind when considering fit for the problem we're meaning to solve. It also explains the widespread success of LLMs generating small scripts and MVPs. Which are largely disposable.
- everdrive 11mo agoSounds like what companies have been scrambling for this whole time. People just want to dump something out there. They don't really care if it works very well.
- 0x3f 11mo agoAt best this would be 1.7x more _discovered_ bugs. The average PR (IMO) is hardly checked. AI could have 10x as many real issues on PRs, but we're just bad at reviewing PRs.
- core-utility 11mo agoCouldn't this be true in the other direction as well? Anecdotally I see developers putting a lot more scrutiny into vibe coded PRs, while AI code tends to be highly commented (by the AI) and potentially easier to read. I've seen way more human comments of "I don't know what this does but if I remove it everything breaks" in systems.
- maerF0x0 11mo agoIt's been suggested that code reviews shouldnt be about finding bugs, because they do not really work well for that goal. https://www.microsoft.com/en-us/research/wp-content/uploads/2015/05/PID3556473.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
- bodge5000 11mo agoAs has already been said, we've been here before. I could ship significantly faster if I ignored any error handling or edge cases and basically just assumed the data would flow 100% how I expect it to all the time. Of course that is almost never the case, so I'd end up with more bugs. I'd like to say that AI just takes this to an extreme but I'm not even sure about that, I think it could produce more code and more bugs than I could in the same amount of time but not significantly so if I just gave up on caring about anything
- dnautics 11mo ago> I could ship significantly faster if I ignored any error handling... time to switch to erlang/elixir?
- nerdjon 11mo agoSomething I have been very curious about for some time now. We know the quality of the code is not very high and has a high likelihood of bugs. But, assuming there are not bugs and the code ships. Has there been any study in resource usage creeping up and an impact of this on a whole system. The tests I have done with trying to build things with AI it always seems like there is zero efficiency unless you notice it and can put it in the right direction. I have been curious about the impact this will have on general computing as more low quality code makes it into applications we use every day.
- lherron 11mo agoThey buried the lede. The last half of the article with ways to ground your dev environment to reduce the most common issues should be its own article. (However implementing the proper techniques somewhat obviates the need for CodeRabbit, so guess it’s understandable.)
- windex 11mo agoI think devs have now split into two camps, the kvetchers and the shippers. It's a new tool, it's fresh. Things will work itself out over the next couple of years/months(?). The kvetching helps keep AI research focused on the problem which is good. Meanwhile continue to ship.
- mmastrac 11mo agoIn the pre-AI days I worked on a system like this that was constructed by a high-profile consulting team but continuously lost data and failed to meet even the basic standards. I think I've seen so much rush-shipped slop (before and after) that I'm really anxiously waiting for this bubble to pop. I have yet to be convinced that AI tooling can provide more than 20% or so speedup for an expert developer working in a modern stack/language.
- yomismoaqui 11mo agoAgentic AI coding is a tool, you can use it wrong. To give an example of how to use AI successfully check the following post: https://friendlybit.com/python/writing-justhtml-with-coding-agents/ https://friendlybit.com/python/writing-justhtml-with-coding-...
- joshribakoff 11mo agoThe problem with this black or white framing is it implies competency is a failure. Finding it not useful doesn’t make you wrong, anymore than finding it useful doesn’t make you inexperienced.
- cmiles8 11mo agoThere are certainly some valid criticisms of vibe coding. That said, it’s not like the quality of most code was amazing before AI came along. In fact, most code is generally pretty terrible and took far too long for teams to ship. Many folks would say that if shipping faster allows for faster iterations across an idea then the silly errors are worth it. I’ve certainly seen a sharp increase on execs calling BS on dev teams saying they need months to develop some basic thing.
- tyleo 11mo agoI think you need a balance. I’ve seen products fall apart due to high error rate. I like to think of intentionalists—people who want to understand systems—and vibe coders—people who just want things to work on screen expediently. I think success requires a balance of both. The current problem I see with AI is that it accelerates the vibe part more than the intentionalist part and throws the system out of balance.
- cmiles8 11mo agoDon’t disagree… I think it’s just applying a lot more pressure on dev teams to do things faster though. Devs tend to be expensive and expectations on productivity have increased dramatically. Nobody wants teams to ship crap, but also folks are increasingly questioning why a bit of final polishing takes so long.
- jmathai 11mo agoMore important than code quality is a joint understanding of the business problem and the technical solution for it. Today, that understanding is spread across multiple parties (eng, pm, etc). Code quality can be poor as long as someone understands the tradeoffs for why it's poor.
- coliveira 11mo agoWhen a team says that a "trivial" feature takes months to ship is not because of the complexity of the algorithm. It's because of the infrastructure and coordination work required for the feature to properly work. It is almost aways a failure of the technical infrastructure previously created in the company. An AI will solve the trivial aspects of the problem, not the real problem.
- exitb 11mo ago1.7x does not look that bad? If "AI code" is a broad classification that includes people using bad tools, or not being very skilful operators of said tools, then we can expect this number to meaningfully improve over time.
- speed_spread 11mo agoTell that to your customers. And tell them how much longer the bugs generated by AI will take to fix by humans. Or tell them that you'll never fix the bugs because you're too busy vibe coding new ones.
- exitb 11mo agoI'm not saying bugs aren't a problem. I'm saying that if an emerging, fast improving tech is only slightly behind a human coder now, it seems conceivable that we're not that far off when they reach parity.
- naasking 11mo agoExactly. I'm sure assembly language programmers from the 1980s could easily write code that ran 2x faster than the code produced by compilers of the time, but compilers only got better and eventually assembly language programming became a rare job, and humans can rarely outperform compilers on whole program compilation.
- cryptonym 11mo agoAssembly experts still write code that runs faster than code produced by compilers. Being slower is predictable and solved with better hardware, or just waiting. This is fine for most so we switched to easier or portable languages. Output of the program remains the same. Impact of having 1.7x more bugs is difficult to assess and is not solved that easily. Comparison would work if that was about optimisations: code that is 1.7x slower / memory hungry.
- 11mo ago
- bgwalter 11mo agoThe report is from cortex.io, based on only 50 self-selected responses from "engineering leaders" as well as from idpcon.com, hosted by cortex. All websites involved are vibe coded garbage that use 100% CPU in Firefox.
- neallindsay 11mo ago1.7x more is not the same as 1.7x as many.
- esafak 11mo agoIt's a lost cause. "It's two times faster!"
- deleted 11mo ago[deleted]
- brainless 11mo agoI use LLMs to generate almost all my code. Currently at 40K lines of Rust, backend and a desktop app. I am a senior engineer with almost all my tech career (16 years) in startups. Coding with agents has forced me to generate more tests than we do in most startups, think through more things than we get the time to do in most startups, create more granular tasks and maintain CI/CD (my pipelines are failing and I need to fix them urgently). These are all good things. I have started thinking through my patterns to generate unit tests. I was generating mostly integration or end to end tests before. I started using helping functions in API handlers and have unit tests for helpers, bypassing the API level arguments (so not API mocking or framework test to deal with). I started breaking tasks down into smaller units, so I can pass on to a cheaper model. There are a few patterns in my prompts but nothing that feels out of place. I do not use agents files and no MCPs. All sources here: https://github.com/brainless/nocodo https://github.com/brainless/nocodo (the product is itself going through a pivot so there is that).
- WhyOhWhyQ 11mo agoI see that your release is GPL 3.0. Are you worried about LLM's effectively laundering your source code a year from now? I've become hesitant about releasing source code since LLM's, though I do use Claude heavily while programming to make suggestions and look for issues etc.., but I'd be interested in hearing your perspective.
- brainless 11mo agoI am using GPL 3.0 mostly from a business standpoint, so that I can sell a commercial license to companies that may want to modify the project. In really LLMs will launder everything they can, I am not sure if that can be stopped. So that affects every project. The reason why LLMs are as good is that they train on all code that is available.
- WhyOhWhyQ 11mo agoWhy do companies pay the commercial license instead of just generating a copycat? Genuinely curious.
- naasking 11mo agoIt's totally plausible that AI codegen produces more bugs. It still seems important to familiarize yourself with these tools now though, because that bug count is only ever going to go down. These tools are here to stay.
- oasisaimlessly 11mo agoAre you trying to assure others or reassure yourself?
- esafak 11mo agoIt produces more bugs but the count goes down?!
- naasking 11mo agoDid the models from 2 years ago produce more bugs, fewer bugs or the same bugs as today's models? Do you think next years AI models will produce the same number of bugs, more bugs, or fewer bugs?
- kentm 11mo ago> Did the models from 2 years ago produce more bugs, fewer bugs or the same bugs as today's models? Is anyone actually tracking that with a methodology not prone to fine-tuning? Specifically, I know a lot of the tests have the problem that you can train the AI to pass the test, so a higher score is not indicative of overall higher performance. I'm not actually being rhetorical here to make a point; I'm genuinely interested if anyone has derived a methodology that gives confidence behind these claims. (Aside: Its not a huge stretch to claim that they're getting better, but it mostly seems anecdotal from this point, or using methods that have the above problem I stated)
- geldedus 11mo agoNot for me.
- 827a 11mo agoArchetypes of prompts that I find AI to be quite good at handling: 1. "Write a couple lines or a function that is pretty much what four years ago I would have gone to npm to solve" (e.g. "find the md5 hash of this blob") 2. "Write a function that is highly represented and sampleable in the rest of the project" (e.g. "write a function to query all posts in the database by author_id" (which might include app-specific steps like typing it into a data model)). 3. "Make this isolated needle-in-a-haystack change" (e.g. "change the text of such-and-such tooltip to XYZ") (e.g. "there's a bug with uploading files where we aren't writing the size of the file to the database, fix that") I've found that it can definitely do wider-ranging tasks than that (e.g. implement all API routes for this new data type per this description of the resource type and desired routes); and it can absolutely work. But, the two problems I run into: 1. Because I don't necessarily have a grokable handle on what it generated, I don't have a sense of what its missing and needed follow-on prompts to create. E.g.: I tell it to write an endpoint that allows users to upload files. A few days later, we realize we aren't MD5-hashing the files that got uploaded; there was a field in the database & resource type to store this value, but it didn't pick up on that, and I didn't prompt it to do this; so its not unreasonable. But oftentimes when I'm writing routes by hand, I'm spending so much time in that function body that follow-on requirements naturally occur to me ("Oh that's right, we talked about needing this route available to both of these two permissions, crap let me implement that"). With AI, it finishes so fast that my brain doesn't have time to remember all the requirements. 2. We've tried to mitigate this by pushing more development into the specs and requirements up-front. This is really hard to get humans to do, first of all. But more critically: None of our data supports the hypothesis that this has shortened cycle times. It mostly just trades writing typescript for reading & writing English (which few engineers I've ever worked with are actually all that good at). The engineers still end up needing long cycle times back-and-forth with the AI to get correct results, and long cycle times in review. 3. The more code you ask it to generate, the more vibeslop you get. Deeply-nested try/catch statements with multiple levels of error handling & throwing. Comments everywhere. Reimplementing the same helper functions five times. These things, we have found, raise the cost and lower the reliability & performance of future prompting, and quickly morph parts of the system into a no-man's-land (literally) where only AIs can really make any change; and every change even by the AIs get harder and harder to ship. Our reported customer issues on these parts of the app are significantly higher than others, and our ability to triage these issues is also impacted because we no longer have SMEs that can just brain-triage issues in our CS channels; everything now requires a full engineering cycle, with AI involvement, to solve. Our engineers run the spectrum of "never wanted to touch AI, never did" to "earnestly trying to make it work". Ultimately I think the consensus position is: Its a tool that is nice to have in the toolbox, but any assertion that its going to fundamentally change the profile of work our engineers do, or even seriously impact hiring over the long-term, is outside the realm of foreseeable possibility. The models and surrounding tooling are not improving fast enough.
- kristopherleads 11mo agoI really think the answer here is human-in-the-loop. Too many people are thinking that AI is a full on drop-in replacement for engineers or managers, but ultimately having it be an augment is the magic. I work at FlowFuse so super biased, but that's something I've really enjoyed with our MCP and Expert Assistant - it's built to help you, not to replace you, so you can ask questions, get insights, etc. faster. I suppose the tl;dr is if you're generating bugs in your flow and they make it to prod, it's not a tool problem - it's a cultural one.
- sailfast 11mo agoHow many more bugs does it produce if we use CodeRabbit to review PRs? I assume the number will be less? (Asking seriously and hopefully if the product will help or would’ve caught the bugs, while also pointing out the natural conclusion of the article is to purchase your service :) )
- jampa 11mo agoI've used it in some repos. It doesn't catch all code review issues, especially around product requirements and logic simplification, and occasionally produces irrelevant comments (suggesting a temporary model downgrade). But it's well worth it. It has saved me some considerable time. I let it run first, even before my own final self-review (so if others do the same, the article's data might be biased). It's particularly good at identifying dead code and logical issues. If you tune it with its own custom rules (like Claude.md), you can also cut a lot of noise.
- carra 11mo agoAm I the only one thinking that 1.7x is a very weird way of saying "70% more"? It's even wrong since, like other comments point out, 1.7x MORE would in fact be 2.7 times as much. Which is not what the bug numbers say.
- visarga 11mo agoAI helps ship faster but we need to code 1.7x more tests (with AI) and it's allright
- kkarpkkarp 11mo agoI can't find if they deducted false-positives before they count the results. I've been using CodeRabbit heavily and like any other AI code reviewing tools it was having a lot of them. Like for example: found missing data validation / sanitization reported, only because the code has already been sanitized / validated but this is not visible in the diff. You can tell CodeRabbit he is wrong about this and the tool accepts it then, though.
- TheAnkurTyagi 11mo agocode reviews take way longer now because you gotta actually read through everything instead of trusting the dev knew what they were writing. Its like the AI is great at the happy path but completely misses edge cases or makes weird assumptions about state... The real kicker is when someone copies AI generated code without understanding it and then 3 months later nobody can figure out why production keeps having these random issues. debugging AI slop is its own special hell
- LurkandComment 11mo agoSorry guys I'm having trouble here: Is it: AI builds product faster but with more bugs in production (Adds overall time to acceptable production) or Is is: Al helps us build faster, enough though we have to fix more bugs before production. (over all less time to acceptable production, but specifically fixing bugs takes longer)
- givemeethekeys 11mo agoIt’s AI helps you ship faster. Ignore everything else.
- daheza 11mo agoDepends on the product compliance / healthcare - shipping faster with bugs is not better marketing / non-critical-workflows - sure go for it
- crowbahr 11mo agoHe didn't say AI helps you ship with fewer bugs, just faster. It's faster. It's not better, it's not more stable - the only promise is velocity on initial ship. Oh you want to follow up? Extend it? Reconfigure? Sry can only go faster.
- nphardon 11mo agoShip fast, let customer do the QA, is that the idea?
- jasonlotito 11mo agoThe title (Our new report: AI code creates 1.7x more problems) is wrong. The article says problems, not bug. Problems seems to include formatting or naming issues. I couldn't find 1.7x bugs specifically. I only see 3 mentions of bugs, with no number attached.
- bilater 11mo agofor now
- yonibot 11mo agoHow useful is this metric if we don't know which LLM produced each MR? There could be massive differences in quality between LLMs.
- core-utility 11mo agoAlso, vibe coding has really become a skill in itself. Does it allow the average non-programmer to go to production easily (and likely have bugs/flaws)? Yes. But as someone who used to be a programmer and just fell out of skill with all of it, I've found that what I do still have is a keen sniff test for good vs bad, and can guide the direction and architecture with proper planning. Garbage in, garbage out still applies.
- stevenfoster 11mo agoSomeone solved this 13 years ago: https://github.com/mattdiamond/fuckitjs https://github.com/mattdiamond/fuckitjs
- alexgotoi 11mo agoThe pattern here feels pretty old: every time something shows up that lets people go much faster, we use it to crash harder first. When cars showed up, people didn’t suddenly become more careful because they could now move at 50 km/h instead of 5 – they just plowed into things faster until seatbelts, traffic rules and driver training caught up. LLMs in coding feel similar. They don’t magically remove the need for tests, specs, and review; they just compress the time between “idea” and “running code” so much that all the missing process shows up as outages instead of slow PRs. The risk isn’t “AI writes code”, it’s orgs refusing to slow down long enough to build the equivalent of traffic lights and driver’s ed around it.
- andrenotgiant 11mo ago... _says company selling a service to use AI to find bugs._
- gwbas1c 11mo agoA few days ago I implemented IComparable in .Net / C#, and copilot "read my mind" and made a suggestion. It saved me 2-3 minutes of typing. It took me about an hour of debugging to realize that copilot swapped a variable prefixed with "x" and a different one prefixed with "y". If I wrote it myself, I wouldn't have made that mistake. But, otherwise copilot wrote the exact code that I was going to write.
- scuff3d 11mo agoYesterday we get a "report" from an AI company talking about how much faster you can go with AI. Today we get an AI code review company telling us how much buggier that code is. It's almost like companies that want to sell you shit are only interested in the parts of the story that benefit them.
- gilbetron 11mo ago1.7x more bugs, not great, not terrible
- xnx 11mo ago> What we learned from analyzing hundreds of open-source pull requests. This does not sound like a sufficiently large dataset
- quantum_state 11mo agoVibe coding does not seem to be able to escape from the saying: he who attempts a free lunch will get diarrhea.