13 ms·

We need a clearer framework for AI-assisted contributions to open source

300 points by keybits 11mo ago

insane_dreamer 11mo ago
related discussion: https://news.ycombinator.com/item?id=45330378 https://news.ycombinator.com/item?id=45330378
deleted 11mo ago
[deleted]
r4victor 11mo ago
So far I prefer the Hashimoto's solution to this that "AI tooling must be disclosed for contributions": https://news.ycombinator.com/item?id=44976568 https://news.ycombinator.com/item?id=44976568 I use it like this: If a PR is LLM-generated, you as a maintainer either merge it if it's good or close if it's not. If it's human-written, you may spend some time reviewing the code and iterating on the PR as you used to. Saves your time without discarding LLM PRs completely.
- mexicocitinluez 11mo ago
  But what does LLM-generated mean? What if I use CoPilot for completions? Is that considered "AI generated"? What if I grab the code from Claude, and update greater than 50%. Am I now taking ownership of it as my code? It's like the ship of theseus
chrischen 11mo ago
Maybe what we need is AI based code review.
lccerina 11mo ago
I have a framework: don't use it, if you never used it don't start using it, public shame people, stop talking about it. Slow down. Think long and deep about your problems. Write less code. There is NOTHING inevitable about this stuff.
- SideburnsOfDoom 11mo ago
  Indeed. "No." is perfectly clear.
bradfa 11mo ago
The Fedora policy on AI-assisted contributions seems very reasonable: https://communityblog.fedoraproject.org/council-policy-proposal-policy-on-ai-assisted-contributions/ https://communityblog.fedoraproject.org/council-policy-propo...
colesantiago 11mo ago
I wouldn't call it "vibe coded slop" the models are getting way better and I can work with my engineers a lot faster. I am the founder and a product person so it helps in reducing the number of needed engineers at my business. We are currently doing $2.5M ARR and the engineers aren't complaining, in fact it is the opposite, they are actually more productive. We still prioritize architecture planning, testing and having a CI, but code is getting less and less important in our team, so we don't need many engineers.
- pards 11mo ago
  > code is getting less and less important in our team, so we don't need many engineers. That's a bit reductive. Programmers write code; engineers build systems. I'd argue that you still need engineers for architecture, system design, protocol design, API design, tech stack evaluation & selection, rollout strategies, etc, and most of this has to be unambiguously documented in a format LLMs can understand. While I agree that the value of code has decreased now that we can generate and regenerate code from specs, we still need a substantial number of experienced engineers to curate all the specs and inputs that we feed into LLMs.
  - HPsquared 11mo ago
    Maybe the code itself is less important now, relative to the specification.
  - didericis 11mo ago
    > we can generate and regenerate code from specs We can (unreliably) write more code in natural english now. At its core it’s the same thing: detailed instructions telling the computer what it should do.
- oompydoompy74 11mo ago
  [flagged]
  - colesantiago 11mo ago
jmpeax 11mo ago
[flagged]
Bengalilol 11mo ago
Shouldn't there be guidelines for open source projects where it is clearly stipulated that code submitted for review must follow the project's code format and conventions?
- c0wb0yc0d3r 11mo ago
  This is the thought that I always have whenever I see the mention of coding standards. Not only should there be standards they should be enforced by tooling. Now that being said a person should feel free to do what they want with their code. It’s somewhat tough to justify the work of setting up infrastructure to do that on small projects, but AI PRs aren’t likely a big issue fit small projects.
- portaouflop 11mo ago
  In a perfect world people would read and understand contribution guidelines before opening a PR or issue. Alas…
- deadbunny 11mo ago
  As if people read guidelines. Sure they're good to have so you can point to them when people violate them but people (in general) will not by default read them before contributing.
- kasey_junk 11mo ago
  I’ve found LLM coding agents to be quite good at writing linters…
- isaacremuant 11mo ago
  Code format and conventions are not the problem. It's the complexity of the change without testing, thinking, or otherwise having ownership of your PR. Some people will absolutely just run something, let the AI work like a wizard and push it in hopes of getting an "open source contribution". They need to understand due diligence and reduce the overhead of maintainers so that maintainers don't review things before it's really needed. It's a hard balance to strike, because you do want to make it easy on new contributors, but this is a great conversation to have.
softwaredoug 11mo ago
Anyone else feel like we're cresting the LLM coding hype curve? Like a recognition that there's value there, but we're passing the frothing-at-the-mouth stage of replacing all software engineers?
- alwa 11mo ago
  It feels that way to me, too—starting to feel closer to maturity. Like Mr. Saffron here, saying “go ham with the AI for prototyping, just communicate that as a demo/branch/video instead of a PR.” It feels like people and projects are moving from a pure “get that slop out of here” attitude toward more nuance, more confidence articulating how to integrate the valuable stuff while excluding the lazy stuff.
  - user-the-name 11mo ago
    [dead]
- deepsquirrelnet 11mo ago
  I think that happened when gpt5 was released and pierced OpenAIs veil. While not a bad model, we found out exactly what Mr. Altman’s words are worth.
- jandrese 11mo ago
  When people talk about the “AI bubble popping” this is what they mean. It is clear that AI will remain useful, but the “singularity is nigh” hype is faltering and the company valuations based on perpetual exponential improvement are just not realistic. Worse, the marginal improvements are coming at ever higher resource requirements with each generation, which puts a soft cap on how good an AI can be and still be economical to run.
  - NewsaHackO 11mo ago
    >When people talk about the “AI bubble popping” this is what they mean. You mean what they have conceded so far to be what they mean. Every new model they start to see that they have to give up a little more.
Toby1VC 11mo ago
Nice jewish word mostly meant to mock. Why would I care what a plugin that I don't even see in use has to say to my face (since I had to read this with all the interpretation potential and receptiveness available). The same kind of inserted judgment that lingers similar to "Yes, I will judge you if you use AI".
- softskunk 11mo ago
  There’s nothing wrong with judgment. Judging someone’s character based on whether they use generative “AI” is a valid practice. You may not like being judged, but that’s another matter entirely.
  - Toby1VC 11mo ago
    >There’s nothing wrong with judgment. Judging someone’s character based on whether they use generative “AI” is a valid practice. You may not like being judged, but that’s another matter entirely. You and I know that using AI is a metric to consider when judging ability and quality. The difference is that it's not judgment but a broadcast, announcement. In this case a snotty one from Discourse. I mention that it lingers because I think that is a real psychological effect that happens. Small announcements like this carry over into the future and flood any evaluation of yourself which can be described as torture and sabotage since it has an effect on decisions you make sometimes destroying things.
    - softskunk 11mo ago
      So people will be more likely to think twice before using Cursor, Copilot et al? Good. I think they should. Your comparison to torture and sabotage is unfounded to the point of being simply bizarre.
      - 1gn15 11mo ago
        I think humans should use AI as much as possible. Just putting that thought out there, so people are less likely to think twice.
darkwater 11mo ago
The title doesn't make justice to the content. I really liked the paragraph about LLMs being "alien intelligence" > Many engineers I know fall into 2 camps, either the camp that find the new class of LLMs intelligent, groundbreaking and shockingly good. In the other camp are engineers that think of all LLM generated content as “the emperor’s new clothes”, the code they generate is “naked”, fundamentally flawed and poison. I like to think of the new systems as neither. I like to think about the new class of intelligence as “Alien Intelligence”. It is both shockingly good and shockingly terrible at the exact same time. Framing LLMs as “Super competent interns” or some other type of human analogy is incorrect. These systems are aliens and the sooner we accept this the sooner we will be able to navigate the complexity that injecting alien intelligence into our engineering process leads to. It's a similitude I find compelling. The way they produce code and the way you have to interact with them really feels "alien", and when you start humanizing them, you get emotions when interacting with it and that's not correct. I mean, I do get emotional and frustrated even when good old deterministic programs misbehaved and there was some bug to find and squash or work-around, but the LLM interactions can bring the game to a complete new level. So, we need to remember they are "alien".
- andai 11mo ago
  Some movements expected alien intelligence to arrive in the early 2020s. They might have been on the mark after all ;)
- wat10000 11mo ago
  I’m reminded of Dijkstra: “The question of whether machines can think is about as relevant as the question of whether submarines can swim.” These new submarines are a lot closer to human swimming than the old ones were, but they’re still very different.
- keiferski 11mo ago
  This is why at a fundamental level, the concept of AGI doesn't make a lot of sense. You can't measure machine intelligence by comparing it to a human's. That doesn't mean machines can't be intelligent...but rather that the measuring stick cannot be an abstracted human being. It can only be the accumulation of specific tasks.
jamesbelchamber 11mo ago
> “I am closing this but this is interesting, head over to our forum/issues to discuss” I really like the way Discourse uses "levels" to slowly open up features as new people interact with the community, and I wonder if GitHub could build in a way of allowing people to only be able to open PRs after a certain amount of interaction, too (for example, you can only raise a large PR if you have spent enough time raising small PRs). This could of course be abused and/or lead to unintended restrictions (e.g. a small change in lots of places), but that's also true of Discourse and it seems to work pretty well regardless.
- warmwaffles 11mo ago
  Mailing lists are used as a filter to raise the barrier to entry to prevent people from contributing code that they have no intention of maintaining and leaving that to the project owners. Github for better or worse has made the barrier to entry much much lower and significantly easier for people to propose changes and then disappear.
mattlondon 11mo ago
The way we do it is to use AI to review the PR before a human reviewer sees it. Obvious errors, non-consistent patterns, weirdness etc is flagged before it goes any further. "Vibe coded" slop usually gets caught, but "vibe engineered" surgical changes that adhere to common patterns and standards and have tests etc get to be seen by a real live human for their normal review. It's not rocket science.
- franktankbank 11mo ago
  Do you work at a profitable company?
andai 11mo ago
>That said, there is a trend among many developers of banning AI. Some go so far as to say “AI not welcome here” find another project. >This feels extremely counterproductive and fundamentally unenforceable to me. Much of the code AI generates is indistinguishable from human code anyway. You can usually tell a prototype that is pretending to be a human PR, but a real PR a human makes with AI assistance can be indistinguishable. Isn't that exactly the point? Doesn't this achieve exactly what the whole article is arguing for? A hard "No AI" rule filters out all the slop, and all the actually good stuff (which may or may not have been made with AI) makes it in. When the AI assisted code is indistinguishable from human code, that's mission accomplished, yeah? Although I can see two counterarguments. First, it might just be Covert Slop. Slop that goes under the radar. And second, there might be a lot of baby thrown out with that bathwater. Stuff that was made in conjunction with AI, contains a lot of "obviously AI", but a human did indeed put in the work to review it. I guess the problem is there's no way of knowing that? Is there a Proof of Work for code review? (And a proof of competence, to boot?)
- jrochkind1 11mo ago
  Well, but why not instead of asking/accepting people will lie undetectably when you say "No AI" and it's okay you're fine with lying, just say instead "Only AI when you spend the time to turn it into a real reviewed PR, which looks like X, Y, and Z", giving some actual tips on how to use AI acceptably. Which is what OP suggests.
- felipeerias 11mo ago
  Personally, I would not contribute to a project that forced me to lie. And from the point of view of the maintainers, it seems a terrible idea to set up rules with the expectation that they will be broken.
  - 1gn15 11mo ago
    I know, right. It's like setting up rules saying "you can't use IDE autocomplete" or "you can't code with background music because that distracts you from bugs". If the final result is indistinguishable, I find it perfectly acceptable to lie. Rules are just words, after all, especially if it's completely unenforceable. Or, the decentralized, no rulers solution: clone the repo on your own website and put your patches there instead.
jrochkind1 11mo ago
Essay is way more interesting than the title, which doesn't actually capture it.
- dang 11mo ago
  Thanks for pointing this out—it made me take the time to find a sentence in the article body that could serve as a less baity title. From https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html: "Please use the original title, unless it is misleading or linkbait" (note that word unless)
- jamesbelchamber 11mo ago
  The title seems perfectly engineered to get upvotes from people who don't read the article, which puts the article in front of more people who would actually read it (which is good because the article is, as you say, very interesting and worth sharing). I don't like it but I can hardly blame them.
  - jrochkind1 11mo ago
    Agreed. Sometimes such rage/engagement-bait titles get changed on HN, but it's risky to do as a submitter cause it's unclear when you are "allowed" to change the title. And I suppose if you want upvotes, why would you change the ragebait title? Usually engagement-bait titles are cover for uninteresting articles, but yeah in this case it's way more interesting than the title to me anyway. i guess it makes it even more obvious when people are discussing the title instead of the actual piece, which is routine on HN but not always obvious! Although to be fair, the title describes one part of the piece, sure. the part with the least original insight.
jcgrillo 11mo ago
> Some go so far as to say “AI not welcome here” find another project. This feels extremely counterproductive and fundamentally unenforceable to me. But it's trivially enforceable. Accept PRs from unverified contributors, look at them for inspiration if you like, but don't ever merge one. It's probably not a satisfying answer, but if you want or need to ensure your project hasn't been infected by AI generated code you need to only accept contributions from people you know and trust.
- anon3242 11mo ago
  This is sad. The barrier of entry will be raised extremely high, maybe even requiring some real world personal connections to the maintainer.
  - jcgrillo 11mo ago
    Real world personal connections are how we establish trust. At some point you have to be able to trust the people you're collaborating with.
lapcat 11mo ago
> That said it is a living demo that can help make an idea feel more real. It is also enormously fun. Think of it as a delightful movie set. [pedantry] It bothers me that the photo for "think of prototype PRs as movie sets" is clearly not a movie set but rather the set of the TV show Seinfeld. Anyone who watched the show would immediately recognize Jerry's apartment.
- DerThorsten 11mo ago
  Its not the set of the TV show I beliefe, but a recreation. https://nypost.com/2015/06/23/you-can-now-visit-the-iconic-seinfeld-set-in-new-york/ https://nypost.com/2015/06/23/you-can-now-visit-the-iconic-s... It looks a bit different wrt. the stuff on the fridge and the items in the cupboard
  - lapcat 11mo ago
    I'm not sure what you mean. Those two photos are very different. The floors are entirely different, the tables are entirely different, one of the chairs/couches is different, even the intercom and light switch are different. In any case, though, neither one is a movie set.
    - DerThorsten 11mo ago
      I think we agree, it looks like the seinfeld set, but it not the orginal set, just something looking very similar.
      - deleted 11mo ago
        [deleted]
  - throwawaysoxjje 11mo ago
    It’s this Sony Picture Studios recreation actually: https://www.reddit.com/r/seinfeld/comments/yfbmn2/sony_pictures_studios_museum_has_a_recreation_of/ https://www.reddit.com/r/seinfeld/comments/yfbmn2/sony_pictu...
bloppe 11mo ago
Maybe we need open source credit scores. PRs from talented engineers with proven track records of high quality contributions would be presumed good enough for review. Unknown, newer contributors could have a size limit on their PRs, with massive PRs rejected automatically.
- selfhoster11 11mo ago
  We don't need more KYC, no.
  - javier123454321 11mo ago
    Reputation building is not kyc. It is actually the thing that enables anonymization to work in a more sophisticated way.
- mfenniak 11mo ago
  The Forgejo project has been gently trying to redirect new contributors into fixing bugs before trying to jump into the project to implement big features (https://codeberg.org/forgejo/discussions/issues/337 https://codeberg.org/forgejo/discussions/issues/337). This allows a new contributor to get into the community, get used to working with the codebase, do something of clear value... but for the project a lot of it is about establishing reputation. Will the contributor respond to code-review feedback? Will they follow-up on work? Will they work within the code-of-conduct and learn the contributor guidelines? All great things to figure out on small bugs, rather than after the contributor has done significant feature work.
prymitive 11mo ago
The problem with AI isn’t new, it’s the same old problem with technology: computers don’t do what you want, only what you tell them. A lot of PRs can be judged by how well they are described and justified, it’s because the code itself isn’t that important, it’s the problem that you are solving with it that is. People are often great at defining problems, AIs less so IMHO. Partially because they simply have no understanding, partially because they over explain everything to a point where you just stop reading, and so you never get to the core of the problem. And even if you do there’s a good chance AI misunderstood the problem and the solution is wrong in a some more or less subtle way. This is further made worse by the sheer overconfidence of AI output, which quickly erodes any trust that they did understand the problem.
Lerc 11mo ago
It is possible that some projects could benefit from triage volunteers? There are plenty of open source projects where it is difficult to get up to speed with the intricacies of the architecture that limits the ability of talented coders to contribute on a small scale. There might be merit in having a channel for AI contributions that casual helpers can assess to see if they pass a minimum threshold before passing on to a project maintainer to assess how the change works within the context of the overall architecture. It would also be fascinating to see how good an AI would be at assessing the quality of a set of AI generated changes absent the instructions that generated them. They may not be able to clearly identify whether the change would work, but can they at least rank a collection of submissions to select the ones most worth looking at? At the very least the pile of PRs count as data of things that people wanted to do, even if the code was completely unusable, placing it into a pile somewhere might be minable for the intentions of erstwhile contributors.
dearilos 11mo ago
We’re fixing this slop problem - engineers write rules that are enforced on PRs. Fixes the problem pretty well so far.
jcgrillo 11mo ago
I guess the main question I'm left with after reading this is "what good is a prototype, then?" In a few of the companies I've worked at there was a quarterly or biannual ritual called "hack week" or "innovation week" or "hackathon" where engineers form small teams and try to bang out a pet project super fast. Sometimes these projects get management's attention, and get "promoted" to a product or feature. Having worked on a few of these "promoted" projects, to the last they were unmitigated disasters. See, "innovation" doesn't come from a single junior engineer's 2AM beer and pizza fueled fever dream. And when you make the mistake of believing otherwise, what seemed like some bright spark's clever little dream turns into a nightmare right quick. The best thing you can do with a prototype is delete it.
- corytheboyd 11mo ago
  Completely agree, I hate the “hackathon” for so many reasons, guess I’ll vent here too. All of this from the perspective of one frustrated software engineer in web tech. First of all, if you want innovation, why are you forcing it into a single week? You very likely have smart people with very good ideas, but they’re held back by your number-driven bullshit. These orgs actively kill innovation by reducing talent to quantifiable rows of data. A product hobbled together from shit prototype code very obviously stands out. It has various pages that don’t quite look/work the same, Cross-functional things that “work everywhere else” don’t in some parts. It rewards only the people who make good presentations, or pick the “current hype thing” to work on. Occasionally something good that addresses real problems is at least mentioned but the hype thing will always win (if judged by your SLT) Shame on you if the slop prototype is handed off to some other team than the hackathon presenters. Presenters take all the promotion points, then implementers have to sort out a bunch of bullshit code, very likely being told to just ship the prototype “it works you idiots, I saw it in the demo, just ship it.” Which is so incredibly short sighted. I think the depressing truth is your executives know it’s all hobbled together bullshit, but that it will sell anyway, so why invest time making it actually good? They all have their golden parachutes, what do they care about the suckers stuck on-call for the house-of-cards they were forced to build, despite possessing the talent to make it stable? All this stupidity happens over and over again, not because it is wise, or even the best way to do this, the truth is just a flaccid “eh, it’ll work though, fuck it, let’s get paid.”
andy99 11mo ago
This is a problem everywhere now, and not just in code. It now takes zero effort to produce something, whether code or a work plan or “deep research” and then lob it over the fence, expecting people to review and act upon it. It’s an extension of the asymmetric bullshit principle IMO, and I think now all workplaces / projects need norms about this.
- solotronics 11mo ago
  This problem statement was actually where the idea for Proof of Work (aka mining) in bitcoin came from. It evolved out of the idea of requiring a computational proof of work for sending an email via cypherpunk remailers as a way of fighting spam. The idea being only a legitimate or determined sender would put in the "proof of work" to use the remailer. I wonder how it would look if open source projects required $5 to submit a PR or ticket and then paid out a bounty to the successful or at least reasonable PRs. Essentially a "paid proof of legitimacy".
  - strogonoff 11mo ago
    The parallel between PoW and barriers to entry many communities (be it Wikipedia editors or open-source contributors) use to sustain themselves seems apt. Unfortunately, there is no community equivalent of PoS—the only alternative is introducing different barriers, like ID verification, payment, in-person interviews, private invite system, etc., which often conflict with the nature of anonymous volunteer communities. Such communities are perhaps one of the greatest things the Web has given us, and it is sad to see them struggle. (I can imagine LLM operators jumping on the opportunity to sell some of these new barriers, to profit from selling both the problematic product and a product to work around those problems.)
    - Frieren 11mo ago
      > (I can imagine LLM operators jumping on the opportunity to sell some of these new barriers, to profit from selling both the problematic product and a product to work around those problems.) That is their business model. Use AI to create posts in LinkedIn, mails in a corporate environment, etc. And then use AI to summarize all that text. AI creates a problem and then offers a solution. My current approach is to look at new sources lie The Guardian, Le Monde, AP news, etc. I know that they put the work, sadly places like Reddit and such are just becoming forums that discuss garbage news with bot comments. (I could use AI to identify non-bot comments and news sources, but it does not really work even if it says that it does, and I should not have to do that in the first place either).
jongjong 11mo ago
2 months ago, after I started using Claude Code on my side project, within the space of days, I went from not allowing a single line of AI code into my codebase to almost 100% AI-written code. It basically codes in my exact style and I know ahead of time what code I expect to see so reviewing is really easy. I cannot justify to myself writing code by hand when there is literally no difference in the output from how I would have done it myself. It might as well be reading my mind, that's what it feels like. For me, vibe coding is essentially a 5x speed increase with no downside. I cannot believe how fast I can churn out features. All the stuff I used to type out by hand now seems impossibly boring. I just don't have the patience to hand-code anymore. I've stuck to vanilla JavaScript because I don't have the patience to wait for the TypeScript transpiler. TS iteration speed is too slow. By the time it finishes transpiling, I can't even remember what I was trying to do. So you bet I don't have the patience to write by hand now. I really need momentum (fast iteration speed) when I code and LLMs provide that.
- listenallyall 11mo ago
  I dont mean to question you personally, after all this is the internet, but comments like yours do make the reader think, if he has 5x'ed his coding, was he any good to begin with? I guess what I'm saying is, without knowing your baseline skill level, I dont know whether to be impressed by your story. Have you become a super-programmer, or is it just cleaning up stupid stuff that you shouldn't have been doing in the first place? If someone is already a clear-headed, efficient, experienced programmer, would that person be seeing anywhere near the benefits you have? Again, this isn't a slight on you personally, it's just, a reader doesnt really know how to place your experience into context.
  - jongjong 11mo ago
    I have a computer science degree and quite experienced. In the last 10 years, I've created a few side projects and also worked as a senior engineer professionally. I was engineer #3 at what later became the #1 biggest Maths e-learning platform in my country. I later built a popular distributed open source WebSocket pub/sub system which auto-scales on Kubernetes (with load balancing and sharding). I lead the development of the P2P layer of a $300 million+ market cap crypto project in Germany. I wrote a multi-chain Decentralized Exchange from scratch along with a minimalist PoS blockchain with support for instant finality. Then I wrote a multi-tenant no-code serverless platform to build/host any kind of app. Then (in part to demonstrate the search and indexing capabilities of the serverless no-code platform), I used this platform to build a multi-tenant people/company search platform which allows users to crawl the web and tag data using vector embeddings using a BFS algorithm (current side project). I'm still running all these projects in parallel because they require very little maintenance. My blockchain nodes have been running for 4 years straight. I restarted a node twice in 4 years due to infrastructure issue so very reliable. Obviously, I suck at business and marketing. I only had one relatively financially successful product (my open source project, ironically) but I'm definitely able to build features quickly and in a stable way according to spec.
quxbar 11mo ago
If one claims to be able to write good code with LLMs, it should just as easy to write comprehensive e2e tests. If you don't hold your code to a high testing standard than you were always going off 'vibes' whether they were from a silicon neural network or your human meatware biases.
- hoherd 11mo ago
  Reviewing test code is arguably harder than reviewing implementation code because tests are enumerated success and failure scenarios. Some times the LOC of the tests is an order of magnitude larger than the implementation code. The biggest place I've seen AI created code with tests produce a false positive is when a specific feature is being tested, but the test case overwrites a global data structure. Fixing the test reveals the implementation to be flawed. Now imagine you get rewarded for shipping new features a test code, but are derided for refactoring old code. The person who goes to fix the AI slop is frowned upon while the AI slop driver gets recognition for being a great coder. This dynamic caused by AI coding tools is creating perverse workplace incentives.
specproc 11mo ago
A bit of a brutal title for what's a pretty constructive and reasonable article. I like the core: AI-produced contributions are prototypes, belong in branches, and require transparency and commitment as a path to being merged.
ninju 11mo ago
Well...just have AI review the PR to have it highlight the slop /s
gordonhart 11mo ago
> As engineers it is our role to properly label our changes. I've found myself wanting line-level blame for LLMs. If my teammate committed something that was written directly by Claude Code, it's more useful to me to know that than to have the blame assigned to the human through the squash+merge PR process. Ultimately somebody needs to be on the hook. But if my teammate doesn't understand it any better than I do, I'd rather that be explicit and avoid the dance of "you committed it, therefore you own it," which is better in principle than in practice IMO.
- andrewflnr 11mo ago
  If your teammate doesn't understand it, they shouldn't have committed it. This isn't a "dance", it's basic responsibility for your actions.
gwbas1c 11mo ago
> You can usually tell a prototype that is pretending to be a human PR, but a real PR a human makes with AI assistance can be indistinguishable. A couple of weeks ago I needed to stuff some binary data into a string, in a way where it wouldn't be corrupted by whitespace changes. I wrote some Rust code to generate the string. After I typed "}" to end the method: 1: Copilot suggested a 100% correct method to parse the string back to binary data, and then 2: Suggested a 100% correct unit test. I read both methods, and they were identical to what I would write. It was as if Copilot could read my brain. BUT: If I relied on Copilot to come up with the serialization form, or even know that it needed to pick something that wouldn't be corrupted by whitespace, it might have picked something completely wrong, that didn't meet what the project needed.
anal_reactor 11mo ago
An idea occurred to me. What if: 1. Someone raises a PR 2. Entry-level maintainers skim through it and either reject or pass higher up 3. If the PR has sufficient quality, the PR gets reviewed by someone who actually has merge permissions
sams99 11mo ago
Author here, thanks heaps for the discussion, I replied to a few of the points in my blog comments: https://discuss.samsaffron.com/t/your-vibe-coded-slop-pr-is-not-welcome/2492/4?u=sam https://discuss.samsaffron.com/t/your-vibe-coded-slop-pr-is-...
vrighter 11mo ago
Simple framework: You must not attempt to offload the actual thinking part to someone else. You should be able to answer any reasonable question about the code you submitted. If you can't you should be banned from any further contributions, permanently.

300 points by keybits 11mo ago

insane_dreamer 11mo ago

related discussion: https://news.ycombinator.com/item?id=45330378 https://news.ycombinator.com/item?id=45330378

deleted 11mo ago

[deleted]

r4victor 11mo ago

So far I prefer the Hashimoto's solution to this that "AI tooling must be disclosed for contributions": https://news.ycombinator.com/item?id=44976568 https://news.ycombinator.com/item?id=44976568 I use it like this: If a PR is LLM-generated, you as a maintainer either merge it if it's good or close if it's not. If it's human-written, you may spend some time reviewing the code and iterating on the PR as you used to. Saves your time without discarding LLM PRs completely.

mexicocitinluez 11mo ago
But what does LLM-generated mean? What if I use CoPilot for completions? Is that considered "AI generated"? What if I grab the code from Claude, and update greater than 50%. Am I now taking ownership of it as my code? It's like the ship of theseus

chrischen 11mo ago

Maybe what we need is AI based code review.

lccerina 11mo ago

I have a framework: don't use it, if you never used it don't start using it, public shame people, stop talking about it. Slow down. Think long and deep about your problems. Write less code. There is NOTHING inevitable about this stuff.

SideburnsOfDoom 11mo ago
Indeed. "No." is perfectly clear.

bradfa 11mo ago

The Fedora policy on AI-assisted contributions seems very reasonable: https://communityblog.fedoraproject.org/council-policy-proposal-policy-on-ai-assisted-contributions/ https://communityblog.fedoraproject.org/council-policy-propo...

colesantiago 11mo ago

I wouldn't call it "vibe coded slop" the models are getting way better and I can work with my engineers a lot faster. I am the founder and a product person so it helps in reducing the number of needed engineers at my business. We are currently doing $2.5M ARR and the engineers aren't complaining, in fact it is the opposite, they are actually more productive. We still prioritize architecture planning, testing and having a CI, but code is getting less and less important in our team, so we don't need many engineers.

pards 11mo ago
> code is getting less and less important in our team, so we don't need many engineers. That's a bit reductive. Programmers write code; engineers build systems. I'd argue that you still need engineers for architecture, system design, protocol design, API design, tech stack evaluation & selection, rollout strategies, etc, and most of this has to be unambiguously documented in a format LLMs can understand. While I agree that the value of code has decreased now that we can generate and regenerate code from specs, we still need a substantial number of experienced engineers to curate all the specs and inputs that we feed into LLMs.
- HPsquared 11mo ago
  Maybe the code itself is less important now, relative to the specification.
- didericis 11mo ago
  > we can generate and regenerate code from specs We can (unreliably) write more code in natural english now. At its core it’s the same thing: detailed instructions telling the computer what it should do.
oompydoompy74 11mo ago
[flagged]
- colesantiago 11mo ago

jmpeax 11mo ago

[flagged]

Bengalilol 11mo ago

Shouldn't there be guidelines for open source projects where it is clearly stipulated that code submitted for review must follow the project's code format and conventions?

c0wb0yc0d3r 11mo ago
This is the thought that I always have whenever I see the mention of coding standards. Not only should there be standards they should be enforced by tooling. Now that being said a person should feel free to do what they want with their code. It’s somewhat tough to justify the work of setting up infrastructure to do that on small projects, but AI PRs aren’t likely a big issue fit small projects.
portaouflop 11mo ago
In a perfect world people would read and understand contribution guidelines before opening a PR or issue. Alas…
deadbunny 11mo ago
As if people read guidelines. Sure they're good to have so you can point to them when people violate them but people (in general) will not by default read them before contributing.
kasey_junk 11mo ago
I’ve found LLM coding agents to be quite good at writing linters…
isaacremuant 11mo ago
Code format and conventions are not the problem. It's the complexity of the change without testing, thinking, or otherwise having ownership of your PR. Some people will absolutely just run something, let the AI work like a wizard and push it in hopes of getting an "open source contribution". They need to understand due diligence and reduce the overhead of maintainers so that maintainers don't review things before it's really needed. It's a hard balance to strike, because you do want to make it easy on new contributors, but this is a great conversation to have.

softwaredoug 11mo ago

Anyone else feel like we're cresting the LLM coding hype curve? Like a recognition that there's value there, but we're passing the frothing-at-the-mouth stage of replacing all software engineers?

alwa 11mo ago
It feels that way to me, too—starting to feel closer to maturity. Like Mr. Saffron here, saying “go ham with the AI for prototyping, just communicate that as a demo/branch/video instead of a PR.” It feels like people and projects are moving from a pure “get that slop out of here” attitude toward more nuance, more confidence articulating how to integrate the valuable stuff while excluding the lazy stuff.
- user-the-name 11mo ago
  [dead]
deepsquirrelnet 11mo ago
I think that happened when gpt5 was released and pierced OpenAIs veil. While not a bad model, we found out exactly what Mr. Altman’s words are worth.
jandrese 11mo ago
When people talk about the “AI bubble popping” this is what they mean. It is clear that AI will remain useful, but the “singularity is nigh” hype is faltering and the company valuations based on perpetual exponential improvement are just not realistic. Worse, the marginal improvements are coming at ever higher resource requirements with each generation, which puts a soft cap on how good an AI can be and still be economical to run.
- NewsaHackO 11mo ago
  >When people talk about the “AI bubble popping” this is what they mean. You mean what they have conceded so far to be what they mean. Every new model they start to see that they have to give up a little more.

Toby1VC 11mo ago

Nice jewish word mostly meant to mock. Why would I care what a plugin that I don't even see in use has to say to my face (since I had to read this with all the interpretation potential and receptiveness available). The same kind of inserted judgment that lingers similar to "Yes, I will judge you if you use AI".

softskunk 11mo ago
There’s nothing wrong with judgment. Judging someone’s character based on whether they use generative “AI” is a valid practice. You may not like being judged, but that’s another matter entirely.
- Toby1VC 11mo ago
  >There’s nothing wrong with judgment. Judging someone’s character based on whether they use generative “AI” is a valid practice. You may not like being judged, but that’s another matter entirely. You and I know that using AI is a metric to consider when judging ability and quality. The difference is that it's not judgment but a broadcast, announcement. In this case a snotty one from Discourse. I mention that it lingers because I think that is a real psychological effect that happens. Small announcements like this carry over into the future and flood any evaluation of yourself which can be described as torture and sabotage since it has an effect on decisions you make sometimes destroying things.
  - softskunk 11mo ago
    So people will be more likely to think twice before using Cursor, Copilot et al? Good. I think they should. Your comparison to torture and sabotage is unfounded to the point of being simply bizarre.
    - 1gn15 11mo ago
      I think humans should use AI as much as possible. Just putting that thought out there, so people are less likely to think twice.

darkwater 11mo ago

The title doesn't make justice to the content. I really liked the paragraph about LLMs being "alien intelligence" > Many engineers I know fall into 2 camps, either the camp that find the new class of LLMs intelligent, groundbreaking and shockingly good. In the other camp are engineers that think of all LLM generated content as “the emperor’s new clothes”, the code they generate is “naked”, fundamentally flawed and poison. I like to think of the new systems as neither. I like to think about the new class of intelligence as “Alien Intelligence”. It is both shockingly good and shockingly terrible at the exact same time. Framing LLMs as “Super competent interns” or some other type of human analogy is incorrect. These systems are aliens and the sooner we accept this the sooner we will be able to navigate the complexity that injecting alien intelligence into our engineering process leads to. It's a similitude I find compelling. The way they produce code and the way you have to interact with them really feels "alien", and when you start humanizing them, you get emotions when interacting with it and that's not correct. I mean, I do get emotional and frustrated even when good old deterministic programs misbehaved and there was some bug to find and squash or work-around, but the LLM interactions can bring the game to a complete new level. So, we need to remember they are "alien".

andai 11mo ago
Some movements expected alien intelligence to arrive in the early 2020s. They might have been on the mark after all ;)
wat10000 11mo ago
I’m reminded of Dijkstra: “The question of whether machines can think is about as relevant as the question of whether submarines can swim.” These new submarines are a lot closer to human swimming than the old ones were, but they’re still very different.
keiferski 11mo ago
This is why at a fundamental level, the concept of AGI doesn't make a lot of sense. You can't measure machine intelligence by comparing it to a human's. That doesn't mean machines can't be intelligent...but rather that the measuring stick cannot be an abstracted human being. It can only be the accumulation of specific tasks.

jamesbelchamber 11mo ago

> “I am closing this but this is interesting, head over to our forum/issues to discuss” I really like the way Discourse uses "levels" to slowly open up features as new people interact with the community, and I wonder if GitHub could build in a way of allowing people to only be able to open PRs after a certain amount of interaction, too (for example, you can only raise a large PR if you have spent enough time raising small PRs). This could of course be abused and/or lead to unintended restrictions (e.g. a small change in lots of places), but that's also true of Discourse and it seems to work pretty well regardless.

warmwaffles 11mo ago
Mailing lists are used as a filter to raise the barrier to entry to prevent people from contributing code that they have no intention of maintaining and leaving that to the project owners. Github for better or worse has made the barrier to entry much much lower and significantly easier for people to propose changes and then disappear.

mattlondon 11mo ago

The way we do it is to use AI to review the PR before a human reviewer sees it. Obvious errors, non-consistent patterns, weirdness etc is flagged before it goes any further. "Vibe coded" slop usually gets caught, but "vibe engineered" surgical changes that adhere to common patterns and standards and have tests etc get to be seen by a real live human for their normal review. It's not rocket science.

franktankbank 11mo ago
Do you work at a profitable company?

andai 11mo ago

>That said, there is a trend among many developers of banning AI. Some go so far as to say “AI not welcome here” find another project. >This feels extremely counterproductive and fundamentally unenforceable to me. Much of the code AI generates is indistinguishable from human code anyway. You can usually tell a prototype that is pretending to be a human PR, but a real PR a human makes with AI assistance can be indistinguishable. Isn't that exactly the point? Doesn't this achieve exactly what the whole article is arguing for? A hard "No AI" rule filters out all the slop, and all the actually good stuff (which may or may not have been made with AI) makes it in. When the AI assisted code is indistinguishable from human code, that's mission accomplished, yeah? Although I can see two counterarguments. First, it might just be Covert Slop. Slop that goes under the radar. And second, there might be a lot of baby thrown out with that bathwater. Stuff that was made in conjunction with AI, contains a lot of "obviously AI", but a human did indeed put in the work to review it. I guess the problem is there's no way of knowing that? Is there a Proof of Work for code review? (And a proof of competence, to boot?)

jrochkind1 11mo ago
Well, but why not instead of asking/accepting people will lie undetectably when you say "No AI" and it's okay you're fine with lying, just say instead "Only AI when you spend the time to turn it into a real reviewed PR, which looks like X, Y, and Z", giving some actual tips on how to use AI acceptably. Which is what OP suggests.
felipeerias 11mo ago
Personally, I would not contribute to a project that forced me to lie. And from the point of view of the maintainers, it seems a terrible idea to set up rules with the expectation that they will be broken.
- 1gn15 11mo ago
  I know, right. It's like setting up rules saying "you can't use IDE autocomplete" or "you can't code with background music because that distracts you from bugs". If the final result is indistinguishable, I find it perfectly acceptable to lie. Rules are just words, after all, especially if it's completely unenforceable. Or, the decentralized, no rulers solution: clone the repo on your own website and put your patches there instead.

jrochkind1 11mo ago

Essay is way more interesting than the title, which doesn't actually capture it.

dang 11mo ago
Thanks for pointing this out—it made me take the time to find a sentence in the article body that could serve as a less baity title. From https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html: "Please use the original title, unless it is misleading or linkbait" (note that word unless)
jamesbelchamber 11mo ago
The title seems perfectly engineered to get upvotes from people who don't read the article, which puts the article in front of more people who would actually read it (which is good because the article is, as you say, very interesting and worth sharing). I don't like it but I can hardly blame them.
- jrochkind1 11mo ago
  Agreed. Sometimes such rage/engagement-bait titles get changed on HN, but it's risky to do as a submitter cause it's unclear when you are "allowed" to change the title. And I suppose if you want upvotes, why would you change the ragebait title? Usually engagement-bait titles are cover for uninteresting articles, but yeah in this case it's way more interesting than the title to me anyway. i guess it makes it even more obvious when people are discussing the title instead of the actual piece, which is routine on HN but not always obvious! Although to be fair, the title describes one part of the piece, sure. the part with the least original insight.

jcgrillo 11mo ago

> Some go so far as to say “AI not welcome here” find another project. This feels extremely counterproductive and fundamentally unenforceable to me. But it's trivially enforceable. Accept PRs from unverified contributors, look at them for inspiration if you like, but don't ever merge one. It's probably not a satisfying answer, but if you want or need to ensure your project hasn't been infected by AI generated code you need to only accept contributions from people you know and trust.

anon3242 11mo ago
This is sad. The barrier of entry will be raised extremely high, maybe even requiring some real world personal connections to the maintainer.
- jcgrillo 11mo ago
  Real world personal connections are how we establish trust. At some point you have to be able to trust the people you're collaborating with.

lapcat 11mo ago

> That said it is a living demo that can help make an idea feel more real. It is also enormously fun. Think of it as a delightful movie set. [pedantry] It bothers me that the photo for "think of prototype PRs as movie sets" is clearly not a movie set but rather the set of the TV show Seinfeld. Anyone who watched the show would immediately recognize Jerry's apartment.

DerThorsten 11mo ago
Its not the set of the TV show I beliefe, but a recreation. https://nypost.com/2015/06/23/you-can-now-visit-the-iconic-seinfeld-set-in-new-york/ https://nypost.com/2015/06/23/you-can-now-visit-the-iconic-s... It looks a bit different wrt. the stuff on the fridge and the items in the cupboard
- lapcat 11mo ago
  I'm not sure what you mean. Those two photos are very different. The floors are entirely different, the tables are entirely different, one of the chairs/couches is different, even the intercom and light switch are different. In any case, though, neither one is a movie set.
  - DerThorsten 11mo ago
    I think we agree, it looks like the seinfeld set, but it not the orginal set, just something looking very similar.
    - deleted 11mo ago
      [deleted]
- throwawaysoxjje 11mo ago
  It’s this Sony Picture Studios recreation actually: https://www.reddit.com/r/seinfeld/comments/yfbmn2/sony_pictures_studios_museum_has_a_recreation_of/ https://www.reddit.com/r/seinfeld/comments/yfbmn2/sony_pictu...

bloppe 11mo ago

Maybe we need open source credit scores. PRs from talented engineers with proven track records of high quality contributions would be presumed good enough for review. Unknown, newer contributors could have a size limit on their PRs, with massive PRs rejected automatically.

selfhoster11 11mo ago
We don't need more KYC, no.
- javier123454321 11mo ago
  Reputation building is not kyc. It is actually the thing that enables anonymization to work in a more sophisticated way.
mfenniak 11mo ago
The Forgejo project has been gently trying to redirect new contributors into fixing bugs before trying to jump into the project to implement big features (https://codeberg.org/forgejo/discussions/issues/337 https://codeberg.org/forgejo/discussions/issues/337). This allows a new contributor to get into the community, get used to working with the codebase, do something of clear value... but for the project a lot of it is about establishing reputation. Will the contributor respond to code-review feedback? Will they follow-up on work? Will they work within the code-of-conduct and learn the contributor guidelines? All great things to figure out on small bugs, rather than after the contributor has done significant feature work.

prymitive 11mo ago

The problem with AI isn’t new, it’s the same old problem with technology: computers don’t do what you want, only what you tell them. A lot of PRs can be judged by how well they are described and justified, it’s because the code itself isn’t that important, it’s the problem that you are solving with it that is. People are often great at defining problems, AIs less so IMHO. Partially because they simply have no understanding, partially because they over explain everything to a point where you just stop reading, and so you never get to the core of the problem. And even if you do there’s a good chance AI misunderstood the problem and the solution is wrong in a some more or less subtle way. This is further made worse by the sheer overconfidence of AI output, which quickly erodes any trust that they did understand the problem.

Lerc 11mo ago

It is possible that some projects could benefit from triage volunteers? There are plenty of open source projects where it is difficult to get up to speed with the intricacies of the architecture that limits the ability of talented coders to contribute on a small scale. There might be merit in having a channel for AI contributions that casual helpers can assess to see if they pass a minimum threshold before passing on to a project maintainer to assess how the change works within the context of the overall architecture. It would also be fascinating to see how good an AI would be at assessing the quality of a set of AI generated changes absent the instructions that generated them. They may not be able to clearly identify whether the change would work, but can they at least rank a collection of submissions to select the ones most worth looking at? At the very least the pile of PRs count as data of things that people wanted to do, even if the code was completely unusable, placing it into a pile somewhere might be minable for the intentions of erstwhile contributors.

dearilos 11mo ago

We’re fixing this slop problem - engineers write rules that are enforced on PRs. Fixes the problem pretty well so far.

jcgrillo 11mo ago

I guess the main question I'm left with after reading this is "what good is a prototype, then?" In a few of the companies I've worked at there was a quarterly or biannual ritual called "hack week" or "innovation week" or "hackathon" where engineers form small teams and try to bang out a pet project super fast. Sometimes these projects get management's attention, and get "promoted" to a product or feature. Having worked on a few of these "promoted" projects, to the last they were unmitigated disasters. See, "innovation" doesn't come from a single junior engineer's 2AM beer and pizza fueled fever dream. And when you make the mistake of believing otherwise, what seemed like some bright spark's clever little dream turns into a nightmare right quick. The best thing you can do with a prototype is delete it.

corytheboyd 11mo ago
Completely agree, I hate the “hackathon” for so many reasons, guess I’ll vent here too. All of this from the perspective of one frustrated software engineer in web tech. First of all, if you want innovation, why are you forcing it into a single week? You very likely have smart people with very good ideas, but they’re held back by your number-driven bullshit. These orgs actively kill innovation by reducing talent to quantifiable rows of data. A product hobbled together from shit prototype code very obviously stands out. It has various pages that don’t quite look/work the same, Cross-functional things that “work everywhere else” don’t in some parts. It rewards only the people who make good presentations, or pick the “current hype thing” to work on. Occasionally something good that addresses real problems is at least mentioned but the hype thing will always win (if judged by your SLT) Shame on you if the slop prototype is handed off to some other team than the hackathon presenters. Presenters take all the promotion points, then implementers have to sort out a bunch of bullshit code, very likely being told to just ship the prototype “it works you idiots, I saw it in the demo, just ship it.” Which is so incredibly short sighted. I think the depressing truth is your executives know it’s all hobbled together bullshit, but that it will sell anyway, so why invest time making it actually good? They all have their golden parachutes, what do they care about the suckers stuck on-call for the house-of-cards they were forced to build, despite possessing the talent to make it stable? All this stupidity happens over and over again, not because it is wise, or even the best way to do this, the truth is just a flaccid “eh, it’ll work though, fuck it, let’s get paid.”

andy99 11mo ago

This is a problem everywhere now, and not just in code. It now takes zero effort to produce something, whether code or a work plan or “deep research” and then lob it over the fence, expecting people to review and act upon it. It’s an extension of the asymmetric bullshit principle IMO, and I think now all workplaces / projects need norms about this.

solotronics 11mo ago
This problem statement was actually where the idea for Proof of Work (aka mining) in bitcoin came from. It evolved out of the idea of requiring a computational proof of work for sending an email via cypherpunk remailers as a way of fighting spam. The idea being only a legitimate or determined sender would put in the "proof of work" to use the remailer. I wonder how it would look if open source projects required $5 to submit a PR or ticket and then paid out a bounty to the successful or at least reasonable PRs. Essentially a "paid proof of legitimacy".
- strogonoff 11mo ago
  The parallel between PoW and barriers to entry many communities (be it Wikipedia editors or open-source contributors) use to sustain themselves seems apt. Unfortunately, there is no community equivalent of PoS—the only alternative is introducing different barriers, like ID verification, payment, in-person interviews, private invite system, etc., which often conflict with the nature of anonymous volunteer communities. Such communities are perhaps one of the greatest things the Web has given us, and it is sad to see them struggle. (I can imagine LLM operators jumping on the opportunity to sell some of these new barriers, to profit from selling both the problematic product and a product to work around those problems.)
  - Frieren 11mo ago
    > (I can imagine LLM operators jumping on the opportunity to sell some of these new barriers, to profit from selling both the problematic product and a product to work around those problems.) That is their business model. Use AI to create posts in LinkedIn, mails in a corporate environment, etc. And then use AI to summarize all that text. AI creates a problem and then offers a solution. My current approach is to look at new sources lie The Guardian, Le Monde, AP news, etc. I know that they put the work, sadly places like Reddit and such are just becoming forums that discuss garbage news with bot comments. (I could use AI to identify non-bot comments and news sources, but it does not really work even if it says that it does, and I should not have to do that in the first place either).

jongjong 11mo ago

2 months ago, after I started using Claude Code on my side project, within the space of days, I went from not allowing a single line of AI code into my codebase to almost 100% AI-written code. It basically codes in my exact style and I know ahead of time what code I expect to see so reviewing is really easy. I cannot justify to myself writing code by hand when there is literally no difference in the output from how I would have done it myself. It might as well be reading my mind, that's what it feels like. For me, vibe coding is essentially a 5x speed increase with no downside. I cannot believe how fast I can churn out features. All the stuff I used to type out by hand now seems impossibly boring. I just don't have the patience to hand-code anymore. I've stuck to vanilla JavaScript because I don't have the patience to wait for the TypeScript transpiler. TS iteration speed is too slow. By the time it finishes transpiling, I can't even remember what I was trying to do. So you bet I don't have the patience to write by hand now. I really need momentum (fast iteration speed) when I code and LLMs provide that.

listenallyall 11mo ago
I dont mean to question you personally, after all this is the internet, but comments like yours do make the reader think, if he has 5x'ed his coding, was he any good to begin with? I guess what I'm saying is, without knowing your baseline skill level, I dont know whether to be impressed by your story. Have you become a super-programmer, or is it just cleaning up stupid stuff that you shouldn't have been doing in the first place? If someone is already a clear-headed, efficient, experienced programmer, would that person be seeing anywhere near the benefits you have? Again, this isn't a slight on you personally, it's just, a reader doesnt really know how to place your experience into context.
- jongjong 11mo ago
  I have a computer science degree and quite experienced. In the last 10 years, I've created a few side projects and also worked as a senior engineer professionally. I was engineer #3 at what later became the #1 biggest Maths e-learning platform in my country. I later built a popular distributed open source WebSocket pub/sub system which auto-scales on Kubernetes (with load balancing and sharding). I lead the development of the P2P layer of a $300 million+ market cap crypto project in Germany. I wrote a multi-chain Decentralized Exchange from scratch along with a minimalist PoS blockchain with support for instant finality. Then I wrote a multi-tenant no-code serverless platform to build/host any kind of app. Then (in part to demonstrate the search and indexing capabilities of the serverless no-code platform), I used this platform to build a multi-tenant people/company search platform which allows users to crawl the web and tag data using vector embeddings using a BFS algorithm (current side project). I'm still running all these projects in parallel because they require very little maintenance. My blockchain nodes have been running for 4 years straight. I restarted a node twice in 4 years due to infrastructure issue so very reliable. Obviously, I suck at business and marketing. I only had one relatively financially successful product (my open source project, ironically) but I'm definitely able to build features quickly and in a stable way according to spec.

quxbar 11mo ago

If one claims to be able to write good code with LLMs, it should just as easy to write comprehensive e2e tests. If you don't hold your code to a high testing standard than you were always going off 'vibes' whether they were from a silicon neural network or your human meatware biases.

hoherd 11mo ago
Reviewing test code is arguably harder than reviewing implementation code because tests are enumerated success and failure scenarios. Some times the LOC of the tests is an order of magnitude larger than the implementation code. The biggest place I've seen AI created code with tests produce a false positive is when a specific feature is being tested, but the test case overwrites a global data structure. Fixing the test reveals the implementation to be flawed. Now imagine you get rewarded for shipping new features a test code, but are derided for refactoring old code. The person who goes to fix the AI slop is frowned upon while the AI slop driver gets recognition for being a great coder. This dynamic caused by AI coding tools is creating perverse workplace incentives.

specproc 11mo ago

A bit of a brutal title for what's a pretty constructive and reasonable article. I like the core: AI-produced contributions are prototypes, belong in branches, and require transparency and commitment as a path to being merged.

ninju 11mo ago

Well...just have AI review the PR to have it highlight the slop /s

gordonhart 11mo ago

> As engineers it is our role to properly label our changes. I've found myself wanting line-level blame for LLMs. If my teammate committed something that was written directly by Claude Code, it's more useful to me to know that than to have the blame assigned to the human through the squash+merge PR process. Ultimately somebody needs to be on the hook. But if my teammate doesn't understand it any better than I do, I'd rather that be explicit and avoid the dance of "you committed it, therefore you own it," which is better in principle than in practice IMO.

andrewflnr 11mo ago
If your teammate doesn't understand it, they shouldn't have committed it. This isn't a "dance", it's basic responsibility for your actions.

gwbas1c 11mo ago

> You can usually tell a prototype that is pretending to be a human PR, but a real PR a human makes with AI assistance can be indistinguishable. A couple of weeks ago I needed to stuff some binary data into a string, in a way where it wouldn't be corrupted by whitespace changes. I wrote some Rust code to generate the string. After I typed "}" to end the method: 1: Copilot suggested a 100% correct method to parse the string back to binary data, and then 2: Suggested a 100% correct unit test. I read both methods, and they were identical to what I would write. It was as if Copilot could read my brain. BUT: If I relied on Copilot to come up with the serialization form, or even know that it needed to pick something that wouldn't be corrupted by whitespace, it might have picked something completely wrong, that didn't meet what the project needed.

anal_reactor 11mo ago

An idea occurred to me. What if: 1. Someone raises a PR 2. Entry-level maintainers skim through it and either reject or pass higher up 3. If the PR has sufficient quality, the PR gets reviewed by someone who actually has merge permissions

sams99 11mo ago

Author here, thanks heaps for the discussion, I replied to a few of the points in my blog comments: https://discuss.samsaffron.com/t/your-vibe-coded-slop-pr-is-not-welcome/2492/4?u=sam https://discuss.samsaffron.com/t/your-vibe-coded-slop-pr-is-...

vrighter 11mo ago

Simple framework: You must not attempt to offload the actual thinking part to someone else. You should be able to answer any reasonable question about the code you submitted. If you can't you should be banned from any further contributions, permanently.