6 ms·
I've been looking a lot at Godot (another big open source project) PRs lately, and there's been kind of a surge of wholy ai-generated PRs (both code and descrip
by Fraterkes 5mo ago
I've been looking a lot at Godot (another big open source project) PRs lately, and there's been kind of a surge of wholy ai-generated PRs (both code and description).
This is agains project-policy, so people creating these PRs usually get mildly told off. What's surprising is that while many submitters take that fairly well, some people get really indignant, essentially calling the maintainers ungrateful.
It's kinda surprising to me that even the people who are all in on ai haven't internalized that there's no inherent value in producing a big lump of code. They've massively decreased the work they put in but still expect the same pre-ai reaction/gratitude when submitting a big PR.
- lucideer 5mo agoThe pre-ai reaction was also unwarranted: committing a massive amount of potentially unmaintainable handwritten code isn't a necessarily positive contribution and any decent engineer (or person tbh) would understand that & not expect gratitude, no matter how concerted their effort. In that context, I wouldn't expect an idiot (of which there has always been far too many in this industry) to change their behaviour in a post-ai world. They were always out of line & continue to be. Fwiw, a non-technical employee in my workplace has begun submitting ai-generated prs to internal repos I maintain & they're of excellent quality, with review feedback graciously received & expediently addressed, so this isn't a matter of the idiots not being technical, it's an attitude problem.
- DrewADesign 5mo agoSure, but I think we should judiciously avoid the false equivalence yielded by only looking at this on a developer-by-developer basis, rather than systemically. The truth is that in practice, AI is not a neutral force. Obviously AI can enhance the output of smart, experienced developers and improve the efficiency of code reviews, mitigating the effects of garbage PRs. However, it increases the percentage of PRs contributed by entirely inexperienced and/or not-smart devs from zero to, potentially, the majority. It entirely removes the barriers inherent to coding that kept Dunning-Krueger cases from submitting ill-conceived or poorly constructed changes— actually getting them to run in some way, even poorly. That makes them much more difficult to distinguish from well-constructed PRs than those from, say, someone cargo-culting code from tutorials. Moreover, as these tools become more expensive, people with money to blow on tokens will be able to drown maintainers that don’t have enough token-cash to help them deal with it. People see this as mostly a matter of time and energy, but I reckon it will soon be a financial issue.
- abirch 5mo agoI see AI as a barrier remover. Unfortunately some barriers are good or minimally necessary. I think we'll need to revert to artificial barriers such as bonds, e.g., if you want to do a PR to my repository you need to pay a 10 dollar bond. If the PR is good and I want future PRs, you keep your bond. If it's slop and spam, I get 10 dollars for my time.
- DrewADesign 5mo agoI agree with the bond in theory, but that would entirely stop contributions from people in economies where a shady maintainer could keep their code, and their weekly food budget.
- applfanboysbgon 5mo agoThis is entirely too much friction in the wrong place. Public open source will simply die before a system like that ever becomes the norm. The previous barriers worked because they were organically perfectly in line with a contributor's internal incentives. A contributor gains very little benefit from submitting a patch; the likelihood is infinitesimally small they'll ever get any career advancement, financial recompense, or even much community recognition for it. At most, it shifts the burden of maintaining the code they're contributing from themselves to the community / long-term maintainers. The real incentive for a contributor was making the patch, because they get to see the feature or fix they want made for the software. The previous barriers were in making the patch, and contributors would overcome that friction to gain the benefit of having the patch they want. Moving the barrier to merely submitting the patch after it has already been made will simply result in people not bothering, because there is very little incentivizing them to deal with the friction.
- abirch 5mo agoI don't disagree, where is the right place for the friction?
- deleted 5mo ago
- Fraterkes 5mo agoGratitude was maybe the wrong word. As the article mentions, before ai I think larger PRs, while sometimes inconsiderate, at least implied some amount of care / effort / good faith. In my experience, that was often rewarded with the maintainers at least taking a look at the code. I meant it's odd to have the same expectation when you dump 3000 lines in a pr that you won't even personally write a description for.
- Klonoar 5mo ago> Fwiw, a non-technical employee in my workplace has begun submitting ai-generated prs to internal repos I maintain & they're of excellent quality, with review feedback graciously received & expediently addressed, so this isn't a matter of the idiots not being technical, it's an attitude problem. It is hard for me to imagine another engineering discipline that would be totally fine accepting work from those who don't have the actual engineering background required to do the work. If I had to push this take to the extreme: software engineers never learned class solidarity and it's now biting the industry in the ass.
- dcrazy 5mo agoMan, you must hate those handymen who put up YouTube videos showing how to do basic home maintenance. A truly class-conscious handyman would insist that the homeowner hire them to replace a light switch.
- Based-A 5mo agoCome on, are you really going to strawman "engineering should be inherently disqualifying" into "so people shouldn't DIY in their own home"? Please try a little bit harder.
- dcrazy 5mo agoI’m not strawmanning anything. I’m pointing out what I believe to be ridiculous gatekeeping. Software engineering isn’t some holy magic that must be kept from the masses. I can go on YouTube and get step-by-step instructions on how to safely wire an entire house. In many jurisdictions I would even be allowed to do that. I can get instructions on how to completely redo a bathroom, down to the studs and up through the waterproofing and tiling. I can get instructions on how to do foundation repair, which might be a bit much for me but can help me ask the right questions to keep the contractor I hired honest. These are all examples of experts acting as “traitors” to their particular group. In reality, technology enables both specialization and despecialization. Some people try to cling to their specializations and cry “class warfare” when threatened.
- jakolaptu 5mo agoBut your company employs said individual, whereas arbitrary drive-by patches from randos on open source projects with no consequence of submitting a mountain of garbage. The answer: require a written proposal for changes before a patch will even be considered unless it is sufficiently small. Also fight AI with AI: have a bot auto reject patches unless they can link to a previously approved enhancement document. Folks who commit minimal effort will f*ck right off. Then the cognitive burden is focused on the ideas, and code authors should have at least conveyed the intent. If they actually care to invest their skin in the game then they need to collaborate and not just drop garbage on the front door.
- lucideer 5mo agojust to clarify: 1. my company also employs its fair share of folk that would fit into the so-called "idiot" category of my post - I just thought it was of note that I have encountered exceptions to this stereotype 2. I fully support what Ladybird is doing here & find it unfortunate that they have to. I didn't intend my post to criticise their move - my example is definitely the rare exception in a sea of unmaintainable garbage. I do think however that it was already a challenging prospect to manage garbage oss prs in the pre-llm era (see umpteen posts on maintainer burnout) & I wouldn't have faulted any open source project for doing what ladybird is doing even pre-llm.
- zzzeek 5mo agoyes. github has very minimal controls for PRs (either shut them off completely, or not at all) but through GH webhooks you can essentially script "auto-close" of PRs that dont have correct preliminary approval. it's time for there to be some really nice workflow tooling that people can plug into their GH repos that does this and other things.
- isityettime 5mo agoOn this topic, "Open-Source Isn't About You": https://gist.github.com/richhickey/1563cddea1002958f96e7ba9519972d9 https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...
- torben-friis 5mo agoThere is certainly a certain... entitlement? (It's not the perfect word, but I fail to find a proper term) from some of the vibe crowd. Like an attachment to the output and refusal to accept that most of the work was not theirs. It is seen in the way they approach contributions but also in regular language. I created X, insistence that their 'curation' was very influencial to the output, difficulty to mention LLM contribution, attitude of 'I care about building while others lose time in details', refusal to engage with potential flaws, and so on. It is surprisingly different to what I'm used to from senior devs, which behave like they always suspect their own work is flawed and half assed. Like impostor syndrome was reversed.
- artyom 5mo agoI agree it's not "entitlement" specifically but there's something there. I guess by now everyone has experienced that type of person that "tries to help" by copy/pasting a bunch of AI slop and expecting you to work through the cognitive load of validating it. The original post sums it pretty well, such big output inherently meant big effort, which was a proxy for good faith. Now that's gone.
- Sharlin 5mo agoIt was a proof of work system. When work becomes cheap, it stops being proof.
- progbits 5mo agoI've experienced the following sequence more than once at work, and I remain baffled by it each time: - Receive a huge vibecoded PR for complicated new feature. - Complain that this needs some design doc to figure out the right approach first. - Author says no need for design doc, easier to have vibed implementation and discuss the concrete code instead of abstract document. - I disagree (obviously), but review the PR with feedback along the lines: this entire approach is flawed, throw this out and start over. - Author gets defensive, says "but this is already working and ready, let's just merge". - I tell them there is no chance in hell this is getting merged. They go sulk to their manager that I'm not interested in helping them launch.
- Forgeties79 5mo ago> What's surprising is that while many submitters take that fairly well, some people get really indignant, essentially calling the maintainers ungrateful. I would be more sympathetic if they actually spent meaningful time on these contributions and could maybe see an argument for wanting their work to be given due consideration (lots of caveats here), but from what I’ve seen that’s the exception rather than the rule with a lot of these case.
- zkry 5mo agoI can imagine in these cases the LLM is telling the "contributor" how smart they are and how much the project is loosing out, maybe saying something like: "It's not about maintaining project boundaries, it’s not about ensuring code quality; it’s a gatekeeping mechanism designed by traditionalists who feel threatened by forward-thinking creators like you who truly master the efficiency of AI."
- cmiles74 5mo agoI wouldn’t be surprised to find out this is part of their RHLF training, the attitude is so prevalent in these models.
- throawayonthe 5mo agoreally? in the way the parent comment describes? (genuine) i suppose i've never encountered a context where it would be relevant for an LLM to output that, interesting
- dofm 5mo agoWhether or not they are, I agree it is entirely possible to imagine them doing that, given what we know about how AI chat has reinforced people doing much worse to themselves and others. (And the whole "miffed AI wrote a shitpost" thing)
- helloplanets 5mo agoIf a project has a rule to not submit AI generated PRs, people should never submit AI generated PRs to that project. It's spam. Or if the rule is more nuanced than that in relation to AI, it should be respected. It's 100% an issue with the people with submitting these PRs. So, if someone has a history of having no issue with breaking project rules (let alone being arrogant about it), it should be a massive red flag about the person for any possible employer or future collaborator checking their profile, etc. Why people are wilfully destroying their own reputation like that is beyond me. It's infinitely better to have no activity at all on your profile than to create a track record of bad behaviour.
- radlad 5mo agoI can empathize with this view but I've also seen one line clear bug fixes rejected because CC signed off on it. There's no other correct fix - why do you care which pen I used to write it?
- dormento 5mo agoI don't really know if this would be the best solution, but we could maybe define a sliding LoC limit per PR? Also a limit on the number of open PRs per contributor. This way those small, reviewable contributions would get a fair shot at being landed instead of being blanket banned, and the maintainer overhead would be more reasonable.
- eschaton 5mo agoEven that small one-line contribution is not the contributor’s own work. Since the contributor does not hold the copyright to it, they do not have the right to contribute it to the project.
- rurban 5mo agoThere massive value in AI PR's. If a feature and ignored, it can forked to provide more value to the users. If unaccepted bugfixes, the maintainers are just silly. They need to be forked off.
- bpicolo 5mo agoI mean, aren't you kind of proving the poster's point? Fork away. If you want to put in the meaningful effort required to maintain and improve upon a project as significant as Godot, and feel that AI is a mechanism you want in order to do so, go for it. Clearly, the maintainers don't feel that that's the best approach to create the product they want to create, and they are not required to accede to the sense of entitlement of the community.
- rurban 5mo agoEven before AI it was trivial to setup a continuous merge script. I did that several years for several projects which refused my PR's. Nowadays it's even more trivial. And a community is more of a burden than an advantage nowadays. Users are ok, but a community not so. See python, perl, ruby, node and countless others.
- losvedir 5mo agoIt's interesting to see this perspective in the wild. In the age of AI I wonder what "massive value" your PR is bringing to the maintainer. $1 worth of tokens?
- rurban 5mo ago[flagged]
- ecshafer 5mo agoI regularly find the code output of opus and gpt 5.5 to be garbage. Overly verbose, unnecessary abstractions, strange duplication of concepts across objects, unnecessary copying of objects and creation of objects. I have found its much more useful to just ping pong some ideas, have it generate helper methods, and do the code implementation by hand. I guess I am a stoneager.
- Intralexical 5mo ago> What's surprising is that while many submitters take that fairly well, some people get really indignant, essentially calling the maintainers ungrateful. Why is it surprising that some people who expect results without spending any effort also feel entitled to receiving gratitude without putting in any thought?
- Fraterkes 5mo agoIf I don’t word these critiques in the most diplomatic way possible, this immediately turns into a discussion about the prevalence of anti-ai sentiment on hn. Which would be boring.
- jstummbillig 5mo ago> They've massively decreased the work they put in but still expect the same pre-ai reaction/gratitude when submitting a big PR. I don't think that follows. They expect things to improve, they do something about it and might (unreasonably) be frustrated by what they think is a policy that stands in the way of quicker progress. The first part is certain, the second part less so, and the third is just speculation. It's clear that open source project are struggling to understand what is going on and coming up with plans – like everyone else – but clearly there are better and worse ways to proceed in this new world, if popularity, adoption and progress are things you want to focus on.
- nunez 5mo agoWhat would be great is something like Pangram for PRs and issues. Detects if the PR or issue is AI-generated (with multiple checkers for redundancy) so that maintainers can choose to auto-close and PR authors have to at least manually write _something_ before they launch 20k SLOC diffs.
- Chu4eeno 5mo agoIsn't Pangram more expensive per token than most (all?) LLMs? Sounds too easily DoSable.
- jakolaptu 5mo agoOne could call a big lump of code a Tumor.
- kzz102 5mo agoSince Godot got mentioned, I thought this recent talk by Juan Linietsky provides a reasonable alternative to heavily restricting contribution. https://youtu.be/LDSwP37y_W4?si=fMIsdQ2yjoiGChb6 https://youtu.be/LDSwP37y_W4?si=fMIsdQ2yjoiGChb6
- matheusmoreira 5mo ago> there's no inherent value in producing a big lump of code I dunno. If that lump of code makes the program do things it wasn't doing before, I'd say there's a lot of value in it. We can definitely scrutinize the code's quality but there's no way anyone can argue with working code that makes the program better or more featureful than it was before.
- InfiniteRand 5mo agoCode that works but is unmaintainable is a burden on a project, that being said with open source, you can always fork the code, add your code, and let people decide
- matheusmoreira 5mo agoNo doubt. The maintainability of the code should definitely be scrutinized and improved. Quality in general should be maximized. Doesn't change the fact that working code does have value. If your program didn't do something before and it does now, that absolutely has value.
- danbolt 5mo agoI’ve seen bounties for re-adding C# to web exports to Godot 4. [1] It’s something that’s had some technical roadblocks for the engine’s developers, and looks like it might take a bit of time. [2] There tend to be a lot of drive-by AI PRs attempt to “re-add” the feature, often not quite addressing the situation comprehensively. It seems like a bit of a local minima trap for the bots. [3] [1] https://app.opire.dev/issues/01J8YJ06HPSY7ZAMAW08T83YBD https://app.opire.dev/issues/01J8YJ06HPSY7ZAMAW08T83YBD [2] https://godotengine.org/article/live-from-godotcon-boston-web-dotnet-prototype/ https://godotengine.org/article/live-from-godotcon-boston-we... [3] https://github.com/godotengine/godot/pull/119972 https://github.com/godotengine/godot/pull/119972
- Chu4eeno 5mo agoWow, you weren't kidding, I found these before I got bored (they all seem to be LLM-authored from a quick skim): - https://github.com/godotengine/godot/pull/115280 https://github.com/godotengine/godot/pull/115280 Implement C# .NET Integration via Headless Glue Bypass (Build 7ae8ec974) by Eliene-byte - https://github.com/godotengine/godot/pull/107146 https://github.com/godotengine/godot/pull/107146 [.NET] Add web export support for C# projects in Godot 4 by Enrique726 - https://github.com/godotengine/godot/pull/117787 https://github.com/godotengine/godot/pull/117787 Readd .NET web export support by santosparra651-arch - https://github.com/godotengine/godot/pull/119405 https://github.com/godotengine/godot/pull/119405 WIP: Add Web C# export pipeline pieces by acidstorm2024-star - https://github.com/godotengine/godot/pull/119119 https://github.com/godotengine/godot/pull/119119 Allow building Web platform with C#/.NET module enabled by AndresFpdhi - https://github.com/godotengine/godot/pull/119330 https://github.com/godotengine/godot/pull/119330 Cross runtime Solution to Enable C# Web Exports by tommygrammar - ... and more (some of the claims on he opire thing seem to have been scrubbed, though, or they never opened PRs).