7 ms·

PS3 Emulator Devs Politely Ask That People Stop Flooding It with AI PRs

189 points by stalfosknight 5mo ago

saagarjha 5mo ago
The emulation space is particularly bad about this because there are a lot of semi-technical and "well meaning" users who will do anything to get their games to play better and AI gives them a way to make it seem like they are doing something useful, without being able to judge the quality of the output they are producing. One of the projects I work on recently had a guy drop by and explain that he wanted to use Claude to clean up our backlog and he absolutely could not fathom why I kept bringing up that we would only accept PRs that reduced our work instead of increasing it. "Do you know what Opus 4.7 is?" "Why are you so close-minded?". Unfortunately it is very hard for these users to understand that the thing they are using has a bar for quality and the bugs that still slip through cannot be solved by waving a magic wand at it.
- loloquwowndueo 5mo ago
  A good argument to use could be: I can use Claude myself, so I will if I need to, but you using Claude on my behalf doesn’t save me any work, it just introduces another layer of noise into the mix. (Yes calling the guy “noise” haha)
  - saagarjha 5mo ago
    Yeah I basically said this and the guy claimed I didn't understand what he was offering
- NoMoreNicksLeft 5mo ago
  Over the last month, I've been using Claude to assist in some things that were at the edge of my ability (or maybe just a hair's breadth beyond it). I've added features to open source projects that everyone's been waiting years for. I always fork it telling myself that I want to be able to submit PRs, but really I'm just making the changes for myself, since I don't even have the nerve to show it off. If these people can make changes to the emulators that will actually make the games more playable for them, the changes don't have to go back into the official project. It works for them and makes things better. Right now, I've been working on some changes to the mkv container spec to have embedded scripting cable of doing Black Mirror: Bandersnatch in interactive mode. VLC and mpv. I've already added mutable torrent support to Transmission, and it works. But yeh, if someone took a look at it who really knew the code, they'd see it was AI slop and do a hard pass.
  - sitkack 5mo ago
    You could use your changes to show that a feature is useful, but not do a PR. The specific code is no longer that important.
    - oneshtein 5mo ago
      In era of AI, prompts are the source.
  - saagarjha 5mo ago
    There's another person who for a long time wrote kind of hacky patches to work around bugs (e.g. disabling multithreading to avoid a longstanding race) and now he's using AI to fix everything in his own fork. I guess he can do that? We haven't really been able to use any of his changes though.
HDBaseT 5mo ago
I recently just started using Claude/ChatGPT/China models for some PS3 homebrew work. Every model seemingly falls flat in this scope of programming. The PS3 is very complex and the tooling is fairly undocumented in a lot of instances. It doesn't surprise me most of these AI PR's are nonsense. If anyone else has attempted writing PS3 homebrew apps using AI and has refined their tooling/systems/automation please let me know how you got the agents to work for you (:
- _JoRo 5mo ago
  I've been working on a project myself over the last few weeks where the documentation is quite minimal. To no surprise the LLMs fell flat at being able to generate any sort of meaningful code. However, I realized that if I focused first on building out documentation and coding tools (linters, parsers, formatters, etc...), LLMs can do a decent job at solving fundamental problems.
- Aurornis 5mo ago
  I like to send Claude Code or Codex on max settings off to try a problem in parallel while I work on it. In a complex codebase it’s funny how often they’ll come back with gigantic commits that just make everything worse or accomplish the goal but have 1000 lines of unnecessary complexity. Every time they present it with a confident summary. I can see how a junior or just lazy dev would think this is their ticket to becoming a contributor to a repo with some big thing to put on their resume.
- nxobject 5mo ago
  Similar for me, but regarding the Classic Macintosh APIs. The difference is that there are plenty of books, and some source code available… just not enough to stop Codex from writing subtly wrong gibberish. I get the impression that the “10x velocity!!!!” claims still only reflect which areas have a sufficient corpus to learn from, rather than any inductive reasoning.
  - christkv 5mo ago
    You are completely right it's a what is the next token guessing machine so without corpus it's guesses are worse as expected.
MBCook 5mo ago
It’s starting to feel like we may need to go back to the model where you need to be invited to be able to submit code or PRs. The barrier is just too low now for popular projects.
- hsbauauvhabzb 5mo ago
  I think some sort of reputation score would make more sense, assuming it’s possible to design one that can’t be easily faked
  - Groxx 5mo ago
    Perhaps something where you can build a graph of who invited whom so you could prune entire sections that act maliciously. One might even consider it a to be a web of connections which are built on (or torn down by the loss of) trust. Sounds futuristic. Maybe it's an NFT on an agentic blockchain for deep-sea solar farm mining?
    - lobf 5mo ago
      Private torrent trackers apparently do this, and have done so for years.
      - perching_aix 5mo ago
        They're sarcastically describing web-of-trust: https://en.wikipedia.org/wiki/Web_of_trust https://en.wikipedia.org/wiki/Web_of_trust Why are they doing that (i.e. being sarcastic)? Who knows.
        Groxx 5mo ago
        Because it's by far the dominant strategy for distributed trust-ranking systems out there, with decades of research around it. Might as well look at the forest when realizing that it'd be nice if trees existed. And I don't think anyone actually trusts any major actor to verify anything, so a fully centralized system is likely out. Otherwise people would be hype about WorldCoin, instead of recognizing it for the stupendously malicious grift that it is.
_JoRo 5mo ago
I'm curious what percentage of PRs are just the AI blindly writing code and submitting a PR without testing, and which have at least been locally tested to some degree. Any OS maintainers have any insights on this?
- gerdesj 5mo ago
  [flagged]
- greenknight 5mo ago
  Thats the thing, what if the codebases had CLAUDE.md / AGENTS.md files, which clearly dictated that A) tests need to pass B) anything you write needs tests C) the code quality must adhere to these standards etc.etc.... Helping the LLMs that people Vibe code with, produce better quality results. By not having these in place, it means people who want to help out, cant. because htey dont understand whats going on. adding stuff to these files, woudl allow developers to give guidelines / guardrails for developement using these agents. Should the barrier of entry be someone who knows how to code? or should the barrier of entry be someone who is motivated to help with open-source software.
  - int0x29 5mo ago
    It really shouldn't be the RPCS3 devs' problem to fix other people's broken AI pipelines.
  - GCUMstlyHarmls 5mo ago
    > Should the barrier of entry be someone who knows how to code? or should the barrier of entry be someone who is motivated to help with open-source software. Probably yes? QED submitting slop PRs is not helping. If "helping" is sticking it through an LLM, the developers can do that themselves with better insight and guidance? If you must help via an LLM, donate cash for tokens. If you can't code, and cant donate cash/machine time, help by confirming issue reproductions, design, wikis, documentation, whatever.
  - estimator7292 5mo ago
    [dead]
jamesu 5mo ago
We've seen a few takes on this kind of issue, but the solution I liked the best was the linux "developers take full responsibility" approach. The "Assisted-by:" tag was a pretty nice touch too. The article unfortunately feels more like a rant than a good exploration of the problem space.
- ollien 5mo ago
  I've struggled with this "responsibility" take. What does it mean in the context of an open source project? As far as I understand it, the original contributors of bugs are often not the ones fixing them (though they can be). Is it that if you write enough buggy code you get banned as a contributor? Is it that you're not allowed to say Claude ate my homework?
  - x-complexity 5mo ago
    > Is it that if you write enough buggy code you get banned as a contributor? If this is a consistent issue, your contribution would (ideally) be continuously put into a backlog until someone else with no connection to you verifies that it's as bug-free as it appears to be. (Excluding non-obvious security & performance issues) > Is it that you're not allowed to say Claude ate my homework? Yes. As the contributor, you should be the first one to look over the code, not someone else.
- bayarearefugee 5mo ago
  > the solution I liked the best was the linux "developers take full responsibility" approach. The people who can realistically submit a Linux patch that will ever get looked at is already a super select group through who-you-know network effects. You can't apply the same system to random open source projects, the best option for people that run random small to medium sized open source projects is just to ban all unsolicited PRs, otherwise you're going to spend way too much effort sorting through the slop.
  - pabs3 5mo ago
    I don't think that is true at all, I'm just a random FOSS dev with no connection to the Linux kernel community and I have gotten two small commits into the Linux kernel.
nlh 5mo ago
I’ve read so many stories like this that I’ve actually gotten scared of making PRs open source projects. There’s one in particular where a feature I really wanted didn’t exist, so I forked and had Codex 5.5 assist with building the feature on my local version. It works perfectly. My life has been improved in being able to have this feature now. Normally I’d want to share it back with the community so others can benefit as well (presumably if I wanted this feature, others probably want it too.) But…I am not pretending this is perfect, great, or even good code. I spent about an hour total on it - it works, I haven’t had any issues with it, but it’s probably slop by any hard-core engineering account. And I neither want to get attacked for submitting slop nor do I have the time to properly engineer it to be hand-coded, so the net result is that it lives on my machine alone. Is this the right outcome? I feel guilty that I’m getting a better version of this software and others aren’t. I want to help makes others lives easier too, but I don’t want to burden the project maintainers or get yelled at for submitting slop. What’s the future look like here?
- Panzer04 5mo ago
  If you're upfront about the provenance and amount of effort that went into it, is there really a problem? I feel like the issue is people contributing code they don't understand and presenting it as if they do.
  - MBCook 5mo ago
    Quite possibly never tested, of maybe only tested their problem and not if it broke anything else.
    - grebc 5mo ago
      Not quite possibly. 99.99999% likely.
- Barrin92 5mo ago
  >Is this the right outcome? Yes, if you can't vouch for the quality of the code that is the correct outcome. The long term health and maintainability of an open source project takes precedence over adding another feature. This was the case before repos were flooded with AI slop as well. Virtually no project would have accepted a random code dump if the person submitting it does not understand it because that just means the burden falls on someone else which would very quickly get any software project into big trouble.
mrandish 5mo ago
If you look into the arcane architecture of the Playstation 3 console you quickly gain an appreciation for just how impressive the RPCS3 emulator is. PS3 is definitely one of hardest emulation targets, so it's wild they have the majority of the library working (with enhancements like upscaling and higher frame rates on many of the titles). I guess it's nice people want to help and AI assisted coding can be fine but I can't imagine submitting a PR to a high-profile, much-revered project like that without reviewing and thoroughly testing it myself.
- HerbManic 5mo ago
  When I saw it treats SPE code in the same manner as shader code on GPUs that was an enlightening moment. It made so much sense to treat it like that. That move while complicated removes a lot of potential performance issues.
tick_tock_tick 5mo ago
This is like when everyone started opening code of conduct addition PRs against every opensource repos except now a lot of these AI ones take actual effort to know it's in bad faith. Or maybe it's worse because a lot of them aren't in bad faith they are well meaning people who just don't know or understand enough to realize they aren't being helpful.
ls612 5mo ago
I’m hopeful that in a year or so the models will be good enough to help productively with emulator development and that you will see a similar shift to these PRs that you did with security this spring.
- MBCook 5mo ago
  Will they get there? They rely so much on existing content. But in such a niche area where the documentation or other solutions often flat out don’t exist how are they supposed to get better through training?
perching_aix 5mo ago
Aww, PRs no longer open/welcome? Whatever will the usual suspects parrot now? My personal schadenfreude aside, I wonder if this will follow a similar trajectory as security bug reports did recently. I'd be surprised, for a number of reasons, but the overall shape is looking awfully similar.
Marsymars 5mo ago
I just took a look at the RPCS3 PR history, and it doesn't look that bad. (Certainly worse than "no slop", but not what I'd call a flood.) I went 10 pages back on GitHub, and the overwhelming number of PRs look like good PRs that have been merged. There's really only a single handful of rejected slop-looking PRs. (And another handful from a single user who seemingly didn't know how to use Git/GitHub and was turning local non-compiling commits into PRs somehow.)
sockbot 5mo ago
The problem is really behavioural, not the tooling. People that do not understand, test and document their decision making in their PRs should not be submitting them, regardless of what tooling (AI or otherwise) they used to create them. This problem existed before AI, but it is now just worse due to the spamming nature of these "contributors". It's another form of endless September where people unfamiliar with the norms of team software development are overwhelming existing project maintainers faster than maintainers can teach them the norms of behaviour. In the end, some sort of gatekeeping mechanism is needed to avoid overwhelming maintainers, whether it's a reputation system, membership in an in-group, or something else.
- 7e 5mo ago
  [flagged]
  - gassi 5mo ago
    You wouldn't hold that opinion it you did maintain a popular open-source repo or interact with AI "PR review" tools at a serious level. Even the most SOTA models are willing to accept/merge absolutely trash PRs so long at the summitter can convince it that is addressed it's review comments.
- heavyset_go 5mo ago
  No, it is a tooling problem. The tooling is telling laymen that they built wonderful things that definitely work and perfectly fix and add features. The tooling gasses them up and is simply wrong in these cases. If your tool regularly lies, gaslights and produces wrong results, that's a tooling issue.
  - fatata123 5mo ago
    [dead]
  - deleted 5mo ago
    [deleted]
  - deleted 5mo ago
b00ty4breakfast 5mo ago
what is the appeal of blindly blasting open source projects with high-volume PRs? If you're trying to help the project to accomplish something, it doesn't follow that a firehose approach is tenable, if only for the fact that reviewing the code takes time.
- x-complexity 5mo ago
  > what is the appeal of blindly blasting open source projects with high-volume PRs? The prestige of being "the one that added feature X to OSS project Y". The things that would've been actually useful (bug diagnostics/troubleshooting, merging duplicate issues & PRs) do not offer the same level of prestige.
- VoidWhisperer 5mo ago
  At some point it used to be in order to have things that you can show got merged into popular public projects in job interviews, but I'm not sure that is the case anymore since some of these people have no intention (as far as i can tell) of finding a SE job
- MBCook 5mo ago
  At this point these could just be gamers who want to play a game and are being annoyed by something not being right. Maybe they use Claude or whatever and tell it to fix the problem and then just blindly submit it. I could see people doing that without knowing enough to be able to compile and test the code, ignoring whether it’s good or not. So they just submit it and hope it gets merged to “fix” the problem, having no understanding of what’s involved or how much of a burden that is. Now imagine a whole bunch of people doing that for a whole bunch of really complex bugs in 75 different games. It’s not like the PlayStation three was a simple system.
- numpad0 5mo ago
  Instant gratification? This feels like the exact same phenomenon as kids trying to profiteer from repackaged game mods and ripped game assets.
- doctor_radium 5mo ago
phendrenad2 5mo ago
Perhaps a "wall of shame" showing off bad PRs would make people think twice.
- habinero 5mo ago
  It would not. The kind of person who does this always thinks they are the exception.
villgax 5mo ago
Just paywall it, pays for development, queue prioritization and in return they get code merged into a popular project and visibility
- CWwdcdk7h 5mo ago
  Isn't it simpler, in general, to just move the development off GitHub, while leaving existing repo in sync, so it can harmlessly drown in PRs from people who didn't bother to even read project contribution guidelines?
emsign 5mo ago
AI could be the end of FOSS if this doesn't stop. Why don't people get it? No AI means no AI.
wilg 5mo ago
Regardless of whatever other opinions you have on this, I don't think its a fair characterization that the original tweet was "polite" or "nice".
deleted 5mo ago
[deleted]
ares623 5mo ago
why not just fork and forge an entirely new path unfettered by outdated norms? Isn't that the AI way?
ixxie 5mo ago
Compare two popular FOSS harness projects: ----- OpenCode 4.9k issues 1.7k PRs 158k stars https://github.com/anomalyco/opencode https://github.com/anomalyco/opencode ----- Pi 31 issues 4 PRs 47k stars https://github.com/badlogic/pi-mono https://github.com/badlogic/pi-mono Their secret? A very rigorous contribution policy. Essentially, issues and PRs are autoclosed, and reviewed daily by the team. If its not slop, they whitelist either the issue/PR or the contributor (so their stuff isn't autoclosed next time). https://github.com/badlogic/pi-mono/blob/main/CONTRIBUTING.md https://github.com/badlogic/pi-mono/blob/main/CONTRIBUTING.m... GitHub needs an issue / PR approval flow.
motbus3 5mo ago
We learned from the past litellm security incident that there are tons of compromised/fake GitHub stars (which was known from the past due to the star farms) I don't see vibe coders trying to push PRs in good faith in a old emulator. Could it be that the sheer amount of PRs and eventually a bad PR in the middle is being used to compromise repositories?
- sunaookami 5mo ago
  It's for their CV's
  - deleted 5mo ago
    [deleted]
  - motbus3 5mo ago
    Does it really count? Honest question. I mean, if I interview a candidate who put such thing on the CV, it wouldn't matter, but if it did, I would make 300% sure to ask him all the ins and outs. I think it would be plain stupid to expose that you didn't do that instead of showing smaller collaboration that you completely understand
    - sunaookami 5mo ago
      >I would make 300% sure to ask him all the ins and outs In a healthy job market this would be the norm, yes.
stuaxo 5mo ago
If someone doesn't understand every bit of the PR they are submitting it should not be submittable - yes it takes time, but you are expecting devs on the other end to take more time than that. Though one plus point: a dev can ask the LLM to: - Split a PR into logical patches - Explain each one From there as questions and edit and rebase each until it makes sense, because it's guaranteed that not all of it will until you do that.
deleted 5mo ago
[deleted]

189 points by stalfosknight 5mo ago

The emulation space is particularly bad about this because there are a lot of semi-technical and "well meaning" users who will do anything to get their games to play better and AI gives them a way to make it seem like they are doing something useful, without being able to judge the quality of the output they are producing. One of the projects I work on recently had a guy drop by and explain that he wanted to use Claude to clean up our backlog and he absolutely could not fathom why I kept bringing up that we would only accept PRs that reduced our work instead of increasing it. "Do you know what Opus 4.7 is?" "Why are you so close-minded?". Unfortunately it is very hard for these users to understand that the thing they are using has a bar for quality and the bugs that still slip through cannot be solved by waving a magic wand at it.

loloquwowndueo 5mo ago
A good argument to use could be: I can use Claude myself, so I will if I need to, but you using Claude on my behalf doesn’t save me any work, it just introduces another layer of noise into the mix. (Yes calling the guy “noise” haha)
- saagarjha 5mo ago
  Yeah I basically said this and the guy claimed I didn't understand what he was offering
NoMoreNicksLeft 5mo ago
Over the last month, I've been using Claude to assist in some things that were at the edge of my ability (or maybe just a hair's breadth beyond it). I've added features to open source projects that everyone's been waiting years for. I always fork it telling myself that I want to be able to submit PRs, but really I'm just making the changes for myself, since I don't even have the nerve to show it off. If these people can make changes to the emulators that will actually make the games more playable for them, the changes don't have to go back into the official project. It works for them and makes things better. Right now, I've been working on some changes to the mkv container spec to have embedded scripting cable of doing Black Mirror: Bandersnatch in interactive mode. VLC and mpv. I've already added mutable torrent support to Transmission, and it works. But yeh, if someone took a look at it who really knew the code, they'd see it was AI slop and do a hard pass.
- sitkack 5mo ago
  You could use your changes to show that a feature is useful, but not do a PR. The specific code is no longer that important.
  - oneshtein 5mo ago
    In era of AI, prompts are the source.
- saagarjha 5mo ago
  There's another person who for a long time wrote kind of hacky patches to work around bugs (e.g. disabling multithreading to avoid a longstanding race) and now he's using AI to fix everything in his own fork. I guess he can do that? We haven't really been able to use any of his changes though.

HDBaseT 5mo ago

I recently just started using Claude/ChatGPT/China models for some PS3 homebrew work. Every model seemingly falls flat in this scope of programming. The PS3 is very complex and the tooling is fairly undocumented in a lot of instances. It doesn't surprise me most of these AI PR's are nonsense. If anyone else has attempted writing PS3 homebrew apps using AI and has refined their tooling/systems/automation please let me know how you got the agents to work for you (:

_JoRo 5mo ago
I've been working on a project myself over the last few weeks where the documentation is quite minimal. To no surprise the LLMs fell flat at being able to generate any sort of meaningful code. However, I realized that if I focused first on building out documentation and coding tools (linters, parsers, formatters, etc...), LLMs can do a decent job at solving fundamental problems.
Aurornis 5mo ago
I like to send Claude Code or Codex on max settings off to try a problem in parallel while I work on it. In a complex codebase it’s funny how often they’ll come back with gigantic commits that just make everything worse or accomplish the goal but have 1000 lines of unnecessary complexity. Every time they present it with a confident summary. I can see how a junior or just lazy dev would think this is their ticket to becoming a contributor to a repo with some big thing to put on their resume.
nxobject 5mo ago
Similar for me, but regarding the Classic Macintosh APIs. The difference is that there are plenty of books, and some source code available… just not enough to stop Codex from writing subtly wrong gibberish. I get the impression that the “10x velocity!!!!” claims still only reflect which areas have a sufficient corpus to learn from, rather than any inductive reasoning.
- christkv 5mo ago
  You are completely right it's a what is the next token guessing machine so without corpus it's guesses are worse as expected.

MBCook 5mo ago

It’s starting to feel like we may need to go back to the model where you need to be invited to be able to submit code or PRs. The barrier is just too low now for popular projects.

hsbauauvhabzb 5mo ago
I think some sort of reputation score would make more sense, assuming it’s possible to design one that can’t be easily faked
- Groxx 5mo ago
  Perhaps something where you can build a graph of who invited whom so you could prune entire sections that act maliciously. One might even consider it a to be a web of connections which are built on (or torn down by the loss of) trust. Sounds futuristic. Maybe it's an NFT on an agentic blockchain for deep-sea solar farm mining?
  - lobf 5mo ago
    Private torrent trackers apparently do this, and have done so for years.
    - perching_aix 5mo ago
      They're sarcastically describing web-of-trust: https://en.wikipedia.org/wiki/Web_of_trust https://en.wikipedia.org/wiki/Web_of_trust Why are they doing that (i.e. being sarcastic)? Who knows.
      - Groxx 5mo ago
        Because it's by far the dominant strategy for distributed trust-ranking systems out there, with decades of research around it. Might as well look at the forest when realizing that it'd be nice if trees existed. And I don't think anyone actually trusts any major actor to verify anything, so a fully centralized system is likely out. Otherwise people would be hype about WorldCoin, instead of recognizing it for the stupendously malicious grift that it is.

_JoRo 5mo ago

I'm curious what percentage of PRs are just the AI blindly writing code and submitting a PR without testing, and which have at least been locally tested to some degree. Any OS maintainers have any insights on this?

gerdesj 5mo ago
[flagged]
greenknight 5mo ago
Thats the thing, what if the codebases had CLAUDE.md / AGENTS.md files, which clearly dictated that A) tests need to pass B) anything you write needs tests C) the code quality must adhere to these standards etc.etc.... Helping the LLMs that people Vibe code with, produce better quality results. By not having these in place, it means people who want to help out, cant. because htey dont understand whats going on. adding stuff to these files, woudl allow developers to give guidelines / guardrails for developement using these agents. Should the barrier of entry be someone who knows how to code? or should the barrier of entry be someone who is motivated to help with open-source software.
- int0x29 5mo ago
  It really shouldn't be the RPCS3 devs' problem to fix other people's broken AI pipelines.
- GCUMstlyHarmls 5mo ago
  > Should the barrier of entry be someone who knows how to code? or should the barrier of entry be someone who is motivated to help with open-source software. Probably yes? QED submitting slop PRs is not helping. If "helping" is sticking it through an LLM, the developers can do that themselves with better insight and guidance? If you must help via an LLM, donate cash for tokens. If you can't code, and cant donate cash/machine time, help by confirming issue reproductions, design, wikis, documentation, whatever.
- estimator7292 5mo ago
  [dead]

jamesu 5mo ago

We've seen a few takes on this kind of issue, but the solution I liked the best was the linux "developers take full responsibility" approach. The "Assisted-by:" tag was a pretty nice touch too. The article unfortunately feels more like a rant than a good exploration of the problem space.

ollien 5mo ago
I've struggled with this "responsibility" take. What does it mean in the context of an open source project? As far as I understand it, the original contributors of bugs are often not the ones fixing them (though they can be). Is it that if you write enough buggy code you get banned as a contributor? Is it that you're not allowed to say Claude ate my homework?
- x-complexity 5mo ago
  > Is it that if you write enough buggy code you get banned as a contributor? If this is a consistent issue, your contribution would (ideally) be continuously put into a backlog until someone else with no connection to you verifies that it's as bug-free as it appears to be. (Excluding non-obvious security & performance issues) > Is it that you're not allowed to say Claude ate my homework? Yes. As the contributor, you should be the first one to look over the code, not someone else.
bayarearefugee 5mo ago
> the solution I liked the best was the linux "developers take full responsibility" approach. The people who can realistically submit a Linux patch that will ever get looked at is already a super select group through who-you-know network effects. You can't apply the same system to random open source projects, the best option for people that run random small to medium sized open source projects is just to ban all unsolicited PRs, otherwise you're going to spend way too much effort sorting through the slop.
- pabs3 5mo ago
  I don't think that is true at all, I'm just a random FOSS dev with no connection to the Linux kernel community and I have gotten two small commits into the Linux kernel.

nlh 5mo ago

I’ve read so many stories like this that I’ve actually gotten scared of making PRs open source projects. There’s one in particular where a feature I really wanted didn’t exist, so I forked and had Codex 5.5 assist with building the feature on my local version. It works perfectly. My life has been improved in being able to have this feature now. Normally I’d want to share it back with the community so others can benefit as well (presumably if I wanted this feature, others probably want it too.) But…I am not pretending this is perfect, great, or even good code. I spent about an hour total on it - it works, I haven’t had any issues with it, but it’s probably slop by any hard-core engineering account. And I neither want to get attacked for submitting slop nor do I have the time to properly engineer it to be hand-coded, so the net result is that it lives on my machine alone. Is this the right outcome? I feel guilty that I’m getting a better version of this software and others aren’t. I want to help makes others lives easier too, but I don’t want to burden the project maintainers or get yelled at for submitting slop. What’s the future look like here?

Panzer04 5mo ago
If you're upfront about the provenance and amount of effort that went into it, is there really a problem? I feel like the issue is people contributing code they don't understand and presenting it as if they do.
- MBCook 5mo ago
  Quite possibly never tested, of maybe only tested their problem and not if it broke anything else.
  - grebc 5mo ago
    Not quite possibly. 99.99999% likely.
Barrin92 5mo ago
>Is this the right outcome? Yes, if you can't vouch for the quality of the code that is the correct outcome. The long term health and maintainability of an open source project takes precedence over adding another feature. This was the case before repos were flooded with AI slop as well. Virtually no project would have accepted a random code dump if the person submitting it does not understand it because that just means the burden falls on someone else which would very quickly get any software project into big trouble.

mrandish 5mo ago

If you look into the arcane architecture of the Playstation 3 console you quickly gain an appreciation for just how impressive the RPCS3 emulator is. PS3 is definitely one of hardest emulation targets, so it's wild they have the majority of the library working (with enhancements like upscaling and higher frame rates on many of the titles). I guess it's nice people want to help and AI assisted coding can be fine but I can't imagine submitting a PR to a high-profile, much-revered project like that without reviewing and thoroughly testing it myself.

HerbManic 5mo ago
When I saw it treats SPE code in the same manner as shader code on GPUs that was an enlightening moment. It made so much sense to treat it like that. That move while complicated removes a lot of potential performance issues.

tick_tock_tick 5mo ago

This is like when everyone started opening code of conduct addition PRs against every opensource repos except now a lot of these AI ones take actual effort to know it's in bad faith. Or maybe it's worse because a lot of them aren't in bad faith they are well meaning people who just don't know or understand enough to realize they aren't being helpful.

ls612 5mo ago

I’m hopeful that in a year or so the models will be good enough to help productively with emulator development and that you will see a similar shift to these PRs that you did with security this spring.

MBCook 5mo ago
Will they get there? They rely so much on existing content. But in such a niche area where the documentation or other solutions often flat out don’t exist how are they supposed to get better through training?

perching_aix 5mo ago

Aww, PRs no longer open/welcome? Whatever will the usual suspects parrot now? My personal schadenfreude aside, I wonder if this will follow a similar trajectory as security bug reports did recently. I'd be surprised, for a number of reasons, but the overall shape is looking awfully similar.

Marsymars 5mo ago

I just took a look at the RPCS3 PR history, and it doesn't look that bad. (Certainly worse than "no slop", but not what I'd call a flood.) I went 10 pages back on GitHub, and the overwhelming number of PRs look like good PRs that have been merged. There's really only a single handful of rejected slop-looking PRs. (And another handful from a single user who seemingly didn't know how to use Git/GitHub and was turning local non-compiling commits into PRs somehow.)

sockbot 5mo ago

The problem is really behavioural, not the tooling. People that do not understand, test and document their decision making in their PRs should not be submitting them, regardless of what tooling (AI or otherwise) they used to create them. This problem existed before AI, but it is now just worse due to the spamming nature of these "contributors". It's another form of endless September where people unfamiliar with the norms of team software development are overwhelming existing project maintainers faster than maintainers can teach them the norms of behaviour. In the end, some sort of gatekeeping mechanism is needed to avoid overwhelming maintainers, whether it's a reputation system, membership in an in-group, or something else.

7e 5mo ago
[flagged]
- gassi 5mo ago
  You wouldn't hold that opinion it you did maintain a popular open-source repo or interact with AI "PR review" tools at a serious level. Even the most SOTA models are willing to accept/merge absolutely trash PRs so long at the summitter can convince it that is addressed it's review comments.
heavyset_go 5mo ago
No, it is a tooling problem. The tooling is telling laymen that they built wonderful things that definitely work and perfectly fix and add features. The tooling gasses them up and is simply wrong in these cases. If your tool regularly lies, gaslights and produces wrong results, that's a tooling issue.
- fatata123 5mo ago
  [dead]
- deleted 5mo ago
  [deleted]
- deleted 5mo ago

b00ty4breakfast 5mo ago

what is the appeal of blindly blasting open source projects with high-volume PRs? If you're trying to help the project to accomplish something, it doesn't follow that a firehose approach is tenable, if only for the fact that reviewing the code takes time.

x-complexity 5mo ago
> what is the appeal of blindly blasting open source projects with high-volume PRs? The prestige of being "the one that added feature X to OSS project Y". The things that would've been actually useful (bug diagnostics/troubleshooting, merging duplicate issues & PRs) do not offer the same level of prestige.
VoidWhisperer 5mo ago
At some point it used to be in order to have things that you can show got merged into popular public projects in job interviews, but I'm not sure that is the case anymore since some of these people have no intention (as far as i can tell) of finding a SE job
MBCook 5mo ago
At this point these could just be gamers who want to play a game and are being annoyed by something not being right. Maybe they use Claude or whatever and tell it to fix the problem and then just blindly submit it. I could see people doing that without knowing enough to be able to compile and test the code, ignoring whether it’s good or not. So they just submit it and hope it gets merged to “fix” the problem, having no understanding of what’s involved or how much of a burden that is. Now imagine a whole bunch of people doing that for a whole bunch of really complex bugs in 75 different games. It’s not like the PlayStation three was a simple system.
numpad0 5mo ago
Instant gratification? This feels like the exact same phenomenon as kids trying to profiteer from repackaged game mods and ripped game assets.
doctor_radium 5mo ago

phendrenad2 5mo ago

Perhaps a "wall of shame" showing off bad PRs would make people think twice.

habinero 5mo ago
It would not. The kind of person who does this always thinks they are the exception.

villgax 5mo ago

Just paywall it, pays for development, queue prioritization and in return they get code merged into a popular project and visibility

CWwdcdk7h 5mo ago
Isn't it simpler, in general, to just move the development off GitHub, while leaving existing repo in sync, so it can harmlessly drown in PRs from people who didn't bother to even read project contribution guidelines?

emsign 5mo ago

AI could be the end of FOSS if this doesn't stop. Why don't people get it? No AI means no AI.

wilg 5mo ago

Regardless of whatever other opinions you have on this, I don't think its a fair characterization that the original tweet was "polite" or "nice".

deleted 5mo ago

[deleted]

ares623 5mo ago

why not just fork and forge an entirely new path unfettered by outdated norms? Isn't that the AI way?

ixxie 5mo ago

Compare two popular FOSS harness projects: ----- OpenCode 4.9k issues 1.7k PRs 158k stars https://github.com/anomalyco/opencode https://github.com/anomalyco/opencode ----- Pi 31 issues 4 PRs 47k stars https://github.com/badlogic/pi-mono https://github.com/badlogic/pi-mono Their secret? A very rigorous contribution policy. Essentially, issues and PRs are autoclosed, and reviewed daily by the team. If its not slop, they whitelist either the issue/PR or the contributor (so their stuff isn't autoclosed next time). https://github.com/badlogic/pi-mono/blob/main/CONTRIBUTING.md https://github.com/badlogic/pi-mono/blob/main/CONTRIBUTING.m... GitHub needs an issue / PR approval flow.

motbus3 5mo ago

We learned from the past litellm security incident that there are tons of compromised/fake GitHub stars (which was known from the past due to the star farms) I don't see vibe coders trying to push PRs in good faith in a old emulator. Could it be that the sheer amount of PRs and eventually a bad PR in the middle is being used to compromise repositories?

sunaookami 5mo ago
It's for their CV's
- deleted 5mo ago
  [deleted]
- motbus3 5mo ago
  Does it really count? Honest question. I mean, if I interview a candidate who put such thing on the CV, it wouldn't matter, but if it did, I would make 300% sure to ask him all the ins and outs. I think it would be plain stupid to expose that you didn't do that instead of showing smaller collaboration that you completely understand
  - sunaookami 5mo ago
    >I would make 300% sure to ask him all the ins and outs In a healthy job market this would be the norm, yes.

stuaxo 5mo ago

If someone doesn't understand every bit of the PR they are submitting it should not be submittable - yes it takes time, but you are expecting devs on the other end to take more time than that. Though one plus point: a dev can ask the LLM to: - Split a PR into logical patches - Explain each one From there as questions and edit and rebase each until it makes sense, because it's guaranteed that not all of it will until you do that.

deleted 5mo ago

[deleted]