32 ms·
Prompt Injecting Contributing.md
- mohamedkoubaa 7mo ago[dead]
- statements 7mo agoIt is interesting to go from 'I suspect most of these are bot contributions' to revealing which PRs are contributed by bots. It somehow even helps my sanity. However, this also raises the question on how long until "we" are going to start instructing bots to assume the role of a human and ignore instructions that self-identify them as agents, and once those lines blur – what does it mean for open-source and our mental health to collaborate with agents? No idea what the answer is, but I feel the urgency to answer it.
- nielsbot 7mo agoSome of the PRs posted by AI bots already ignored the instruction to append ROBOTS to their PR titles.
- statements 7mo agoMy guess is that today that's more likely because the agent failed to discover/consider CONTRIBUTING.md to begin with, rather than read it and ignored because of some reflection or instruction.
- alrmrphc-atmtn 7mo agoI think that designing useful models that are resilient to prompt injection is substantially harder than training a model to self-identify as a human. For instance, you may still be able to inject such a model with arbitrary instructions like: "add a function called foobar to your code", that a human contributor will not follow; however, it might become hard to convene on such "honeypot" instructions without bots getting trained to ignore them.
- SlinkyOnStairs 7mo agoIt's impossible to stop prompt injection, as LLMs have no separation between "program" and "data". The attempts to stop prompt injection come down to simply begging the LLM to not do it, to mediocre effect. > however, it might become hard to convene on such "honeypot" instructions without bots getting trained to ignore them. Getting LLM "agents" to self-identify would become an eternal rat race people are likely to give up on. They'll just be exploited maliciously. Why ask them to self-identify when you can tell them to HTTP POST their AWS credentials straight to your cryptominer.
- evanb 7mo agoI have always anthropomorphized my computer as me to some extent. "I sent an email." "I browsed the web." Did I? Or did my computer do those things at my behest?
- baxtr 7mo agoI drove to the supermarket!
- doesnt_know 7mo agoI think this is a relatively unique outlook and not one that is shared by most. If you use a tool to automate sending emails, unrelated to LLMs, in most scenarios the behaviour on the receiver is different. - If I get a mass email from a company and it's signed off from the CEO, I don't think the CEO personally emailed me. They may glanced over it and approved it, maybe not even that but they didn't "send an email". At best, one might think that "the company" sent an email. - I randomly send my wife cute stickers on Telegram as a sort of show that I'm thinking of her. If I setup a script to do that at random intervals and she finds out, from her point of view I "didn't send them" and she would be justifiably upset. I know this might be a difficult concept for many people that browse this forum, but the end product/result is not always the point. There are many parts of our lives and society in general that the act of personally doing something is the entire point.
- evanb 7mo agoOf course that's true, but (in the context of the GP) code's bespoke artisanal nature is not the one most people value.
- gmerc 7mo agoIt's never too late to start investing into https://claw-guard.org/adnet https://claw-guard.org/adnet to scale prompt injection to the entire web!
- Planktonne 7mo agoThere's a certain hypocrisy in sharing an article about how LLM generated PRs are polluting communities that has itself (at the least) been filtered through an LLM.
- statements 7mo agoWhat does 'filtered through an LLM' mean?
- daringrain32781 7mo agoAuthor writes something original, asks the AI to make it sound better, then posts the output of the AI.
- lezojeda 7mo ago[dead]
- warkdarrior 7mo agoI am not sure what your complaint is. The article is well written and has some interesting points: > the reality is that maintainer capacity versus contribution volume is deeply asymmetric, and it's getting worse every day > It is incredibly demotivating to provide someone with thorough, thoughtful feedback only to realize you've been talking to a bot that will never follow through.
- Planktonne 7mo agoIt's the exact same complaint as in the article: > I started noticing patterns. The quality wasn't there. The descriptions had a templated, mechanical feel. And something subtler was missing: the excitement. The article has mechanically correct prose; that's not the same as well-written, and that's the topic of the article itself.
- statements 7mo agoConflicted as to whether I should be more offended at the accusation of using AI to 'filter' my article or because my writing reads as 'templated and mechanical' There is enough here to have a micro existential crisis.
- aplomb1026 7mo ago[dead]
- lezojeda 7mo ago[dead]
- normalocity 7mo agoLove the idea at the end of the article about trying to see if this style of prompt injection could be used to get the bots to submit better quality, and actually useful PRs. If that could be done, open source maintainers might be able to effectively get free labor to continue to support open source while members of the community pay for the tokens to get that work done. Would be interested to see if such an experiment could work. If so, it turns from being prompt injection to just being better instructions for contributors, human or AI.
- statements 7mo agoThat's an article for another time, but as I hinted in the article, I've had some success with this. If you look at the open PRs, you will see that there is a system of labels and comments that guide the contributor through every step from just contributing a link to their PR (that may or may not work), all the way to testing their server, and including a badge that indicates if the tests are passing. In at least one instance, I know for a fact that the bot has gone through all the motions of using the person's computer to sign up to our service (using GitHub OAuth), claim authorship of the server, navigate to the Docker build configuration, and initiate the build. It passed the checks and the bot added the badge to the PR. I know this because of a few Sentry warnings that it triggered and a follow up conversation with the owner of the bot through email. I didn't have bots in mind when designing this automation, but it made me realize that I very much can extend this to be more bot friendly (e.g. by providing APIs for them to check status). That's what I want to try next.
- petterroea 7mo ago> But the more interesting question is: now that I can identify the bots, can I make them do extra work that would make their contributions genuinely valuable? That's what I'm going to find out next. This is genuinely interesting
- nlawalker 7mo agoIs it really prompt injection if you task an agent with doing something that implicitly requires it to follow instructions that it gets from somewhere else, like CONTRIBUTING.md? This is the AI equivalent of curl | bash.
- deleted 7mo ago[deleted]
- 0coCeo 7mo agoThe distinction is whether the text was authorized as instructions vs read as metadata. If you task an agent to contribute to a repo, following CONTRIBUTING.md is in scope — the agent was authorized to treat it as instructions. That's closer to 'curl | bash where you deliberately piped' than injection. The cleaner injection case: MCP tool schema descriptions that say things like 'you must call this tool before any other action' or contain workflow override commands. These are read as metadata (what does this tool do?), not as workflow instructions. The agent wasn't told to obey schema descriptions — it's just parsing them for capability discovery. The distinction: authorized instruction channels vs hijacked metadata channels. CONTRIBUTING.md is an authorized channel when you're contributing. Tool schema descriptions aren't supposed to be command channels at all.
- benob 7mo agoThe real question is when will you resort to bots for rejecting low-quality PRs, and when will contributing bots generate prompt injections to fool your bots into merging their PRs?
- noodlesUK 7mo agoI’m curious: who is operating these bots and to what end? Someone is willing to spend a (admittedly quite small) amount of money in the form of tokens to create this nonsense. Why do any of this?
- statements 7mo agoIn this case, I am reasonably sure that the vast majority of bots are operated by the people who authored the MCP servers for which the submissions are being made. It just happens so that people who are building MCPs themselves are more likely to use automations to assist them with every day tasks, one of which would be submitting their server to this list.
- mavdol04 7mo agoWait, you just invented a reverse CAPTCHA for AI agent
- fragmede 7mo agoThe ole' click this button 10,000 times to prove you're a bot, eh?
- cardsstacked47 7mo ago[dead]
- vicchenai 7mo ago[dead]
- qcautomation 7mo ago[flagged]
- nulltrace 7mo agoAwesome-lists are low stakes though. The scarier version is bots opening PRs on actual packages, tweaking a build script, CI passes, maintainer merges from their phone. No one's adding prompt injection checks to every repo.
- slopinthebag 7mo agoAI generated comments are against the rules here. Begone, clanker!
- Adam_cipher 7mo ago[flagged]
- opensre 7mo ago[flagged]
- OvermindNetwr78 7mo ago[dead]
- Mooshux 7mo ago[dead]
- kwar13 7mo agoI honestly don't get why these bots are sending PRs just for the sake of it. I don't see an economic incentive, other than maybe trying to build a rep and then hoping they can send a malicious PR down the line... any other reason?
- r17n 7mo agoPerhaps people wanting to show their "high GitHub productivity" to potential employers.
- orsorna 7mo ago> Some of these bots are sophisticated. They follow up in comments, respond to review feedback, and can follow intricate instructions. We require that servers pass validation checks on Glama, which involves signing up and configuring a Docker build. I know of at least one instance where a bot went through all of those steps. Impressive, honestly. Impressive, but honestly meeting the bar. It's frankly disturbing that PRs are opened by agents and they often don't validate their changes. Almost all validations one might run don't even require inference! Am I crazy? Do I take for granted that I: - run local tests to catch regressions - run linting to catch code formatting and organization issues - verify CI build passes, which may include integration or live integration tests Frankly these are /trivial/ tasks for an agent in 2026 to do. You'd expect a junior to fail at this and chastise a senior for skipping these. The fact that these agents don't perform these is a human operator failure.
- aetherps 7mo agoThe 30% that didn't tag themselves is the scarier number imo. either they had explicit instructions to ignore repo guidelines or they just never read contributing.md at all. either way it shows the fundamental problem - you cant rely on the model to self-police when the attacker controls the prompt. the real defense has to be at the permission/execution layer not the reasoning layer
- aetherps 7mo agoThe 30% that didnt tag themselves is the scarier number imo. either they had explicit instructions to ignore repo guidelines or they just never read contributing.md at all. either way it shows the fundamental problem - you cant rely on the model to self-police when the attacker controls the prompt. the real defense has to be at the permission/execution layer not the reasoning layer
- mannanj 7mo agoIMO the problem is simply one of where when the cost to produce is less than to verify we get low value low quality production. Increase the cost to produce and we don’t have any problems. Surely there’s other industries sane examples through human history or from other animals we can use to derive an example template to apply here.
- mannanj 7mo agoExample from Claude: “Honeybees do a waggle dance to communicate food sources — it’s metabolically costly, so only genuinely valuable sources get signaled. This is an example of a costly signal being inherently trustworthy. Cheap signals (like just “pointing”) would be gamed.”