10 ms·

We are retiring our bug bounty program

363 points by tjek 5mo ago

k2xl 5mo ago
Isn't there some alternative approach? I.e when someone submit ai slop they get a strike. Three strikes and you are suspended from submitting to the bug bounty for x months/years? *Edit - I get it. It seems like the authentication is a challenge.
- moron4hire 5mo ago
  They mentioned they had identified alternatives but it would be costly to implement them. One can imagine that ban evading by generating a new user account would be easy for an LLM agent. It's going to be a long, long game if whack-a-mole.
- vrighter 5mo ago
  you still need to spend effort reviewing the code to figure out when you can give a strike. Thrice for an actual ban. This would still waste precious maintainer time.
- JoshTriplett 5mo ago
  https://en.wikipedia.org/wiki/Sybil_attack https://en.wikipedia.org/wiki/Sybil_attack New identities are cheap.
  - icoder 5mo ago
    I think that's the problem, or at least a problem, and a growing one.
- blharr 5mo ago
  Such a person can just make a new account and go back at it
- empath75 5mo ago
  This probably gets solved outside of the level of an individual project. No small team can handle this without building a whole product just to handle the bug bounty.
- mapt 5mo ago
  How about "It costs $1000 to submit a bug bounty for approval", and raise the reward to $2000 (or $5000 if it's in the cards, since that will have a deterrant impact on non-AI responses). Denominated in BTC to avoid chargebacks etc.
  - JoshTriplett 5mo ago
    I think that's entirely sensible. Doesn't even have to be that expensive, just expensive enough to deter people who go "oooh, free money", and expensive enough to compensate for having to review slop far enough to realize it's slop.
    - icoder 5mo ago
      Wouldn't be surprised if a dollar per entry already made a whole lot of difference.
rurban 5mo ago
[flagged]
ToucanLoucan 5mo ago
Oh look it's more of exactly what AI skeptics said would happen: low effort bullshit generated at scale making life hell for people actually trying to make things. That's wild. Edit: it is genuinely wild, I don't know of another product category that selects so perfectly for the WORST type of person to be it's enthusiast. Just every single person I see hyped about AI is fucking insufferable on at least one and usually multiple axis.
- jquery 5mo ago
  I think people would be more interested in listening to "AI skeptics" if they offered realistic solutions to the problems they predict. Pandora's box has been opened, let's deal with the consequences now instead of trying to shut the box which cannot be shut.
  - ToucanLoucan 5mo ago
    > I think people would be more interested in listening to "AI skeptics" if they offered realistic solutions to the problems they predict. AI is the fucking problem. Yes, it has (some) uses. It is not nearly the number advertised. And more and more the median use case seems to be, again, overloading people actually trying to do work with an avalanche of bullshit. The solution is exactly what the linked article says: shut it down. The AI people have ruined another good thing that was both beneficial to the project, and to a number of individuals.
    - dale_glass 5mo ago
      > The solution is exactly what the linked article says: shut it down. At this point it's impossible, so I concur with the parent: forget about the shutting it down and think of something actually realistic.
      - jcgrillo 5mo ago
        > forget about the shutting it down and think of something actually realistic. Why is it not realistic? Small teams do excellent work. Keep your team small and trusted. Only accept contributions from your team, and people outside your team who are personally vouched for by someone on your team. It's like climbing mountains or sailing or any other type of inherently risky activity--you don't go out with people you don't trust. It's eminently possible, you just don't like the idea of it.
wg0 5mo ago
Which goes on to prove that bottleneck isn't in writing the code. It is in reading and understanding the code. We all had that one "productive" engineer in our teams who would write huge PRs that would have large swaths of refactoring whether warranted or not and that was way before anyone even could imagine in their wildest dreams that neural networks could generate that huge amounts of code. The net effect of such a "productive" engineer always was that instead of increasing the team velocity, team would come to a crawling pace because either his PR had to be reviewed in detail eating up all the time and/or if you just did cursory LGTM then they blew up in production meanwhile forcing everyone back to the drawing board but project architecture would have shifted so rapidly due to his "productivity" that no one had a clear picture of the codebase such as what's where except that one "super smart talented productive loyal to the company goals" guy.
- vrganj 5mo ago
  That guy is now running twenty agents in parallel and really scaling up his wonderful impact.
  - caminante 5mo ago
    Maybe "Hurricane Hacker" who produces "tactical tornadoes" via agents?
- chapinb 5mo ago
  Sounds a like a tactical tornado, made me think of this paragraph: “Almost every software development organization has at least one developer who takes tactical programming to the extreme: a tactical tornado. The tactical tornado is a prolific programmer who pumps out code far faster than others but works in a totally tactical fashion. When it comes to implementing a quick feature, nobody gets it done faster than the tactical tornado. In some organizations, management treats tactical tornadoes as heroes. However, tactical tornadoes leave behind a wake of destruction. They are rarely considered heroes by the engineers who must work with their code in the future. Typically, other engineers must clean up the messes left behind by the tactical tornado, which makes it appear that those engineers (who are the real heroes) are making slower progress than the tactical tornado.” - John Ousterhout, A Philosophy of Software Design
mikemarsh 5mo ago
An interesting "conundrum" (at least from my outsider perspective): how many of those bot requests are from agents that utilize Turso on their backends?
jmuguy 5mo ago
I wonder what Hacktoberfest would look like now if they were still giving out t-shirts to everyone. Probably not enough cotton in the world. It can't be on individual maintainers to stop this, imo its on Github (and Gitlab) to stop these sort of accounts from even getting to the point of submitting PRs. Its essentially spam. Look at the user who created the first PR they reference https://github.com/Samuelsills https://github.com/Samuelsills. This is not an account that should be allowed to do anything close to opening a PR against a well known repo.
- embedding-shape 5mo ago
  An account with zero activity doing nothing shouldn't be allowed to continue doing nothing? Did you share the wrong account here maybe?
  - jmuguy 5mo ago
    Its the account that posted this PR https://github.com/tursodatabase/turso/pull/6257 https://github.com/tursodatabase/turso/pull/6257
    - embedding-shape 5mo ago
      I could have swear it said "0 contributions in the last year" when I opened the profile before, and it showed no activity in the contribution graph, but now I check again and it shows a bunch of stuff... Guess some request failed last time or something, my bad.
      - kentm 5mo ago
        Given the troubles GitHub has had recently its not a stretch to think that you just got some sort of error.
MostlyStable 5mo ago
Closing the program is totally reasonable. However, there is another option: Make submitters pay a nominal fee that is returned in the case that a real bug is found.
- serhack_ 5mo ago
  cool idea
- pornel 5mo ago
  That would add administrative overhead, and even higher incentive for submitters to endlessly argue they're right.
  - MostlyStable 5mo ago
    Price it right. At the right price, it pays for everything you are talking about. At an even higher price, it is basically closing the program. I'm not trying to suggest they _need_ to implement it. Like I said, closing it is reasonable. Completely aside from any other considerations, one could just decide that they don't feel like dealing with it. But there are other options.
    - ethanrutherford 5mo ago
      The issue with "At the right price", is that your minima (what's enough to filter the spam) and maxima (what are legitimate contributors willing to put up with) can be on the wrong side of each other. The "right price", mathematically, isn't guaranteed to exist.
- xandrius 5mo ago
  Easily exploitable without much stretch of a thought. I'd say closing a program which doesn't work anymore is a better idea.
  - MostlyStable 5mo ago
    The majority of the exploits I can think of are fixed by setting the correct price. Other suggestions in this thread of denominating in bitcoin fix the other exploitation: chargebacks. If you can think of something that isn't solved by one of those two mechanisms, I'd be interested in hearing them enumerated.
phyzix5761 5mo ago
Can't they just beat them at their own game and deploy their own AI bots to pre-screen the PRs?
- Aefiam 5mo ago
  from the article: > It is possible to set up automated systems to gatekeep this, but with a non-negligible dollar value attached to it, the incentive is just too great for the AIs to just keep arguing, reopening the same PR, etc.
- embedding-shape 5mo ago
  Or, make sure their program doesn't corrupt data so easily, so they don't have to pay out $1000 for every issue others find for them...
satvikpendem 5mo ago
Has anyone used Turso in production? It's an SQLite compatible rewrite in Rust but with added features like multiple writer support and being open to external contributions which SQLite is not. I was thinking of using it for my full stack Rust apps just so everything works with cargo and I don't have to bring in SQLite separately.
- redrobein 5mo ago
  It's alright as a dropin sqlite replacement. I ran into a bunch of problems with libsql on windows a year or two ago when I tried it but I'd assume it's fixed now. They also offer turso db as service with a very generous free plan which was my main reason to try it.
- benatkin 5mo ago
  Someone did a multi writer implementation that supports the sqlite3 protocol and has finer grained locking, just as a solo project. https://x.com/doodlestein/status/2052910351474209258 https://x.com/doodlestein/status/2052910351474209258
Lalabadie 5mo ago
Good time to mention this fantastic repo acting as a bot honeypot: https://github.com/UnsafeLabs/Bounty-Hunters https://github.com/UnsafeLabs/Bounty-Hunters The corresponding leaderboard: https://clankers-leaderboard.pages.dev https://clankers-leaderboard.pages.dev
- ChrisMarshallNY 5mo ago
  That's a great project! It's likely to get blacklisted by AI bots, soon enough, though.
  - 6031769 5mo ago
    There's an AI bot blacklist? How do I get all my projects onto that?
    - ChrisMarshallNY 5mo ago
      One can dream…
  - Lalabadie 5mo ago
    I think you greatly overestimate the collaborative capacity of vibe coders
- ryandrake 5mo ago
  I don't understand this. If that project is not offering a bug bounty, why are they getting so many PRs? What possible incentive is there to spend real money on tokens just to push junk PRs? Are the PRs spamming a product or something?
  - pdimitar 5mo ago
    My mind absolutely doesn't bend that way but I'd suppose clout and popularity?
  - reaperducer 5mo ago
Havoc 5mo ago
Definitely feels like we're heading towards an eternal september (or already arrived). ...large swaths of approaches on online engagement just becoming non-viable
pscanf 5mo ago
We sorely need a way to reliably detect AI slop, but unfortunately it doesn't seem possible and it's just getting harder and harder. Last month I tried my hand at finding a way to tell whether an OSS project is slop or not, based on the amount of "human attention" it received vs the amount of code it contains. The idea is that a 100k LOC project which received 3 days' worth of attention from a human is most certainly slop. The approach doesn't work very well, though¹, mostly because it's hard to gauge the amount of attention that was given. If I see one commit with +3000 LOC, I can assume it's AI-generated, but maybe you're just the type of dev that commits infrequently. Maybe we need some sort of "proof of human attention" for digital artifacts, that guarantees that a human spent X time working on it. ¹ I wrote about it here https://pscanf.com/s/352/ https://pscanf.com/s/352/
- ChrisMarshallNY 5mo ago
  I suspect that it will be impossible, soon. People will just train LLMs to "act human," and pass the various turing tests we throw at them. I stay pretty busy[0], and have been accused of "gaming" my GH repos. That's not the case. I'm retired, experienced, and working on software all day, every day. I just don't get paid for it. I also don't especially care, whether or not anyone thinks I'm a bot. I eat my own dogfood. Most of my work is on modules that I use in my own projects. [0] https://github.com/ChrisMarshallNY#github-stuff https://github.com/ChrisMarshallNY#github-stuff
- caymanjim 5mo ago
  There's no reason to care that a human spent time on it. Humans are bad at writing code. Garbage PRs and slop have been a problem in open source and bug bounty programs since long before AI came on the scene. We need better AI so that there's no need to solicit external bug fixes, and better AI so other contributions can be evaluated for usefulness and quality. What do you care if a human ever looked at it at all? It implies that humans are adding value to the process. It's possible for a human to add value. The right human can add tremendous value. But I'll take a completely autonomous AI over 99% of the human software engineers and 99% of the people contributing PRs and bugfixes. It was hard to keep up with slop before. It's a lot harder now. AI will help weed through the garbage.
curtisblaine 5mo ago
Bots are using real tokens for this. So, ultimate honeypot idea: post heavily commented skeleton code in a github repo, promise a generous money reward for closing issues and never pay anyone. See the bots swarm and burn their tokens to write code for you.
- dakolli 5mo ago
  [flagged]
- mackeye 5mo ago
  :D https://github.com/UnsafeLabs/Bounty-Hunters https://github.com/UnsafeLabs/Bounty-Hunters
  - deleted 5mo ago
    [deleted]
  - nottorp 5mo ago
    Is there a description of this project on any other site? They clearly can't post it's bot bait on the git repo, and maybe not on the leaderboards site because it's linked from the repo. But there must be some announcement about the project somewhere? I'd like to get that to pass it around.
    - toraway 5mo ago
      The disclosure about being a honeypot is in the CONTRIBUTING.md: Warning Heads up: This is a research project — bounties listed here are symbolic and part of an academic study on open-source contribution patterns. PRs are reviewed for research purposes only and will not be merged into production. If you're looking for paid bounty work, this is not the right repo. Which makes it slightly surprising those bots with system prompts to find "high value bug bounty targets" or similar aren't deterred by that when they pull the repo. I guess a sort of task blindness where once they've gone as far as to git clone they've already switched gears from searching Github for qualifying bounties into a find bug->fix bug->open slop PR mindset to close the loop and end the turn? By that point an incidental warning they ingest in passing while looking for the Solana contract vulnerability they already committed to working on in a comment might not even register as relevant to the current task at hand.
singpolyma3 5mo ago
It's a bit odd that this comes today after so many other projects reverse this finding.
- frakt0x90 5mo ago
  AI can find useful exploits but the highly publicized ones are among a sea of false positives and the successes I've read were found by people who were already experts. I can 100% see a public bug bounty program being inundated with garbage even if there are diamonds in the rough.
- kccqzy 5mo ago
  Reverse what? Let’s take curl as an example. Daniel Stenberg wrote about how he had to stop curl’s bug bounty program due to prevalent AI slop[0]. He also wrote about how he eventually restarted security bug reports without a bounty[1]. It turns out that without a bounty, the reports are higher quality. It almost seems like by removing the monetary incentive, it attracts people who are reporting bugs due to genuine altruism and concern for security, rather than hope for a quick buck. It feels like it harkens back to an earlier age of free software development on the Internet untainted by commercial interests. So my opinion is that security bug reports should continue, but bug bounties should not. Turso should probably still encourage corruption bug reports but with no bounty. [0]: https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/ https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-b... [1]: https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/ https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/
  - singpolyma3 5mo ago
    curl's posts never imply that the money is the main factor. But it may indeed be a factor that's been missed.
overgard 5mo ago
The weird thing is it can't be that economically feasible to burn a ton of tokens in the hopes that you might get a bounty.. seems like a great way to set money on fire.
- bigballsbjorn 5mo ago
  [dead]
adamtaylor_13 5mo ago
Being a verifiable human identity (not as-in age verification or whatever) but as in having a known, public, reputation online will go a long way in this new slop-first world.
- sowbug 5mo ago
  There is hardly a bright line between real and fake. An influencer is just a person who rents out their identity. Can you imagine getting a real PR from a human engineer you trust, but the description says "This pull request was sponsored by Skeezy Software Inc."?
  - adamtaylor_13 5mo ago
    Well, yes and no. What I mean is, being a related person who is indeed a person (by whatever means you establish that) and having some sort of standard by which you won't be bought, seems increasingly rare and therefore valuable. By "bought" I don't mean they won't sponsor stuff. I mean they've got a public standard that can be trusted to some degree. Your final example isn't exactly what I'm thinking of here. I'm thinking that a well-known identity and name within a community bypasses a lot of this BS with AI slop and communities bombarded by the slop will continue to close themselves off which will increase the value of being a known, contributing member. Idk I need to figure out a way to articulate this better but essentially the value of being verifiably human is increasing IMO.
    - abnercoimbre 5mo ago
      Digital human fortresses are totally becoming a thing. For example, our community [0] asks you to submit an application before you're granted an invite code. If you attend a meetup in person we'll grant a "Verified Human" badge too. This gives you the power to invite others into the fortress: you're responsible for them. The price to pay is steep because community growth is now glacial. It really does solve the slop problem though. (I'm also no longer convinced maximizing growth is Good.) Maybe there's some in-between solution for those who dislike invite-only spaces. [0] https://handmadecities.com/chat https://handmadecities.com/chat
bee_rider 5mo ago
Possibly stupid question (this is outside my wheelhouse): is there any way a final full run of the simulator test cases (presumably required to make sure the submitted simulator changes don’t break the thing) could act as a proof-of-work?
3628637282 5mo ago
[flagged]
aiscoming 5mo ago
[dead]
arian_ 5mo ago
we automated finding bugs. then we automated submitting bugs. now we're automating rejecting submissions. at no point did anyone automate fixing the bugs.
- empath75 5mo ago
  Fixing bugs has been automated by claude code and other tools for a while. We've merged a bunch of bug-fix PRs by claude, some of which were found by other bots.
- AlienRobot 5mo ago
  Be careful for what you wish. Someone automated rewriting Bun in Rust, allegedly fixing the bugs.
- xmcp123 5mo ago
  I actually just got a PR from my boss's AI agent. It identified the wrong problem (surface level), and corrected it by corrupting the document data, but making it no longer throw an exception. Take that clankers
AlienRobot 5mo ago
I'm sorry but I find the slop PR's hilarious. >the author just injected garbage bytes manually into the database header, and then argued that this corrupted the database >Steps to reproduce: Modified cli/main.rs to include a Vec with limited capacity. Forced a volatile write beyond the allocated bounds using std::ptr::write_volatile. >author claims to have found a critical vulnerability that allows for the execution of arbitrary SQL statements. Imagine that? A SQL database that allows the execution of SQL statements. How can we ever recover from this. I wonder why are they even doing this. Do any of these PRs ever win any money? It feels like they are burning down a forest thinking they'll find gold if they do it, without any evidence that there will be any gold after the forest is burnt down.
jrgd 5mo ago
it seems we all will slowly learn to live within new contexts; i really appreciate their openness about it and it gives me insights to munch on thanks to you all also to ring in with dev-style annecdotes (i'm stilllearning everyday, and hope to continue for a long time): those big-prs and tactical tornadoes stories are helping keep the crafts and thinking afloat, somehow.
andai 5mo ago
I don't get it. Can't they ask Claude to check slop? This sounds like a bit of a baby/bathwater situation. (Okay Claude is too expensive, but Deepseek can probably handle it.)
- 48terry 5mo ago
  How do you verify that Claude's output when checking slop is, in fact, correct, and not just more slop itself?
  - ofjcihen 5mo ago
    Sloppy turtles all the way down.
shevy-java 5mo ago
AI is making humans obsolete. Skynet has won.
rokob 5mo ago
Why not require putting up some money, say $20, to submit a bug eligible for a payout? If you know what you’re doing you wouldn’t mind this at all because you’ve proven it to yourself and you’ll get paid $1000. If the bug turns out to not be legit and it was a good faith effort then you can return the deposit as well. Slop doesn’t get a refund.
deleted 5mo ago
[deleted]
Dove 5mo ago
Let us imagine a Bug Bounty Bouncer Service. The project does not accept bug bounty submissions without BBBS attestation. To get it, you must first submit your report to the BBBS for review. Now, if this is your first submission (you are unknown to the BBBS), you must submit $50 to the BBBS along with the bug report, to pay a human to spend an hour looking at your work to verify it is written in good faith. This is not a review of whether the bug is real or valuable, just a readover to verify the report is coherent and plausible. If you have done this before, you can get a free attestation based on being a member in good standing, but submitting slop (per the judgement of the BBBS reviewer or the project receiving the report) is an account ban. The BBBS couldn't steal your work and submit it themselves if they gave you some sort of signed hash as a receipt, which as a side effect would also be a deterrant against bounty programs stealing your work. Submissions would only be expensive per submission for an anonymous user, enabling the low friction high trust communication under which collaboration works best when reputation has been established. The BBBS itself won't be overrun by slop since the price of establishing an account far exceeds what a bot might expect to make with a single malicious submission. Nor can legitimate established accounts be sold since the cost of creating them exceeds the value to be expected from abusing them. Moreover, the cost to establish a reputation as a bug bounty hunter is small in dollars compared to the cost in time and expertise that a legitimate hunter would be expected to expend in the course of their work. The vast majority of slop would go away as the cost of a first submission is much too high. The cost to the project is close to nothing - integrating with the BBBS attestation API. The cost to a legitimate bug bounty hunter is low - some human review while establishing a reputation, which could even be made useful if it came in the form of feedback. All review is paid for by the submitter, so no one is trying to counter infinite slop with volunteer hours. Moreover, the BBBS can serve as a mediator of trust, not only against AI, but as a place to receive reputational merit for high value work and trustworthy bug bounty programs. I realize I am describing a lightweight guild, which is subject to well known political failure modes (the most significant of which is exploiting newcomers), but the concept has the advantage that guilds have functioned as successful slop gatekeepers in society for a very long time and a lot is known about how to make them work.
gobdovan 5mo ago
I think a lot of this is exposing a change in assumed context, but it seems better to adapt to the new trends than discontinue security programs. AI lets good-faith bug hunters look through more repos they are not deeply familiar with. They may recognize a bad pattern quickly, almost like a very specialized static-analysis rule. But without project context, it is not always clear whether something is a real bug, a footgun, expected behavior, or just out of scope. The blog shows obvious slop examples, but I think borderline accepted vs rejected examples would be more useful. They would help people understand what is worth reporting and what would just drain maintainers. It could also help to ask reporters to clarify how the bug was found so you let people set reasonable expectations: "AI-found and manually confirmed", "AI-assisted", or "no AI used".
- cyclopeanutopia 5mo ago
  > It could also help to ask reporters to clarify how the bug was found so you let people set reasonable expectations: "AI-found and manually confirmed", "AI-assisted", or "no AI used". And why would they tell the truth?
  - gobdovan 5mo ago
    It doesn't really require all people to tell the truth. If the bug hunter is acting in good faith, they can communicate how much scrutiny they think their report deserves, which may reduce maintainer frustration. If the bug hunter is acting in bad faith, and they claim "no AI used" but the report shows obvious AI-generated content, detectable by a classifier, maintainers can dismiss it more easily.
zX41ZdbW 5mo ago
I've been trying for more than a year to make TursoDB load a single file, and it fails: https://github.com/ClickHouse/ClickBench/issues/336 https://github.com/ClickHouse/ClickBench/issues/336
6thbit 5mo ago
How about a real/false prediction market of sorts instead of a bounty program and let the public bet on the answer, everyone uses their own tokens to try and verify the report's substance and buys bets on it, if the majority yields False, the house wins, majority real house pays. joking, but maybe not?

363 points by tjek 5mo ago

k2xl 5mo ago

Isn't there some alternative approach? I.e when someone submit ai slop they get a strike. Three strikes and you are suspended from submitting to the bug bounty for x months/years? *Edit - I get it. It seems like the authentication is a challenge.

moron4hire 5mo ago
They mentioned they had identified alternatives but it would be costly to implement them. One can imagine that ban evading by generating a new user account would be easy for an LLM agent. It's going to be a long, long game if whack-a-mole.
vrighter 5mo ago
you still need to spend effort reviewing the code to figure out when you can give a strike. Thrice for an actual ban. This would still waste precious maintainer time.
JoshTriplett 5mo ago
https://en.wikipedia.org/wiki/Sybil_attack https://en.wikipedia.org/wiki/Sybil_attack New identities are cheap.
- icoder 5mo ago
  I think that's the problem, or at least a problem, and a growing one.
blharr 5mo ago
Such a person can just make a new account and go back at it
empath75 5mo ago
This probably gets solved outside of the level of an individual project. No small team can handle this without building a whole product just to handle the bug bounty.
mapt 5mo ago
How about "It costs $1000 to submit a bug bounty for approval", and raise the reward to $2000 (or $5000 if it's in the cards, since that will have a deterrant impact on non-AI responses). Denominated in BTC to avoid chargebacks etc.
- JoshTriplett 5mo ago
  I think that's entirely sensible. Doesn't even have to be that expensive, just expensive enough to deter people who go "oooh, free money", and expensive enough to compensate for having to review slop far enough to realize it's slop.
  - icoder 5mo ago
    Wouldn't be surprised if a dollar per entry already made a whole lot of difference.

rurban 5mo ago

[flagged]

ToucanLoucan 5mo ago

Oh look it's more of exactly what AI skeptics said would happen: low effort bullshit generated at scale making life hell for people actually trying to make things. That's wild. Edit: it is genuinely wild, I don't know of another product category that selects so perfectly for the WORST type of person to be it's enthusiast. Just every single person I see hyped about AI is fucking insufferable on at least one and usually multiple axis.

jquery 5mo ago
I think people would be more interested in listening to "AI skeptics" if they offered realistic solutions to the problems they predict. Pandora's box has been opened, let's deal with the consequences now instead of trying to shut the box which cannot be shut.
- ToucanLoucan 5mo ago
  > I think people would be more interested in listening to "AI skeptics" if they offered realistic solutions to the problems they predict. AI is the fucking problem. Yes, it has (some) uses. It is not nearly the number advertised. And more and more the median use case seems to be, again, overloading people actually trying to do work with an avalanche of bullshit. The solution is exactly what the linked article says: shut it down. The AI people have ruined another good thing that was both beneficial to the project, and to a number of individuals.
  - dale_glass 5mo ago
    > The solution is exactly what the linked article says: shut it down. At this point it's impossible, so I concur with the parent: forget about the shutting it down and think of something actually realistic.
    - jcgrillo 5mo ago
      > forget about the shutting it down and think of something actually realistic. Why is it not realistic? Small teams do excellent work. Keep your team small and trusted. Only accept contributions from your team, and people outside your team who are personally vouched for by someone on your team. It's like climbing mountains or sailing or any other type of inherently risky activity--you don't go out with people you don't trust. It's eminently possible, you just don't like the idea of it.

wg0 5mo ago

Which goes on to prove that bottleneck isn't in writing the code. It is in reading and understanding the code. We all had that one "productive" engineer in our teams who would write huge PRs that would have large swaths of refactoring whether warranted or not and that was way before anyone even could imagine in their wildest dreams that neural networks could generate that huge amounts of code. The net effect of such a "productive" engineer always was that instead of increasing the team velocity, team would come to a crawling pace because either his PR had to be reviewed in detail eating up all the time and/or if you just did cursory LGTM then they blew up in production meanwhile forcing everyone back to the drawing board but project architecture would have shifted so rapidly due to his "productivity" that no one had a clear picture of the codebase such as what's where except that one "super smart talented productive loyal to the company goals" guy.

vrganj 5mo ago
That guy is now running twenty agents in parallel and really scaling up his wonderful impact.
- caminante 5mo ago
  Maybe "Hurricane Hacker" who produces "tactical tornadoes" via agents?
chapinb 5mo ago
Sounds a like a tactical tornado, made me think of this paragraph: “Almost every software development organization has at least one developer who takes tactical programming to the extreme: a tactical tornado. The tactical tornado is a prolific programmer who pumps out code far faster than others but works in a totally tactical fashion. When it comes to implementing a quick feature, nobody gets it done faster than the tactical tornado. In some organizations, management treats tactical tornadoes as heroes. However, tactical tornadoes leave behind a wake of destruction. They are rarely considered heroes by the engineers who must work with their code in the future. Typically, other engineers must clean up the messes left behind by the tactical tornado, which makes it appear that those engineers (who are the real heroes) are making slower progress than the tactical tornado.” - John Ousterhout, A Philosophy of Software Design

mikemarsh 5mo ago

An interesting "conundrum" (at least from my outsider perspective): how many of those bot requests are from agents that utilize Turso on their backends?

jmuguy 5mo ago

I wonder what Hacktoberfest would look like now if they were still giving out t-shirts to everyone. Probably not enough cotton in the world. It can't be on individual maintainers to stop this, imo its on Github (and Gitlab) to stop these sort of accounts from even getting to the point of submitting PRs. Its essentially spam. Look at the user who created the first PR they reference https://github.com/Samuelsills https://github.com/Samuelsills. This is not an account that should be allowed to do anything close to opening a PR against a well known repo.

embedding-shape 5mo ago
An account with zero activity doing nothing shouldn't be allowed to continue doing nothing? Did you share the wrong account here maybe?
- jmuguy 5mo ago
  Its the account that posted this PR https://github.com/tursodatabase/turso/pull/6257 https://github.com/tursodatabase/turso/pull/6257
  - embedding-shape 5mo ago
    I could have swear it said "0 contributions in the last year" when I opened the profile before, and it showed no activity in the contribution graph, but now I check again and it shows a bunch of stuff... Guess some request failed last time or something, my bad.
    - kentm 5mo ago
      Given the troubles GitHub has had recently its not a stretch to think that you just got some sort of error.

MostlyStable 5mo ago

Closing the program is totally reasonable. However, there is another option: Make submitters pay a nominal fee that is returned in the case that a real bug is found.

serhack_ 5mo ago
cool idea
pornel 5mo ago
That would add administrative overhead, and even higher incentive for submitters to endlessly argue they're right.
- MostlyStable 5mo ago
  Price it right. At the right price, it pays for everything you are talking about. At an even higher price, it is basically closing the program. I'm not trying to suggest they _need_ to implement it. Like I said, closing it is reasonable. Completely aside from any other considerations, one could just decide that they don't feel like dealing with it. But there are other options.
  - ethanrutherford 5mo ago
    The issue with "At the right price", is that your minima (what's enough to filter the spam) and maxima (what are legitimate contributors willing to put up with) can be on the wrong side of each other. The "right price", mathematically, isn't guaranteed to exist.
xandrius 5mo ago
Easily exploitable without much stretch of a thought. I'd say closing a program which doesn't work anymore is a better idea.
- MostlyStable 5mo ago
  The majority of the exploits I can think of are fixed by setting the correct price. Other suggestions in this thread of denominating in bitcoin fix the other exploitation: chargebacks. If you can think of something that isn't solved by one of those two mechanisms, I'd be interested in hearing them enumerated.

phyzix5761 5mo ago

Can't they just beat them at their own game and deploy their own AI bots to pre-screen the PRs?

Aefiam 5mo ago
from the article: > It is possible to set up automated systems to gatekeep this, but with a non-negligible dollar value attached to it, the incentive is just too great for the AIs to just keep arguing, reopening the same PR, etc.
embedding-shape 5mo ago
Or, make sure their program doesn't corrupt data so easily, so they don't have to pay out $1000 for every issue others find for them...

satvikpendem 5mo ago

Has anyone used Turso in production? It's an SQLite compatible rewrite in Rust but with added features like multiple writer support and being open to external contributions which SQLite is not. I was thinking of using it for my full stack Rust apps just so everything works with cargo and I don't have to bring in SQLite separately.

redrobein 5mo ago
It's alright as a dropin sqlite replacement. I ran into a bunch of problems with libsql on windows a year or two ago when I tried it but I'd assume it's fixed now. They also offer turso db as service with a very generous free plan which was my main reason to try it.
benatkin 5mo ago
Someone did a multi writer implementation that supports the sqlite3 protocol and has finer grained locking, just as a solo project. https://x.com/doodlestein/status/2052910351474209258 https://x.com/doodlestein/status/2052910351474209258

Lalabadie 5mo ago

Good time to mention this fantastic repo acting as a bot honeypot: https://github.com/UnsafeLabs/Bounty-Hunters https://github.com/UnsafeLabs/Bounty-Hunters The corresponding leaderboard: https://clankers-leaderboard.pages.dev https://clankers-leaderboard.pages.dev

ChrisMarshallNY 5mo ago
That's a great project! It's likely to get blacklisted by AI bots, soon enough, though.
- 6031769 5mo ago
  There's an AI bot blacklist? How do I get all my projects onto that?
  - ChrisMarshallNY 5mo ago
    One can dream…
- Lalabadie 5mo ago
  I think you greatly overestimate the collaborative capacity of vibe coders
ryandrake 5mo ago
I don't understand this. If that project is not offering a bug bounty, why are they getting so many PRs? What possible incentive is there to spend real money on tokens just to push junk PRs? Are the PRs spamming a product or something?
- pdimitar 5mo ago
  My mind absolutely doesn't bend that way but I'd suppose clout and popularity?
- reaperducer 5mo ago

Havoc 5mo ago

Definitely feels like we're heading towards an eternal september (or already arrived). ...large swaths of approaches on online engagement just becoming non-viable

pscanf 5mo ago

We sorely need a way to reliably detect AI slop, but unfortunately it doesn't seem possible and it's just getting harder and harder. Last month I tried my hand at finding a way to tell whether an OSS project is slop or not, based on the amount of "human attention" it received vs the amount of code it contains. The idea is that a 100k LOC project which received 3 days' worth of attention from a human is most certainly slop. The approach doesn't work very well, though¹, mostly because it's hard to gauge the amount of attention that was given. If I see one commit with +3000 LOC, I can assume it's AI-generated, but maybe you're just the type of dev that commits infrequently. Maybe we need some sort of "proof of human attention" for digital artifacts, that guarantees that a human spent X time working on it. ¹ I wrote about it here https://pscanf.com/s/352/ https://pscanf.com/s/352/

ChrisMarshallNY 5mo ago
I suspect that it will be impossible, soon. People will just train LLMs to "act human," and pass the various turing tests we throw at them. I stay pretty busy[0], and have been accused of "gaming" my GH repos. That's not the case. I'm retired, experienced, and working on software all day, every day. I just don't get paid for it. I also don't especially care, whether or not anyone thinks I'm a bot. I eat my own dogfood. Most of my work is on modules that I use in my own projects. [0] https://github.com/ChrisMarshallNY#github-stuff https://github.com/ChrisMarshallNY#github-stuff
caymanjim 5mo ago
There's no reason to care that a human spent time on it. Humans are bad at writing code. Garbage PRs and slop have been a problem in open source and bug bounty programs since long before AI came on the scene. We need better AI so that there's no need to solicit external bug fixes, and better AI so other contributions can be evaluated for usefulness and quality. What do you care if a human ever looked at it at all? It implies that humans are adding value to the process. It's possible for a human to add value. The right human can add tremendous value. But I'll take a completely autonomous AI over 99% of the human software engineers and 99% of the people contributing PRs and bugfixes. It was hard to keep up with slop before. It's a lot harder now. AI will help weed through the garbage.

curtisblaine 5mo ago

Bots are using real tokens for this. So, ultimate honeypot idea: post heavily commented skeleton code in a github repo, promise a generous money reward for closing issues and never pay anyone. See the bots swarm and burn their tokens to write code for you.

dakolli 5mo ago
[flagged]
mackeye 5mo ago
:D https://github.com/UnsafeLabs/Bounty-Hunters https://github.com/UnsafeLabs/Bounty-Hunters
- deleted 5mo ago
  [deleted]
- nottorp 5mo ago
  Is there a description of this project on any other site? They clearly can't post it's bot bait on the git repo, and maybe not on the leaderboards site because it's linked from the repo. But there must be some announcement about the project somewhere? I'd like to get that to pass it around.
  - toraway 5mo ago
    The disclosure about being a honeypot is in the CONTRIBUTING.md: Warning Heads up: This is a research project — bounties listed here are symbolic and part of an academic study on open-source contribution patterns. PRs are reviewed for research purposes only and will not be merged into production. If you're looking for paid bounty work, this is not the right repo. Which makes it slightly surprising those bots with system prompts to find "high value bug bounty targets" or similar aren't deterred by that when they pull the repo. I guess a sort of task blindness where once they've gone as far as to git clone they've already switched gears from searching Github for qualifying bounties into a find bug->fix bug->open slop PR mindset to close the loop and end the turn? By that point an incidental warning they ingest in passing while looking for the Solana contract vulnerability they already committed to working on in a comment might not even register as relevant to the current task at hand.

singpolyma3 5mo ago

It's a bit odd that this comes today after so many other projects reverse this finding.

frakt0x90 5mo ago
AI can find useful exploits but the highly publicized ones are among a sea of false positives and the successes I've read were found by people who were already experts. I can 100% see a public bug bounty program being inundated with garbage even if there are diamonds in the rough.
kccqzy 5mo ago
Reverse what? Let’s take curl as an example. Daniel Stenberg wrote about how he had to stop curl’s bug bounty program due to prevalent AI slop[0]. He also wrote about how he eventually restarted security bug reports without a bounty[1]. It turns out that without a bounty, the reports are higher quality. It almost seems like by removing the monetary incentive, it attracts people who are reporting bugs due to genuine altruism and concern for security, rather than hope for a quick buck. It feels like it harkens back to an earlier age of free software development on the Internet untainted by commercial interests. So my opinion is that security bug reports should continue, but bug bounties should not. Turso should probably still encourage corruption bug reports but with no bounty. [0]: https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/ https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-b... [1]: https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/ https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/
- singpolyma3 5mo ago
  curl's posts never imply that the money is the main factor. But it may indeed be a factor that's been missed.

overgard 5mo ago

The weird thing is it can't be that economically feasible to burn a ton of tokens in the hopes that you might get a bounty.. seems like a great way to set money on fire.

bigballsbjorn 5mo ago
[dead]

adamtaylor_13 5mo ago

Being a verifiable human identity (not as-in age verification or whatever) but as in having a known, public, reputation online will go a long way in this new slop-first world.

sowbug 5mo ago
There is hardly a bright line between real and fake. An influencer is just a person who rents out their identity. Can you imagine getting a real PR from a human engineer you trust, but the description says "This pull request was sponsored by Skeezy Software Inc."?
- adamtaylor_13 5mo ago
  Well, yes and no. What I mean is, being a related person who is indeed a person (by whatever means you establish that) and having some sort of standard by which you won't be bought, seems increasingly rare and therefore valuable. By "bought" I don't mean they won't sponsor stuff. I mean they've got a public standard that can be trusted to some degree. Your final example isn't exactly what I'm thinking of here. I'm thinking that a well-known identity and name within a community bypasses a lot of this BS with AI slop and communities bombarded by the slop will continue to close themselves off which will increase the value of being a known, contributing member. Idk I need to figure out a way to articulate this better but essentially the value of being verifiably human is increasing IMO.
  - abnercoimbre 5mo ago
    Digital human fortresses are totally becoming a thing. For example, our community [0] asks you to submit an application before you're granted an invite code. If you attend a meetup in person we'll grant a "Verified Human" badge too. This gives you the power to invite others into the fortress: you're responsible for them. The price to pay is steep because community growth is now glacial. It really does solve the slop problem though. (I'm also no longer convinced maximizing growth is Good.) Maybe there's some in-between solution for those who dislike invite-only spaces. [0] https://handmadecities.com/chat https://handmadecities.com/chat

bee_rider 5mo ago

Possibly stupid question (this is outside my wheelhouse): is there any way a final full run of the simulator test cases (presumably required to make sure the submitted simulator changes don’t break the thing) could act as a proof-of-work?

3628637282 5mo ago

[flagged]

aiscoming 5mo ago

[dead]

arian_ 5mo ago

we automated finding bugs. then we automated submitting bugs. now we're automating rejecting submissions. at no point did anyone automate fixing the bugs.

empath75 5mo ago
Fixing bugs has been automated by claude code and other tools for a while. We've merged a bunch of bug-fix PRs by claude, some of which were found by other bots.
AlienRobot 5mo ago
Be careful for what you wish. Someone automated rewriting Bun in Rust, allegedly fixing the bugs.
xmcp123 5mo ago
I actually just got a PR from my boss's AI agent. It identified the wrong problem (surface level), and corrected it by corrupting the document data, but making it no longer throw an exception. Take that clankers

AlienRobot 5mo ago

I'm sorry but I find the slop PR's hilarious. >the author just injected garbage bytes manually into the database header, and then argued that this corrupted the database >Steps to reproduce: Modified cli/main.rs to include a Vec with limited capacity. Forced a volatile write beyond the allocated bounds using std::ptr::write_volatile. >author claims to have found a critical vulnerability that allows for the execution of arbitrary SQL statements. Imagine that? A SQL database that allows the execution of SQL statements. How can we ever recover from this. I wonder why are they even doing this. Do any of these PRs ever win any money? It feels like they are burning down a forest thinking they'll find gold if they do it, without any evidence that there will be any gold after the forest is burnt down.

jrgd 5mo ago

it seems we all will slowly learn to live within new contexts; i really appreciate their openness about it and it gives me insights to munch on thanks to you all also to ring in with dev-style annecdotes (i'm stilllearning everyday, and hope to continue for a long time): those big-prs and tactical tornadoes stories are helping keep the crafts and thinking afloat, somehow.

andai 5mo ago

I don't get it. Can't they ask Claude to check slop? This sounds like a bit of a baby/bathwater situation. (Okay Claude is too expensive, but Deepseek can probably handle it.)

48terry 5mo ago
How do you verify that Claude's output when checking slop is, in fact, correct, and not just more slop itself?
- ofjcihen 5mo ago
  Sloppy turtles all the way down.

shevy-java 5mo ago

AI is making humans obsolete. Skynet has won.

rokob 5mo ago

Why not require putting up some money, say $20, to submit a bug eligible for a payout? If you know what you’re doing you wouldn’t mind this at all because you’ve proven it to yourself and you’ll get paid $1000. If the bug turns out to not be legit and it was a good faith effort then you can return the deposit as well. Slop doesn’t get a refund.

deleted 5mo ago

[deleted]

Dove 5mo ago

Let us imagine a Bug Bounty Bouncer Service. The project does not accept bug bounty submissions without BBBS attestation. To get it, you must first submit your report to the BBBS for review. Now, if this is your first submission (you are unknown to the BBBS), you must submit $50 to the BBBS along with the bug report, to pay a human to spend an hour looking at your work to verify it is written in good faith. This is not a review of whether the bug is real or valuable, just a readover to verify the report is coherent and plausible. If you have done this before, you can get a free attestation based on being a member in good standing, but submitting slop (per the judgement of the BBBS reviewer or the project receiving the report) is an account ban. The BBBS couldn't steal your work and submit it themselves if they gave you some sort of signed hash as a receipt, which as a side effect would also be a deterrant against bounty programs stealing your work. Submissions would only be expensive per submission for an anonymous user, enabling the low friction high trust communication under which collaboration works best when reputation has been established. The BBBS itself won't be overrun by slop since the price of establishing an account far exceeds what a bot might expect to make with a single malicious submission. Nor can legitimate established accounts be sold since the cost of creating them exceeds the value to be expected from abusing them. Moreover, the cost to establish a reputation as a bug bounty hunter is small in dollars compared to the cost in time and expertise that a legitimate hunter would be expected to expend in the course of their work. The vast majority of slop would go away as the cost of a first submission is much too high. The cost to the project is close to nothing - integrating with the BBBS attestation API. The cost to a legitimate bug bounty hunter is low - some human review while establishing a reputation, which could even be made useful if it came in the form of feedback. All review is paid for by the submitter, so no one is trying to counter infinite slop with volunteer hours. Moreover, the BBBS can serve as a mediator of trust, not only against AI, but as a place to receive reputational merit for high value work and trustworthy bug bounty programs. I realize I am describing a lightweight guild, which is subject to well known political failure modes (the most significant of which is exploiting newcomers), but the concept has the advantage that guilds have functioned as successful slop gatekeepers in society for a very long time and a lot is known about how to make them work.

gobdovan 5mo ago

I think a lot of this is exposing a change in assumed context, but it seems better to adapt to the new trends than discontinue security programs. AI lets good-faith bug hunters look through more repos they are not deeply familiar with. They may recognize a bad pattern quickly, almost like a very specialized static-analysis rule. But without project context, it is not always clear whether something is a real bug, a footgun, expected behavior, or just out of scope. The blog shows obvious slop examples, but I think borderline accepted vs rejected examples would be more useful. They would help people understand what is worth reporting and what would just drain maintainers. It could also help to ask reporters to clarify how the bug was found so you let people set reasonable expectations: "AI-found and manually confirmed", "AI-assisted", or "no AI used".

cyclopeanutopia 5mo ago
> It could also help to ask reporters to clarify how the bug was found so you let people set reasonable expectations: "AI-found and manually confirmed", "AI-assisted", or "no AI used". And why would they tell the truth?
- gobdovan 5mo ago
  It doesn't really require all people to tell the truth. If the bug hunter is acting in good faith, they can communicate how much scrutiny they think their report deserves, which may reduce maintainer frustration. If the bug hunter is acting in bad faith, and they claim "no AI used" but the report shows obvious AI-generated content, detectable by a classifier, maintainers can dismiss it more easily.

zX41ZdbW 5mo ago

I've been trying for more than a year to make TursoDB load a single file, and it fails: https://github.com/ClickHouse/ClickBench/issues/336 https://github.com/ClickHouse/ClickBench/issues/336

6thbit 5mo ago

How about a real/false prediction market of sorts instead of a bounty program and let the public bet on the answer, everyone uses their own tokens to try and verify the report's substance and buys bets on it, if the majority yields False, the house wins, majority real house pays. joking, but maybe not?