35 ms·

You did this with an AI and you do not understand what you're doing here

1178 points by redbell 1y ago

redbell 1y ago
Jump to the point: https://hackerone.com/reports/3340109#:~:text=you%20did%20this%20with%20an%20AI https://hackerone.com/reports/3340109#:~:text=you%20did%20th...
- deleted 1y ago
  [deleted]
- mrsvanwinkle 1y ago
  this LLM-emboldened, mass Dunning-Kruger schizophrenia has gone from hilarious to sad to simply invoking disgust. this isn't even an earnest altruistic effort but some insecure fever dream of finally being acknowledged as a "genius" of some sort. the worst i've seen of this is some random redditor claiming to have _the_ authoritative version of a theory of everything and spamming it in every theoretical physics adjacent subreddit, claims to have a phd but anonymous and doesn't represent any research group/institution nor does the spam have any citations.
  - scns 1y ago
    Only found a short but good article about such a case [0], i'm sure someone has bookmarked the original. There are support groups for people like this now! [0] https://www.bgnes.com/technology/chatgpt-convinced-canadian-man-he-was-a-math-genius https://www.bgnes.com/technology/chatgpt-convinced-canadian-...
    - Retr0id 1y ago
      This aspect is fascinating > The breakdown came when another chatbot — Google Gemini — told him: “The scenario you describe is an example of the ability of language models to lead convincing but completely false narratives.” Presumably, humans had already told him the same thing, but he only believed it when an AI said it. I wonder if Gemini has any kind of special training to detect these situations.
  - coldpie 1y ago
    The good news is this AI stuff is not profitable. Big companies and VCs are subsidizing all this AI slop. If it had cost this moron $5 to generate the slop to file this bug they probably would not have bothered. Hopefully the bubble bursts soon, very hard, and forces the money people to figure out how to charge for these services.
misnome 1y ago
I wonder where the balance of “Actual time saved for me” vs “Everyone else's time wasted” lies in this technological “revolution”.
- stahorn 1y ago
  You're doing it wrong: You should just feed other peoples AI-generated responses into your own AI tools and let the tool answer for you! The loop is then closed, no human time wasted, and the only effect is wasted energy to run the AI tools. It's the perfect business model to turn energy into money.
  - jsheard 1y ago
    You joke, but some companies are pushing this idea unironically by putting "use AI to expand a short message into a bloated mess" and "use AI to turn a bloated mess into a brief summary" into both sides of the same product. Good job everyone, we've invented the opposite of data compression.
    - gloxkiqcza 1y ago
      Reminded me of this - an URL lengthener: https://loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.ng/ https://looooooooooooooooooooooooooooooooooooooooooooooooooo...
    - tsimionescu 1y ago
      The next HTTP standard should include `Transfer-Encoding: polite` for AI-enabled servers and user agents.
    - taneq 1y ago
      We could call it “bsencode.
    - throwaway0236 1y ago
      Great cartoon with comment about this problem: https://marketoonist.com/2023/03/ai-written-ai-read.html https://marketoonist.com/2023/03/ai-written-ai-read.html
DetroitThrow 1y ago
There must be other corporate bounty programs they could DDOS with fake reports - doing it to curl surely won't yield much profit.
- ares623 1y ago
  This is headline driven development. Sooner or later one of these reports will make it and there will be much rejoicing.
  - baq 1y ago
    s/much rejoicing/pandora's box/ I guess. the thing is, these people aren't necessarily wrong - they're just 1) clueless 2) early. the folks with proper know-how and perhaps tuned models are probably selling zero days found this way as we speak.
    - jdefr89 1y ago
      Professional Security Researcher here.. I haven't really seen any models reliably find and exploit a 0day. Folks are are at least TRYING to develop such models internally at the MIT lab where I work, but not sure how far along they are coming yet.. If a model is developed that can find a 0day or two (like Big Sleep which I think maybe found some) I won't be surprised but keep in mind fuzzers find thousands of real 0days with far less compute... These capabilities are of course something worth looking into, but too many people are promising 0day oracles already and that simply just isn't where we are right now (or ever? ). Sorry for bad grammar typing quickly from phone here.
- nenenejej 1y ago
  Maybe using curl for RLHF training/tuning before running it on the money sites.
keyle 1y ago
Resume hit piece, <failed/>. What an absolute shamble of an industry we have ended up with.
dncornholio 1y ago
These are the people that I imagine who go on forums and threads to announce how great AI is and are unable to provide any critique. They are blinded by ignorance.
rsynnott 1y ago
This must be _absolutely exhausting_.
- zelphirkalt 1y ago
  Yeah, I guess if I was him, I would just close issues silently and ban the person who created them, if possible. I don't think I could be as nice as he is.
  - joz1-k 1y ago
    The problem is that AI can generate answers and code that look relevant and as if they were written by someone very competent. Since AI can generate a huge amount of code in a short time, it's difficult for the human brain to analyze it all and determine whether it's useful or just BS. And the worst case is when AI generates great code with a tiny, hard-to-discover catch that takes hours to spot and understand.
    - zelphirkalt 1y ago
      True, that is in some cases a problem. Though in this case it was pretty clear cut. At least the obvious time wasters would get the treatment.
  - ares623 1y ago
    Imagine the headline if a slop security report ends up real but the maintainer ignored it. It’s a lose-lose situation for the maintainers
    - xnickb 1y ago
      Thankfully in this case it's a curl vulnerability that doesn't use curl in the reproducer. That's a fairly safe call.
  - palmotea 1y ago
    > Yeah, I guess if I was him, I would just close issues silently and ban the person who created them, if possible. I don't think I could be as nice as he is. I think the shaming the use of LLMs to do stuff like this is a valuable public service.
dansmith1919 1y ago
Crazy how he doubled down by just pasting badger's answer into Chat and submitting the (hilariously obvious AI) reply: > Thanks for the quick review. You’re right — my attached PoC does not exercise libcurl and therefore does not demonstrate a cURL bug. I retract the cookie overflow claim and apologize for the noise. Please close this report as invalid. If helpful, I can follow up separately with a minimal C reproducer that actually drives libcurl’s cookie parser (e.g., via an HTTP response with oversized Set-Cookie or using CURLOPT_COOKIELIST) and reference the exact function/line in lib/cookie.c should I find an issue.
- deleted 1y ago
  [deleted]
- dansmith1919 1y ago
  At some point they told ChatGPT to put emoji's everywhere which is also a dead giveaway on the original report that it's AI. They're the new em dash.
  - badgersnake 1y ago
    Some people actually do that on Github too. Absolute psychopaths.
    - jsheard 1y ago
      I think the JS/Node scene was the pioneer in spamming emojis absolutely everywhere, well before AI. Maybe that's where the models picked it up from.
      - pjmlp 1y ago
        I really hate all those CLI applications and terminal configurations that look like circus came to town.
        henrebotha 1y ago
ttyyzz 1y ago
Over time, I've gotten a feel for what kind of content is AI-generated (e.g., images, text, and especially code...), and this text screams "AI" from top to bottom. I think badger responded very professionally; I'd be interested to see Linus Torvalds' reaction in such a situation :D
- hopelite 1y ago
  It’s interesting that you say that because besides the other perspectives on this type of matter, something I have come across is accusations of AI text that at the very least were not at all clearly AI, but also seemed like the accusation was simply a coping mechanism to deflect/evade having to accept or face new informatio/reality that was counter to one’s mental model or framework. I think of that recent situation where video showed two black bags supposedly being thrown out of a White House window. I don’t really care enough to find out whether or not that video was real, but I did find it interesting that Trump immediately dismissed it as AI after immediately glancing at it. Regardless of whether it was real or not, it seems to me that his immediate “that’s AI” response was just a rather new form of lie, a type of blame shifting to AI. I would argue that as stupid and meaningless as that kind of example is, a better response would have been something like “we will look into it” and then moving on. But it also feels like blaming AI for innocuous things preconditioned the public to deny and gaslight the public on other, more important things, e.g., for example claiming that Israel raining down bombs on civilian people in Gaza and mass murdering probably hundreds of thousands of innocent people in what looks like the start to the Terminator wars, is merely a figment of your imagination because you will be told that AI was used and AI will be scrubbed off that information so you also will never be told about it. It’s memory holed in the TelescreenAI. These types of developments don’t exactly fill me with optimism. Remember how in 1984 the war never ended, always changed, while at the same time both always existed and also did not actually exist? It feels like we are heading in that direction, the gaslighting form here on out, especially in all the forms of overt and clandestine war will be so off the charts that it will likely cause unpredictable mass “hysterias” and various undulations in societies. Most people have no idea just how much media is used to train humans like an AI would be trained or controlled, now throw in ever more believable AI generated audio, visual, and not even to mention the text slop.
rpigab 1y ago
"I heard you were extremely quick at math" Me: "yes, as a matter of fact I am" Interviewer: "Whats 14x27" Me: "49" Interviewer: "that's not even close" me: "yeah, but it was fast"
- nenenejej 1y ago
  The lowest latency responses in my load tests is when something went wrong!
- jtwaleson 1y ago
  There should be a language that uses "Almost-In-Time" compilation. If it runs out of time, it just gives a random answer.
  - phinnaeus 1y ago
    Best I can do is a system that gives you a random answer no matter how much time you give it.
    - zelphirkalt 1y ago
      Great! 80-20, Pareto principle, we're gonna use that! We are as good as done with the task. Everyone take phinnaeus as an example. This is how you get things done. We move quickly and break things. Remember our motto.
      - card_zero 1y ago
        Break things and run away, got it.
        Applejinx 1y ago
        [neddieseagoon] …and they did! [/neddieseagoon]
  - fer 1y ago
    Soft real time systems often work like that. "Can't complete in time, best I can do is X".
malux85 1y ago
Wow even the followup response apologising for noise was full of noise. It finishes "I can follow up ... blah blah blah ... should I find an issue" Tone deaf and utterly infuriating.
- teapot7 1y ago
  For me the followup was the most obviously AI bit of writing - it's exactly the tone you get when the AI admits it's been utterly wasting your time.
- nenenejej 1y ago
  Gaslighting at scale
  - EasyMark 1y ago
    GaaS, I like it!
TheSilva 1y ago
50/50 title here: it can be the app devs or it can be the reporter.
eithed 1y ago
Why not verify these reports using LLMs first?
- varjag 1y ago
  It's the same problem, false positives.
  - elzbardico 1y ago
    And false negatives too.
- elzbardico 1y ago
  Once you're at the 12th month of trying to shoehorn LLMs in several use cases at your job, you'll find the answer to this question: BECAUSE YOU CAN'T FUCKING TRUST THOSE LYING HALLUCINATING PIECES OF SHIT.
  - efreak 1y ago
    Clearly you just set an LLM to respond to messages that appear to be written by LLMs, then disregard that thread from that point on.
- deleted 1y ago
  [deleted]
joz1-k 1y ago
We will see more problems related to the attitude: "I know AI, and therefore I'm smarter than trilobites who coded this before the AI boom." I suppose there's a reason why kids are usually banned from using calculators during their first years of school when they're learning basic math.
- jennyholzer 1y ago
  I know React, and therefore I'm smarter than trilobites who coded this before the Web App boom
  - MarsIronPI 1y ago
    HN is so outdated! Let's rewrite this old legacy code in React to make it modern!
rob_c 1y ago
Same as watching someone in school try to translate between French and English by a dictionary one word at a time ignoring context... But frankly security theatre was always going to descend into this with a thousand wannabe l33ts targeting big projects with LLMs to be "that guy" who found some "bug" and "saved the world". Shellshock showed how bad a large part of the industry is. It was not a bug. "Fixing" it caused a lot of old tried and tested solutions to break, but hey, we as an industry need to protect against the lowest common denominator who refuse to learn better...
elzbardico 1y ago
I see this kind of things with new hires in my company. It is becoming depressing, stupid overly detailed but content free issue comments, stupid code that does not do what it is supposed to do but it is a fucking lot of code for you to review.
ale 1y ago
It's kind of depressing to read Daniel's article[1] on this issue given the rising "popularity" of these lazy attempts at cash grabbing. I hope they manage to combat the AI slop in a way that does not involve fighting fire with fire though. [1] https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s...
- brap 1y ago
  why don’t they just limit the report to 100 chars or something? “Here’s the input, here’s the output, here’s why it sucks”. Easy to make a maybe/no decision at a glance.
- brovonov 1y ago
  I went through some of these and the one that stood out to me was this one https://hackerone.com/reports/2823554 https://hackerone.com/reports/2823554 Where the reporter says, "Sorry didnt mean to waste anyones time Badger, I thought you would be happy about this.". People using LLMs think they are helping but in reality, they are not.
  - yifanl 1y ago
    There's this very weird idea that makes some people think that the maintainer must have a godawful workflow and if I just showed him the output of _my_ workflow, I can ~~save the day~~ fix a bug for them.
littlecranky67 1y ago
What is the motivation behind posting such things? I understand if there is a bug bounty program, does cURL have one?
- ceejayoz 1y ago
  https://curl.se/docs/bugbounty.html https://curl.se/docs/bugbounty.html
- sailorganymede 1y ago
  So you can put this on your resume: Open Source Contributor: - Diagnosed and fixed a key bug on Curl
  - netsharc 1y ago
    Hah, the opposite of "AI" meaning "Actually Indian"... "Here's my CV, but actually all my work will be done by AI". With apologies for stereotyping.
    - palmotea 1y ago
      > "Here's my CV, but actually all my work will be done by AI". What AI did you use? Because we want to hire that, not you. If AI exceeds human capabilities, it won't because it achieved "superintelligence," it will because it caused human abilities to degrade until the AI looks good in comparison.
- progbits 1y ago
  Yes they do. But I also wonder why curl seems to get so many of these. They don't have the highest payouts, have been around for long time so presumably most low hanging fruit the AI has even a remote chance of finding was fixed, and they are well known to be on the lookout and strict about AI reports.
  - jmuguy 1y ago
    Might be easier for AI to generate this specific bullshit because of curl's long history.
scosman 1y ago
Spent 15 minutes the other day testing a patch I received that claimed to fix a bug (Linux UI bug, not my forte). The “fix” was setting completely fictitious properties. Someone has plugged the GitHub issue into ChatGPT, spat out an untested answer. What’s even the point…
- dboreham 1y ago
  Ultimately it's always about someone somewhere getting a bigger boat.
- thenickdude 1y ago
  It's all in aid of some streetsweeper being able to add "contributor to X, Y, Z projects!" to their GitHub résumé. Before LLMs were a thing I also received worthless spelling-incorrection pull requests with the same aim.
  - craftkiller 1y ago
    Are spelling correction PRs not welcome? I'd never put it on a résumé but if I'm following a README and I see a typo, I'll generally open a quick PR to fix that. (no automated tools, not scanning for typos, just a human reading a README).
    - palmotea 1y ago
      > Are spelling correction PRs not welcome? I think a true spelling correction would be welcome. But I think the kind BS attitude the GP is describing often leads to useless reformatting/language tweaks, because the goal isn't to make the repo better, it's to make a change for making a change's sake with as little effort as possible.
      - HankStallone 1y ago
        Kind of like how on the Salesforce support forums, there are a lot of incorrect answers from people who don't appear to have understood the question, but they all have "Please mark my answer as helpful" at the bottom. (And this started long before AI.) If there's an incentive to "contribute," even if it's something as small as being able to put "Salesforce support volunteer" on your LinkedIn page, and it's very easy to do so, you'll get a lot of low-effort (or worse) contributions.
antiquark 1y ago
Nice ending: > The reporter was banned and now it looks like he has removed his account.
- byb 1y ago
  We are witnessing a new eternal summer and the only way to stem to tide is to increase the amount of required personal identifying information to register, and then publicly shame these people as a warning to others. Maybe it is a good thing that I don't run any massively popular open source projects.
  - ekjhgkejhgk 1y ago
    > the only way to stem to tide is I see no evidence thats the only way. Its the only way that has crossed your mind as you were writing that message.
    - byb 1y ago
      The two alternative solutions posed so far are 1) drop the ai bounty program 2) charge for submissions. Present an alternative.
      - ekjhgkejhgk 1y ago
        No, I dont have to. I never claimed I had an alternative. All I claimed was that saying theres not alternative is unsubstantiated and you proved me right, by listing those alternatives.
        antiquark 1y ago
        You're a bot, aren't you.
        byb 1y ago
        They were listed in the numerous other replies, and in the initial blog post. Both have been dismissed as undesirable/unworkable. What's your point. I'm not interested in a discussion as to whether alternatives exist, I'm interested in discussing the merits of those alternatives.
preommr 1y ago
Is there something about cUrl that attracts these AI bots, or is it just better documented by them - because I was going to say that this is old, but then I checked the date and realized that this is a new problem. Going down the rabbit-hole, @badger has made multiple posts [0][1] about AI slop. [0] https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s... [1] https://gist.github.com/bagder/07f7581f6e3d78ef37dfbfc81fd1d1cd https://gist.github.com/bagder/07f7581f6e3d78ef37dfbfc81fd1d...
- saulpw 1y ago
  Curl is one of the most widely used/deployed libraries on the planet.
- bgwalter 1y ago
  My theory is that the cURL maintainer is independent and can respond forcefully to the "AI" nonsense. Many other projects always have some corporate maintainers who are directed to push "AI" and will try to cover it up.
deleted 1y ago
[deleted]
nialse 1y ago
Imagine if these “benevolent” erroneous AI bug reports were part of a coordinated effort to map how vulnerable the projects and maintainers are, not the code. Slow response, no response is a likely target for take over or exploits, and accepting code without review is an indication of ease of injecting a vulnerability.
- panstromek 1y ago
  It's interesting idea, I just wouldn't consider slow or no response as likely target, I think that's actually a good defense strategy for spam like this.
  - nialse 1y ago
    The line of thought is that a slow response makes the time windows of an eventually found vulnerability exploit longer. Thus, increasing its value.
panstromek 1y ago
> Thanks for the quick review. You’re right — my attached PoC does not exercise libcurl and therefore does not demonstrate a cURL bug. I don't even... You just have to laugh at this I guess.
karel-3d 1y ago
This is the AI that is now writing the next version of your operating system.
- belter 1y ago
  This is the AI adding more "growth" to the US economy than all consumer spending combined. "The AI bubble is so big it's propping up the US economy" - https://www.bloodinthemachine.com/p/the-ai-bubble-is-so-big-its-propping https://www.bloodinthemachine.com/p/the-ai-bubble-is-so-big-...
  - novemp 1y ago
    This is the AI that half of HackerNews insists is "the future" that you'll be "left behind" from if you don't embrace it.
kijjure 1y ago
[flagged]
- tryauuum 1y ago
  I assure you, people don't import millions of wannabe bugbounty hunters :)
  - kijjure 1y ago
    [flagged]
- rsynnott 1y ago
  … Eh? This isn’t a person, it’s a magic robot. Or are you suggesting that use of LLMs is confined to one country? I regret to inform you that it is not.
- retired_account 1y ago
  > account created 49 minutes ago > exclusively spreading hate So, last one get banned, then?
  - krapp 1y ago
    Given the nature of every other social media platform I wonder how many of these racebaiting green accounts are themselves just AI bots. I know people copy and paste comments from AI all the time now, but someone has to be full on botting HN at this point.
  - kijjure 1y ago
    [flagged]
- AmazingTurtle 1y ago
  Idk what foolish use of AI has to do with immigrants also: reminder that someone wasted his precious time creating an account and writing this ragebait comment just for a little bit of internet visibility
vjk800 1y ago
What is the motivation for people doing this? Is it just for the lols or are they making money out of this somehow?
- punkbit 1y ago
  Possible bug bounty program.
- bxsioshc 1y ago
  I believe it's so they can put on their CV that they're contributors to XYZ famous projects.
a235 1y ago
Maintainer or curl gave recently a talk on AI slop in security reports, showing this and other examples: https://youtu.be/6n2eDcRjSsk?si=p5ay52dOhJcgQtxo https://youtu.be/6n2eDcRjSsk?si=p5ay52dOhJcgQtxo -- AI slop attacks on the curl project - Daniel Stenberg. Keynote at the FrOSCon 2025 conference, August 16, in Bonn Germany by Daniel Stenberg. Plus, linked above, his blogpost on the same subject https://daniel.haxx.se/blog/2025/08/18/ai-slop-attacks-on-the-curl-project/ https://daniel.haxx.se/blog/2025/08/18/ai-slop-attacks-on-th...
ioteg 1y ago
[flagged]
flumpcakes 1y ago
Has anyone seen a good use of AI in the wild? Every example I see is honestly depressing, such as this.
- belter 1y ago
  Its Code Generators all the way down...
- Retr0id 1y ago
  If someone is using AI effectively, there's often no way to tell that they're using AI at all. Toupée fallacy etc.
- jdefr89 1y ago
  It is best used for yack shaving in my opinion. Anything other than that and I feel like I cannot trust its output.
- EasyMark 1y ago
  Code? not much, other than small functions/classes/prototype libraries to get started, but I've often used it to figure out where code was that I was concerned with in huge project code bases and analyze where some of the edges of interfaces are without digging for a few hours. Copilot can give a decent summary of where to look in a couple of seconds instead of a half hour of marking what I think are important sections and jumping around/grepping
shoo 1y ago
last month curl developer Daniel Stenberg gave a talk "AI slop attacks on the curl project" : https://www.youtube.com/watch?v=6n2eDcRjSsk https://www.youtube.com/watch?v=6n2eDcRjSsk
weddpros 1y ago
You know what was an actual issue, that any AI would have correctly identified as an issue, but HackerOne dismissed? the 1.1.1.1 rogue certificate that later made the news...
nurettin 1y ago
Just filter messages with emojis.
dimaor 1y ago
maybe submitters should pay a dollar to submit bugs which they will get a refund for when bug is confirmed? even if not AI, there are probably many un skilled developers which submit bogus bug reports, even un knowingly.
- moi2388 1y ago
  That actually sounds like a good idea.
- andrewflnr 1y ago
  It might only need to be the first N reports from a given account. It's hard to imagine a spammer coming up with 5 legit security issues just to enable their GPT spamming operation. As long as they're not real-but-trivial typo types of issues...
PicassoCTs 1y ago
[dead]
ath3nd 1y ago
[dead]
tdeck 1y ago
This kind of thing isn't new. When I maintained a Google owned project on GitHub in the pre-LLM era someone submitted a slop PR "fixing" some tests, seemingly generated with some kind of static analysis tool. The description was clearly copy-pasted as well.
- the_biot 1y ago
  Still better than the old style reports from tools like that. They're typically commercial, and evidently came with some kind of licensing restriction that you couldn't give out their output. So open source projects would get bug reports like "my commercial static analysis tool says there's a problem in this function, but I can't tell you what the problem is."
- barnabee 1y ago
  Yep. We also saw people run any fuzzing, scanning, etc. tool they could get their hands on and pretty much just paste the results in a bug report email, well before AI was a thing. Completely useless 99% of the time but that didn’t stop a good number of them following up asking for money, sometimes quite aggressively.
hermannj314 1y ago
Start charging users to submit a vulnerability report. It doesn't matter if it made by AI or a human, spammers operate by cheaply overproducing and externalizing their work onto you to validate their shit. And it works because sometimes they do deliver value by virtue of large numbers. But they are a net negative for society. Their model stops working if they have to pay for the time they wasted.
- sealeck 1y ago
  Even a deposit works well (and doesn't have to be large). Someone who has actually found a serious bug in cURL will probably pay $2-5 dollars as a deposit to report (especially given the high probability of a payout).
  - zupa-hu 1y ago
    Exactly my thoughts. I’d love to have this for phone calls and sms as well. If you didn’t spam me, I’ll refund.
  - kg 1y ago
    I can afford it but I would never spend money to submit a vulnerability report. I'd need to be reporting dozens of vulnerabilities on a single site like hackerone to work up the motivation to plug in payment details and risk having them leaked/stolen in order to do someone else's work for them. I'd sooner click sponsor for the cURL project on github (something I already do for some OSS I use) than spend money to report a bug.
    - Analemma_ 1y ago
      That's my attitude towards this sort of thing as well, but unfortunately it seems that this attitude is unsustainable now that the cost of generating plausible-looking bullshit has been driven to 0. "Pay to prove humanity" seems like one of the only ways to keep something like this running if we don't built a hugely-invasive system of attestation.
  - SAI_Peregrinus 1y ago
throwawayExSUSE 1y ago
[flagged]
deleted 1y ago
[deleted]
spicyusername 1y ago
The amount of text alone in the original post was a giveaway. LLMs produce so much text, including code, and most of it is not needed.
- cgearhart 1y ago
  I keep talking to people who say stuff like “Claude wrote it all for me in a day”, but when I look at the code (or try it myself) it’s just so much useless code. I recently asked for Python code to parse some data into a Pandas dataframe and got 1k lines plus tests. Whatever—I’m just importing it, so let’s YOLO and see what happens. Worked like a charm in my local environment. But I wanted to share this in a Jupyter notebook and for semi-complicated reasons I couldn’t import any project-local modules in the target environment. So I asked a much more targeted question like “give me a pandas one-liner to…” and it spit out 3 lines of code that produced the same end result. The rest of that 1k lines was decomposing the problem into a bunch of auxiliary/utility functions to handle every imaginable edge case and adding comments to almost every line. It seems the current default settings for these tools is approximately the “enterprise-grade fizzbuzz” repo. Sure, I’ll get better at prompting and whatever else to reduce this problem over time, but this is not viable when the costs are being pushed onto other people in the process today.
  - yonatan8070 1y ago
    I'm using ChatGPT to generate some code for me quite often, and my instructions prompt for all chats is slowly gaining more and more ways to say "Answer shortly". And I need to prompt defensively to repeatedly tell it to only do what I tell it.
  - Anamon 1y ago
    And many of those utilities and edge cases will have been wrong or inconsistent, too. That's what the new "100x engineers" don't realise, because they never check those 1,000 lines of code they generated for themselves in a few minutes. I've made similar experiences to yours for some one-shot scripts, and once decided to actually look inside. It did stuff like writing three different validators for the same data, each called only once, each validating slightly differently, and no doubt each with their own set of subtle bugs and gotchas. These tools are intrinsically incapable of creating clean architectures and adhering to consistent standards and best practices. They are not cost-cutting or raising efficiency, they're simply very good at camouflaging the immense time costs they will cause down the line.
alexisread 1y ago
> The reporter was banned and now it looks like he has removed his account. I'm wondering (sadly) if this is a kind of defense-prodding phishing similar to the XZ utils hack, curl is a pretty fundamental utility. Similar to 419 scams, it tests the gullibility, response time/workload of the team, etc. We have an AI DDoS problem here, which may need a completely new pathway for PRs or something. Maybe Nostr based so PRs can be validated in a WOT?
- duxup 1y ago
  I see it on forums now too. On Reddit, midsized subs that get a mild amount of traffic get these brand new accounts that post what reads like an amalgam of past posts. Often in help forums where people ask questions. They have that uncanny thing where yes it's on topic, but also not how a human would likely ask exactly AND they always let slip in just a hint of human drama that really draws in other users... They almost never respond to comments, when they do it's pretty clear they're AI (much like the response in this story). I've unsubscribed from a good half dozen subs in the past few months because of it.
  - andrewflnr 1y ago
    > they always let slip in just a hint of human drama I haven't seen this so it's hard to visualize, but that seems potentially kind of tricky to do via AI. Is it actually tricky, are they donw in a way where AI could conceivably do it on its own, or are those hints easy to drop in without disturbing the bulk of the slop?
    - duxup 1y ago
      I think the AI might pick it up from the most popular / engaged with posts anyway.
    - dmiracle 1y ago
      The ones I have seen are like "my wife thinks I just need to blah but I think ..." or something
Miraltar 1y ago
This example is much worse: https://hackerone.com/reports/2298307 https://hackerone.com/reports/2298307
- jordigh 1y ago
  That's interesting. Was AI slop harder to spot in 2023? I can't remember anymore when did everything really start getting flooded with it.
- volkk 1y ago
  wow this is infuriating--from 2023 so i guess the proliferation of chatgpt's vernacular wasn't yet carved into the curl dev
- gigatree 1y ago
  Wow that’s infuriating. Fascinating watching the maintainer respond in good faith.
  - philipwhiuk 1y ago
    bagder is both extremely grumpy about the state of it and fascinatingly patient. He's like 80% wise old barn owl.
    - hersko 1y ago
      He's a pillar of the community. When i was starting out i made a basic PR to cURL to fix some typos and he was kind enough to engage and walk me through some other related changes i could add to the PR. I think he's a genuinely nice person.
    - genter 1y ago
      Here's a list of AI slop reports: https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s... I've read all of them. It's interesting how over the last 2 years badger moved from being polite to zero fucks given.
ilaksh 1y ago
I wonder if there could be some kind of platform where you have to pay a $5 deposit or something to be able to post bugs. If you waste people's time with total nonsense then you lose the $5 and can no longer report. If it's less egregious than this, like they at least made a human effort, then maybe you keep some of the deposit. Although maybe $10 or $50 would be better.
zipy124 1y ago
It is quite clear from this that a major implication of LLM's in today's society is making spam much much more difficult to discern from actual content. I empathize with any website or project popular enough to draw this kind of attention, as it must be exhausting to deal with. I wonder if burnout rates in open source will drive even higher.
AtNightWeCode 1y ago
Fun fact. I posted that post into Claude and asked if it was AI. Claude totally trashed the post.
- rchaud 1y ago
  Perhaps for your next post you could ask Claude for the definition of a "fun fact".
  - AtNightWeCode 1y ago
    Typical Claude answer to not see the irony and whine about it.
nicklevin 1y ago
It’s really quite disappointing to see how fast just copy/pasting AI responses has proliferated, even into things that don’t benefit the copy/pasters. I’m doing an online course currently that has absolutely no benefit outside of learning the content (i.e. the certificate or whatever you get for completing means nothing) - yet classmates are very clearly just copying/pasting in responses for the exercises. How does that benefit them? More than any slop I’ve experienced thus far, this instance has made me the most worried/sad/pessimistic to see. If even people who are supposedly motivated to learn (why else would you pay for this course?) just revert to the easiest AI slop path, what hope do we have for avoiding it in stuff that more resembles “work”?
2OEH8eoCRo0 1y ago
Doing this should be a stain on your career. Since anons can't be named and shamed or have careers when do we start ignoring anons? Also, if AI were so great we could trust it to review and test these CVE reports autonomously.
moomoo11 1y ago
I’ve never read something that made my blood boil and blood pressure go through the roof before lol. Fuck!! Off!!! What a professional interaction by badger. Kudos to him.
VladVladikoff 1y ago
If I was running hackerone I would add a grey list filter for any submission with emojis in it.
spacecow 1y ago
Lord, did anyone else click through and read the actual attached "POC"? It's (for now) hilariously obviously doing nothing interesting at all, but my blood runs cold at AI potentially being able to generate more plausible-looking POC code in the future to waste even more dev time...
ramon156 1y ago
On another note, I actually received a clearly GPT generated GitHub PR but eventually merged it. The changes were just doc changes but they seemed okay enough to add. I feel like the goal is to get your name on a project, but I don't really lose anything from contributions like this
deleted 1y ago
[deleted]
jmuguy 1y ago
This is essentially what teachers are dealing with every day, across the majority of their students, for every subject where its even remotely possible to use AI.
- mock-possum 1y ago
  Why not deal with it the same way teachers have always felt with students breaking the rules?
  - jmuguy 1y ago
    Wife is a high school history teacher - she would have to flunk 75% of her students. That is after proving they used AI, which would be extremely time consuming. Its very demoralizing for her, she has to spend a lot of time reading written essays generated by AI. I think given time educators will adapt. Unless they get burnt out first. She could also just not give a shit and they let go on to be some college professor's problem, who could also not give a shit, and then they become our problem when they enter the workforce.
    - simulator5g 1y ago
      Maybe this is the answer to the fermi paradox. Intelligent life eventually invents the LLM, education collapses, dumb people empowered by technology destroy the environment.
    - criddell 1y ago
      The educators will adapt. They might use AIs to grade papers written by AIs. Or, do what my kids' school did for some classes. Instead of teaching in class and then assigning homework, the homework will be reading a text book and classroom time will be spent writing essays by hand, doing exercises, answering questions, etc...
      - EasyMark 1y ago
        How does grading with AI help when the content is generated by AI? might as well just take humans out of the loop, but I don't think we're in a post scarcity society yet...
progforlyfe 1y ago
and this fucking slop is going to further pollute search engine results and future LLM models as it gets scraped up. Bleak future!
nojs 1y ago
What is the motivation for people submitting these?
smusamashah 1y ago
Once you are sure, these users should be shadow banned and an AI clone should keep them engaged. There isn't a way around it, no one deserves wasting their time on this spam.
wnevets 1y ago
Say what you want about AI but it has undeniably made aspects of life worse. Unfortunately I foresee effective bug bounty programs that are open to the public going away because of the sheer amount of spam like this.
rasz 1y ago
>printf("<unicode icon that HN seems to remove> hello LLM
rikschennink 1y ago
Recently a customer pasted a complete ChatGPT chat in the support system and then wrote “it doesn’t work” as subject. I kindly declined. I’ve also received tickets where the code snippets contained API calls that I never added to the API. A real “am I crazy” situation where I started to doubt I added it and had to double check. On top of that you get “may I get a refund” emails but expanded to four paragraphs by our friend Chat. It’s getting kinda ridiculous. Overall it’s been a huge additional time drain. I think it may be time to update the “what’s included in support” section of my softwares license agreement.
otikik 1y ago
More than half of the ads I get on Youtube these days are shovel-sellers with messages like "We have reached a point where anyone can build an app without knowing how to code". So obviously this kind of thing is going to happen. People are being encouraged by misleading marketing.
benbojangles 1y ago
I wonder how many university degrees have been passed using ai?
raffraffraff 1y ago
Verification Status: CONFIRMED bullet points Pity HN doesn't support all of those green checkboxes and bold bullet points. Every time I see these in supposedly humans generated documents and pull requests I laugh.
bogwog 1y ago
It's funny to think that the criminal underworld that trades in zerodays also has to deal with AI spam like this.
rdtsc 1y ago
I wonder what's going on in the minds of these people. I would just be terribly embarrassed and not be able to look at myself in the mirror if I did shit like this. > batuhanilgarr posted a comment (6 days ago) Thanks for the quick review. You’re right ... On one hand, it's sort of surprising that they double down, copy and paste the response to the llm prompt, paste back that response and hope for the best. But, of course it shouldn't be surprising. This is not just a mistake, it's deliberate lying and manipulating.
- strbean 1y ago
  This one is fun: https://hackerone.com/reports/2981245 https://hackerone.com/reports/2981245 > submitter: After thinking it through, I’m really sad to say that I’m not comfortable with disclosing the report . I’d prefer to keep it private . I hope this doesn’t cause any issues, and I appreciate your understanding." > bagder: I am willing to give you some time to think about your life choices, but I am going to disclose this report later. For human kind, for research, for everyone to learn. Including you. > submitter: After thinking it over, I’ve decided I’m okay with disclosing the report. Honestly, the best way for me and others to learn is by learning from our mistakes, and I think sharing this will help .
  - rdtsc 1y ago
    A good one! I like how Daniel pretended like not disclosing it was an option just to show their reaction. > "the best way for me and others to learn is by learning from our mistakes, and I think sharing this will help" I guess it worked, that's their only hackerone report they made from that account. Well, in reality the probably abandoned it, created another account and continued on with the script.
- bombcar 1y ago
  It’s a game to them, they don’t care. They likely live somewhere where a $50 beg bounty would be half a year’s work. How do you feel about pixels in a video game? That’s all the maintainer is to them.
- simulator5g 1y ago
hintymad 1y ago
This reminded me of an interview I listened to by a startup founder talking about how his company integrates AI into all of its workflows. During the Q&A, he said that they could tackle any challenge simply by iteratively constructing better contexts for the AI. At first this sounded optimistic, but then it struck me that it was actually the ultimate pessimistic view of what current AI can do. His assumption seemed to be that software engineers have already implemented all the primitives humans will ever need. If that’s true, then the only task left is to phrase our instructions in the right way so the model can stitch those primitives together into a production system.
rvz 1y ago
The emoji usage was another dead giveaway that this was done by an AI.
clearleaf 1y ago
There's a phenomenon of fraudulent "security researchers" which has sprung out of the AI world. I became aware of it when someone on discord posted a video covering an "ACE exploit" against users of a particular AI coding assistant. The exploit was this: 1. You accidentally grab a malicious config file for the assistant 2. For some reason, you would pipe this entire file into curl and then into bash 3. This results in downloading and running a script that sets up malware. It didn't make sense at any point but I was gripped by a need to know the intention such a worthless video. It made sense when the host started shilling his online course about how to be a "security researcher" like him. Not only that, paying members get premium first access to the latest "disclosures" that professional engineers are afraid to admit exist. It's likely that the creator of this bug report is building up their own repertoire of exploits that have been ignored. Or perhaps they're trying to put their course knowledge to use.
zahlman 1y ago
When I view this page without JavaScript (on my current small monitor), there is a micro-scroll vertically down to a banner which reads > It looks like your JavaScript is disabled. To use HackerOne, enable JavaScript in your browser and refresh this page. on a rgba(206, 0, 0, 0.3) background (this apparently interpolates onto pure white, so it's actually something like (240, 178, 178) ), and otherwise nothing but blank white. I know I've complained about lack of "graceful degradation" before, but this seems like a new level.
snoozeZzz 1y ago
Time pressures during sprints have started to change, and it's forcing many people to use AI for everything. So when they interview for their next role they are rusty for some tasks
tootyskooty 1y ago
I've been getting a lot of vulnerability "spam mail" recently that's clearly AI-generated. It's a surprise every public bounty program isn't completely buried in automatic reports by now, but it likely won't take long.
userbinator 1y ago
IMHO the first reply looks very automated and may even encourage them to do stuff like this, as this should've been a "fuck off" after a quick glance at the "Verified POC Code".
stuaxo 1y ago
This should be a t-shirt.
moktonar 1y ago
Given the stubbornness with which slop continued in the replies, I’m starting to doubt that this is actually part of an ongoing experiment with AI in vulnerability r&d.

1178 points by redbell 1y ago

Jump to the point: https://hackerone.com/reports/3340109#:~:text=you%20did%20this%20with%20an%20AI https://hackerone.com/reports/3340109#:~:text=you%20did%20th...

deleted 1y ago
[deleted]
mrsvanwinkle 1y ago
this LLM-emboldened, mass Dunning-Kruger schizophrenia has gone from hilarious to sad to simply invoking disgust. this isn't even an earnest altruistic effort but some insecure fever dream of finally being acknowledged as a "genius" of some sort. the worst i've seen of this is some random redditor claiming to have _the_ authoritative version of a theory of everything and spamming it in every theoretical physics adjacent subreddit, claims to have a phd but anonymous and doesn't represent any research group/institution nor does the spam have any citations.
- scns 1y ago
  Only found a short but good article about such a case [0], i'm sure someone has bookmarked the original. There are support groups for people like this now! [0] https://www.bgnes.com/technology/chatgpt-convinced-canadian-man-he-was-a-math-genius https://www.bgnes.com/technology/chatgpt-convinced-canadian-...
  - Retr0id 1y ago
    This aspect is fascinating > The breakdown came when another chatbot — Google Gemini — told him: “The scenario you describe is an example of the ability of language models to lead convincing but completely false narratives.” Presumably, humans had already told him the same thing, but he only believed it when an AI said it. I wonder if Gemini has any kind of special training to detect these situations.
- coldpie 1y ago
  The good news is this AI stuff is not profitable. Big companies and VCs are subsidizing all this AI slop. If it had cost this moron $5 to generate the slop to file this bug they probably would not have bothered. Hopefully the bubble bursts soon, very hard, and forces the money people to figure out how to charge for these services.

misnome 1y ago

I wonder where the balance of “Actual time saved for me” vs “Everyone else's time wasted” lies in this technological “revolution”.

stahorn 1y ago
You're doing it wrong: You should just feed other peoples AI-generated responses into your own AI tools and let the tool answer for you! The loop is then closed, no human time wasted, and the only effect is wasted energy to run the AI tools. It's the perfect business model to turn energy into money.
- jsheard 1y ago
  You joke, but some companies are pushing this idea unironically by putting "use AI to expand a short message into a bloated mess" and "use AI to turn a bloated mess into a brief summary" into both sides of the same product. Good job everyone, we've invented the opposite of data compression.
  - gloxkiqcza 1y ago
    Reminded me of this - an URL lengthener: https://loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.ng/ https://looooooooooooooooooooooooooooooooooooooooooooooooooo...
  - tsimionescu 1y ago
    The next HTTP standard should include `Transfer-Encoding: polite` for AI-enabled servers and user agents.
  - taneq 1y ago
    We could call it “bsencode.
  - throwaway0236 1y ago
    Great cartoon with comment about this problem: https://marketoonist.com/2023/03/ai-written-ai-read.html https://marketoonist.com/2023/03/ai-written-ai-read.html

DetroitThrow 1y ago

There must be other corporate bounty programs they could DDOS with fake reports - doing it to curl surely won't yield much profit.

ares623 1y ago
This is headline driven development. Sooner or later one of these reports will make it and there will be much rejoicing.
- baq 1y ago
  s/much rejoicing/pandora's box/ I guess. the thing is, these people aren't necessarily wrong - they're just 1) clueless 2) early. the folks with proper know-how and perhaps tuned models are probably selling zero days found this way as we speak.
  - jdefr89 1y ago
    Professional Security Researcher here.. I haven't really seen any models reliably find and exploit a 0day. Folks are are at least TRYING to develop such models internally at the MIT lab where I work, but not sure how far along they are coming yet.. If a model is developed that can find a 0day or two (like Big Sleep which I think maybe found some) I won't be surprised but keep in mind fuzzers find thousands of real 0days with far less compute... These capabilities are of course something worth looking into, but too many people are promising 0day oracles already and that simply just isn't where we are right now (or ever? ). Sorry for bad grammar typing quickly from phone here.
nenenejej 1y ago
Maybe using curl for RLHF training/tuning before running it on the money sites.

keyle 1y ago

Resume hit piece, <failed/>. What an absolute shamble of an industry we have ended up with.

dncornholio 1y ago

These are the people that I imagine who go on forums and threads to announce how great AI is and are unable to provide any critique. They are blinded by ignorance.

rsynnott 1y ago

This must be _absolutely exhausting_.

zelphirkalt 1y ago
Yeah, I guess if I was him, I would just close issues silently and ban the person who created them, if possible. I don't think I could be as nice as he is.
- joz1-k 1y ago
  The problem is that AI can generate answers and code that look relevant and as if they were written by someone very competent. Since AI can generate a huge amount of code in a short time, it's difficult for the human brain to analyze it all and determine whether it's useful or just BS. And the worst case is when AI generates great code with a tiny, hard-to-discover catch that takes hours to spot and understand.
  - zelphirkalt 1y ago
    True, that is in some cases a problem. Though in this case it was pretty clear cut. At least the obvious time wasters would get the treatment.
- ares623 1y ago
  Imagine the headline if a slop security report ends up real but the maintainer ignored it. It’s a lose-lose situation for the maintainers
  - xnickb 1y ago
    Thankfully in this case it's a curl vulnerability that doesn't use curl in the reproducer. That's a fairly safe call.
- palmotea 1y ago
  > Yeah, I guess if I was him, I would just close issues silently and ban the person who created them, if possible. I don't think I could be as nice as he is. I think the shaming the use of LLMs to do stuff like this is a valuable public service.

dansmith1919 1y ago

Crazy how he doubled down by just pasting badger's answer into Chat and submitting the (hilariously obvious AI) reply: > Thanks for the quick review. You’re right — my attached PoC does not exercise libcurl and therefore does not demonstrate a cURL bug. I retract the cookie overflow claim and apologize for the noise. Please close this report as invalid. If helpful, I can follow up separately with a minimal C reproducer that actually drives libcurl’s cookie parser (e.g., via an HTTP response with oversized Set-Cookie or using CURLOPT_COOKIELIST) and reference the exact function/line in lib/cookie.c should I find an issue.

deleted 1y ago
[deleted]
dansmith1919 1y ago
At some point they told ChatGPT to put emoji's everywhere which is also a dead giveaway on the original report that it's AI. They're the new em dash.
- badgersnake 1y ago
  Some people actually do that on Github too. Absolute psychopaths.
  - jsheard 1y ago
    I think the JS/Node scene was the pioneer in spamming emojis absolutely everywhere, well before AI. Maybe that's where the models picked it up from.
    - pjmlp 1y ago
      I really hate all those CLI applications and terminal configurations that look like circus came to town.
      - henrebotha 1y ago

ttyyzz 1y ago

Over time, I've gotten a feel for what kind of content is AI-generated (e.g., images, text, and especially code...), and this text screams "AI" from top to bottom. I think badger responded very professionally; I'd be interested to see Linus Torvalds' reaction in such a situation :D

hopelite 1y ago
It’s interesting that you say that because besides the other perspectives on this type of matter, something I have come across is accusations of AI text that at the very least were not at all clearly AI, but also seemed like the accusation was simply a coping mechanism to deflect/evade having to accept or face new informatio/reality that was counter to one’s mental model or framework. I think of that recent situation where video showed two black bags supposedly being thrown out of a White House window. I don’t really care enough to find out whether or not that video was real, but I did find it interesting that Trump immediately dismissed it as AI after immediately glancing at it. Regardless of whether it was real or not, it seems to me that his immediate “that’s AI” response was just a rather new form of lie, a type of blame shifting to AI. I would argue that as stupid and meaningless as that kind of example is, a better response would have been something like “we will look into it” and then moving on. But it also feels like blaming AI for innocuous things preconditioned the public to deny and gaslight the public on other, more important things, e.g., for example claiming that Israel raining down bombs on civilian people in Gaza and mass murdering probably hundreds of thousands of innocent people in what looks like the start to the Terminator wars, is merely a figment of your imagination because you will be told that AI was used and AI will be scrubbed off that information so you also will never be told about it. It’s memory holed in the TelescreenAI. These types of developments don’t exactly fill me with optimism. Remember how in 1984 the war never ended, always changed, while at the same time both always existed and also did not actually exist? It feels like we are heading in that direction, the gaslighting form here on out, especially in all the forms of overt and clandestine war will be so off the charts that it will likely cause unpredictable mass “hysterias” and various undulations in societies. Most people have no idea just how much media is used to train humans like an AI would be trained or controlled, now throw in ever more believable AI generated audio, visual, and not even to mention the text slop.

rpigab 1y ago

"I heard you were extremely quick at math" Me: "yes, as a matter of fact I am" Interviewer: "Whats 14x27" Me: "49" Interviewer: "that's not even close" me: "yeah, but it was fast"

nenenejej 1y ago
The lowest latency responses in my load tests is when something went wrong!
jtwaleson 1y ago
There should be a language that uses "Almost-In-Time" compilation. If it runs out of time, it just gives a random answer.
- phinnaeus 1y ago
  Best I can do is a system that gives you a random answer no matter how much time you give it.
  - zelphirkalt 1y ago
    Great! 80-20, Pareto principle, we're gonna use that! We are as good as done with the task. Everyone take phinnaeus as an example. This is how you get things done. We move quickly and break things. Remember our motto.
    - card_zero 1y ago
      Break things and run away, got it.
      - Applejinx 1y ago
        [neddieseagoon] …and they did! [/neddieseagoon]
- fer 1y ago
  Soft real time systems often work like that. "Can't complete in time, best I can do is X".

malux85 1y ago

Wow even the followup response apologising for noise was full of noise. It finishes "I can follow up ... blah blah blah ... should I find an issue" Tone deaf and utterly infuriating.

teapot7 1y ago
For me the followup was the most obviously AI bit of writing - it's exactly the tone you get when the AI admits it's been utterly wasting your time.
nenenejej 1y ago
Gaslighting at scale
- EasyMark 1y ago
  GaaS, I like it!

TheSilva 1y ago

50/50 title here: it can be the app devs or it can be the reporter.

eithed 1y ago

Why not verify these reports using LLMs first?

varjag 1y ago
It's the same problem, false positives.
- elzbardico 1y ago
  And false negatives too.
elzbardico 1y ago
Once you're at the 12th month of trying to shoehorn LLMs in several use cases at your job, you'll find the answer to this question: BECAUSE YOU CAN'T FUCKING TRUST THOSE LYING HALLUCINATING PIECES OF SHIT.
- efreak 1y ago
  Clearly you just set an LLM to respond to messages that appear to be written by LLMs, then disregard that thread from that point on.
deleted 1y ago
[deleted]

joz1-k 1y ago

We will see more problems related to the attitude: "I know AI, and therefore I'm smarter than trilobites who coded this before the AI boom." I suppose there's a reason why kids are usually banned from using calculators during their first years of school when they're learning basic math.

jennyholzer 1y ago
I know React, and therefore I'm smarter than trilobites who coded this before the Web App boom
- MarsIronPI 1y ago
  HN is so outdated! Let's rewrite this old legacy code in React to make it modern!

rob_c 1y ago

Same as watching someone in school try to translate between French and English by a dictionary one word at a time ignoring context... But frankly security theatre was always going to descend into this with a thousand wannabe l33ts targeting big projects with LLMs to be "that guy" who found some "bug" and "saved the world". Shellshock showed how bad a large part of the industry is. It was not a bug. "Fixing" it caused a lot of old tried and tested solutions to break, but hey, we as an industry need to protect against the lowest common denominator who refuse to learn better...

elzbardico 1y ago

I see this kind of things with new hires in my company. It is becoming depressing, stupid overly detailed but content free issue comments, stupid code that does not do what it is supposed to do but it is a fucking lot of code for you to review.

ale 1y ago

It's kind of depressing to read Daniel's article[1] on this issue given the rising "popularity" of these lazy attempts at cash grabbing. I hope they manage to combat the AI slop in a way that does not involve fighting fire with fire though. [1] https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s...

brap 1y ago
why don’t they just limit the report to 100 chars or something? “Here’s the input, here’s the output, here’s why it sucks”. Easy to make a maybe/no decision at a glance.
brovonov 1y ago
I went through some of these and the one that stood out to me was this one https://hackerone.com/reports/2823554 https://hackerone.com/reports/2823554 Where the reporter says, "Sorry didnt mean to waste anyones time Badger, I thought you would be happy about this.". People using LLMs think they are helping but in reality, they are not.
- yifanl 1y ago
  There's this very weird idea that makes some people think that the maintainer must have a godawful workflow and if I just showed him the output of _my_ workflow, I can ~~save the day~~ fix a bug for them.

littlecranky67 1y ago

What is the motivation behind posting such things? I understand if there is a bug bounty program, does cURL have one?

ceejayoz 1y ago
https://curl.se/docs/bugbounty.html https://curl.se/docs/bugbounty.html
sailorganymede 1y ago
So you can put this on your resume: Open Source Contributor: - Diagnosed and fixed a key bug on Curl
- netsharc 1y ago
  Hah, the opposite of "AI" meaning "Actually Indian"... "Here's my CV, but actually all my work will be done by AI". With apologies for stereotyping.
  - palmotea 1y ago
    > "Here's my CV, but actually all my work will be done by AI". What AI did you use? Because we want to hire that, not you. If AI exceeds human capabilities, it won't because it achieved "superintelligence," it will because it caused human abilities to degrade until the AI looks good in comparison.
progbits 1y ago
Yes they do. But I also wonder why curl seems to get so many of these. They don't have the highest payouts, have been around for long time so presumably most low hanging fruit the AI has even a remote chance of finding was fixed, and they are well known to be on the lookout and strict about AI reports.
- jmuguy 1y ago
  Might be easier for AI to generate this specific bullshit because of curl's long history.

scosman 1y ago

Spent 15 minutes the other day testing a patch I received that claimed to fix a bug (Linux UI bug, not my forte). The “fix” was setting completely fictitious properties. Someone has plugged the GitHub issue into ChatGPT, spat out an untested answer. What’s even the point…

dboreham 1y ago
Ultimately it's always about someone somewhere getting a bigger boat.
thenickdude 1y ago
It's all in aid of some streetsweeper being able to add "contributor to X, Y, Z projects!" to their GitHub résumé. Before LLMs were a thing I also received worthless spelling-incorrection pull requests with the same aim.
- craftkiller 1y ago
  Are spelling correction PRs not welcome? I'd never put it on a résumé but if I'm following a README and I see a typo, I'll generally open a quick PR to fix that. (no automated tools, not scanning for typos, just a human reading a README).
  - palmotea 1y ago
    > Are spelling correction PRs not welcome? I think a true spelling correction would be welcome. But I think the kind BS attitude the GP is describing often leads to useless reformatting/language tweaks, because the goal isn't to make the repo better, it's to make a change for making a change's sake with as little effort as possible.
    - HankStallone 1y ago
      Kind of like how on the Salesforce support forums, there are a lot of incorrect answers from people who don't appear to have understood the question, but they all have "Please mark my answer as helpful" at the bottom. (And this started long before AI.) If there's an incentive to "contribute," even if it's something as small as being able to put "Salesforce support volunteer" on your LinkedIn page, and it's very easy to do so, you'll get a lot of low-effort (or worse) contributions.

antiquark 1y ago

Nice ending: > The reporter was banned and now it looks like he has removed his account.

byb 1y ago
We are witnessing a new eternal summer and the only way to stem to tide is to increase the amount of required personal identifying information to register, and then publicly shame these people as a warning to others. Maybe it is a good thing that I don't run any massively popular open source projects.
- ekjhgkejhgk 1y ago
  > the only way to stem to tide is I see no evidence thats the only way. Its the only way that has crossed your mind as you were writing that message.
  - byb 1y ago
    The two alternative solutions posed so far are 1) drop the ai bounty program 2) charge for submissions. Present an alternative.
    - ekjhgkejhgk 1y ago
      No, I dont have to. I never claimed I had an alternative. All I claimed was that saying theres not alternative is unsubstantiated and you proved me right, by listing those alternatives.
      - antiquark 1y ago
        You're a bot, aren't you.
      - byb 1y ago
        They were listed in the numerous other replies, and in the initial blog post. Both have been dismissed as undesirable/unworkable. What's your point. I'm not interested in a discussion as to whether alternatives exist, I'm interested in discussing the merits of those alternatives.

preommr 1y ago

Is there something about cUrl that attracts these AI bots, or is it just better documented by them - because I was going to say that this is old, but then I checked the date and realized that this is a new problem. Going down the rabbit-hole, @badger has made multiple posts [0][1] about AI slop. [0] https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s... [1] https://gist.github.com/bagder/07f7581f6e3d78ef37dfbfc81fd1d1cd https://gist.github.com/bagder/07f7581f6e3d78ef37dfbfc81fd1d...

saulpw 1y ago
Curl is one of the most widely used/deployed libraries on the planet.
bgwalter 1y ago
My theory is that the cURL maintainer is independent and can respond forcefully to the "AI" nonsense. Many other projects always have some corporate maintainers who are directed to push "AI" and will try to cover it up.

deleted 1y ago

[deleted]

nialse 1y ago

Imagine if these “benevolent” erroneous AI bug reports were part of a coordinated effort to map how vulnerable the projects and maintainers are, not the code. Slow response, no response is a likely target for take over or exploits, and accepting code without review is an indication of ease of injecting a vulnerability.

panstromek 1y ago
It's interesting idea, I just wouldn't consider slow or no response as likely target, I think that's actually a good defense strategy for spam like this.
- nialse 1y ago
  The line of thought is that a slow response makes the time windows of an eventually found vulnerability exploit longer. Thus, increasing its value.

panstromek 1y ago

> Thanks for the quick review. You’re right — my attached PoC does not exercise libcurl and therefore does not demonstrate a cURL bug. I don't even... You just have to laugh at this I guess.

karel-3d 1y ago

This is the AI that is now writing the next version of your operating system.

belter 1y ago
This is the AI adding more "growth" to the US economy than all consumer spending combined. "The AI bubble is so big it's propping up the US economy" - https://www.bloodinthemachine.com/p/the-ai-bubble-is-so-big-its-propping https://www.bloodinthemachine.com/p/the-ai-bubble-is-so-big-...
- novemp 1y ago
  This is the AI that half of HackerNews insists is "the future" that you'll be "left behind" from if you don't embrace it.

kijjure 1y ago

[flagged]

tryauuum 1y ago
I assure you, people don't import millions of wannabe bugbounty hunters :)
- kijjure 1y ago
  [flagged]
rsynnott 1y ago
… Eh? This isn’t a person, it’s a magic robot. Or are you suggesting that use of LLMs is confined to one country? I regret to inform you that it is not.
retired_account 1y ago
> account created 49 minutes ago > exclusively spreading hate So, last one get banned, then?
- krapp 1y ago
  Given the nature of every other social media platform I wonder how many of these racebaiting green accounts are themselves just AI bots. I know people copy and paste comments from AI all the time now, but someone has to be full on botting HN at this point.
- kijjure 1y ago
  [flagged]
AmazingTurtle 1y ago
Idk what foolish use of AI has to do with immigrants also: reminder that someone wasted his precious time creating an account and writing this ragebait comment just for a little bit of internet visibility

vjk800 1y ago

What is the motivation for people doing this? Is it just for the lols or are they making money out of this somehow?

punkbit 1y ago
Possible bug bounty program.
bxsioshc 1y ago
I believe it's so they can put on their CV that they're contributors to XYZ famous projects.

a235 1y ago

Maintainer or curl gave recently a talk on AI slop in security reports, showing this and other examples: https://youtu.be/6n2eDcRjSsk?si=p5ay52dOhJcgQtxo https://youtu.be/6n2eDcRjSsk?si=p5ay52dOhJcgQtxo -- AI slop attacks on the curl project - Daniel Stenberg. Keynote at the FrOSCon 2025 conference, August 16, in Bonn Germany by Daniel Stenberg. Plus, linked above, his blogpost on the same subject https://daniel.haxx.se/blog/2025/08/18/ai-slop-attacks-on-the-curl-project/ https://daniel.haxx.se/blog/2025/08/18/ai-slop-attacks-on-th...

ioteg 1y ago

[flagged]

flumpcakes 1y ago

Has anyone seen a good use of AI in the wild? Every example I see is honestly depressing, such as this.

belter 1y ago
Its Code Generators all the way down...
Retr0id 1y ago
If someone is using AI effectively, there's often no way to tell that they're using AI at all. Toupée fallacy etc.
jdefr89 1y ago
It is best used for yack shaving in my opinion. Anything other than that and I feel like I cannot trust its output.
EasyMark 1y ago
Code? not much, other than small functions/classes/prototype libraries to get started, but I've often used it to figure out where code was that I was concerned with in huge project code bases and analyze where some of the edges of interfaces are without digging for a few hours. Copilot can give a decent summary of where to look in a couple of seconds instead of a half hour of marking what I think are important sections and jumping around/grepping

shoo 1y ago

last month curl developer Daniel Stenberg gave a talk "AI slop attacks on the curl project" : https://www.youtube.com/watch?v=6n2eDcRjSsk https://www.youtube.com/watch?v=6n2eDcRjSsk

weddpros 1y ago

You know what was an actual issue, that any AI would have correctly identified as an issue, but HackerOne dismissed? the 1.1.1.1 rogue certificate that later made the news...

nurettin 1y ago

Just filter messages with emojis.

dimaor 1y ago

maybe submitters should pay a dollar to submit bugs which they will get a refund for when bug is confirmed? even if not AI, there are probably many un skilled developers which submit bogus bug reports, even un knowingly.

moi2388 1y ago
That actually sounds like a good idea.
andrewflnr 1y ago
It might only need to be the first N reports from a given account. It's hard to imagine a spammer coming up with 5 legit security issues just to enable their GPT spamming operation. As long as they're not real-but-trivial typo types of issues...

PicassoCTs 1y ago

[dead]

ath3nd 1y ago

[dead]

tdeck 1y ago

This kind of thing isn't new. When I maintained a Google owned project on GitHub in the pre-LLM era someone submitted a slop PR "fixing" some tests, seemingly generated with some kind of static analysis tool. The description was clearly copy-pasted as well.

the_biot 1y ago
Still better than the old style reports from tools like that. They're typically commercial, and evidently came with some kind of licensing restriction that you couldn't give out their output. So open source projects would get bug reports like "my commercial static analysis tool says there's a problem in this function, but I can't tell you what the problem is."
barnabee 1y ago
Yep. We also saw people run any fuzzing, scanning, etc. tool they could get their hands on and pretty much just paste the results in a bug report email, well before AI was a thing. Completely useless 99% of the time but that didn’t stop a good number of them following up asking for money, sometimes quite aggressively.

hermannj314 1y ago

Start charging users to submit a vulnerability report. It doesn't matter if it made by AI or a human, spammers operate by cheaply overproducing and externalizing their work onto you to validate their shit. And it works because sometimes they do deliver value by virtue of large numbers. But they are a net negative for society. Their model stops working if they have to pay for the time they wasted.

sealeck 1y ago
Even a deposit works well (and doesn't have to be large). Someone who has actually found a serious bug in cURL will probably pay $2-5 dollars as a deposit to report (especially given the high probability of a payout).
- zupa-hu 1y ago
  Exactly my thoughts. I’d love to have this for phone calls and sms as well. If you didn’t spam me, I’ll refund.
- kg 1y ago
  I can afford it but I would never spend money to submit a vulnerability report. I'd need to be reporting dozens of vulnerabilities on a single site like hackerone to work up the motivation to plug in payment details and risk having them leaked/stolen in order to do someone else's work for them. I'd sooner click sponsor for the cURL project on github (something I already do for some OSS I use) than spend money to report a bug.
  - Analemma_ 1y ago
    That's my attitude towards this sort of thing as well, but unfortunately it seems that this attitude is unsustainable now that the cost of generating plausible-looking bullshit has been driven to 0. "Pay to prove humanity" seems like one of the only ways to keep something like this running if we don't built a hugely-invasive system of attestation.
- SAI_Peregrinus 1y ago

throwawayExSUSE 1y ago

[flagged]

deleted 1y ago

[deleted]

spicyusername 1y ago

The amount of text alone in the original post was a giveaway. LLMs produce so much text, including code, and most of it is not needed.

cgearhart 1y ago
I keep talking to people who say stuff like “Claude wrote it all for me in a day”, but when I look at the code (or try it myself) it’s just so much useless code. I recently asked for Python code to parse some data into a Pandas dataframe and got 1k lines plus tests. Whatever—I’m just importing it, so let’s YOLO and see what happens. Worked like a charm in my local environment. But I wanted to share this in a Jupyter notebook and for semi-complicated reasons I couldn’t import any project-local modules in the target environment. So I asked a much more targeted question like “give me a pandas one-liner to…” and it spit out 3 lines of code that produced the same end result. The rest of that 1k lines was decomposing the problem into a bunch of auxiliary/utility functions to handle every imaginable edge case and adding comments to almost every line. It seems the current default settings for these tools is approximately the “enterprise-grade fizzbuzz” repo. Sure, I’ll get better at prompting and whatever else to reduce this problem over time, but this is not viable when the costs are being pushed onto other people in the process today.
- yonatan8070 1y ago
  I'm using ChatGPT to generate some code for me quite often, and my instructions prompt for all chats is slowly gaining more and more ways to say "Answer shortly". And I need to prompt defensively to repeatedly tell it to only do what I tell it.
- Anamon 1y ago
  And many of those utilities and edge cases will have been wrong or inconsistent, too. That's what the new "100x engineers" don't realise, because they never check those 1,000 lines of code they generated for themselves in a few minutes. I've made similar experiences to yours for some one-shot scripts, and once decided to actually look inside. It did stuff like writing three different validators for the same data, each called only once, each validating slightly differently, and no doubt each with their own set of subtle bugs and gotchas. These tools are intrinsically incapable of creating clean architectures and adhering to consistent standards and best practices. They are not cost-cutting or raising efficiency, they're simply very good at camouflaging the immense time costs they will cause down the line.

alexisread 1y ago

> The reporter was banned and now it looks like he has removed his account. I'm wondering (sadly) if this is a kind of defense-prodding phishing similar to the XZ utils hack, curl is a pretty fundamental utility. Similar to 419 scams, it tests the gullibility, response time/workload of the team, etc. We have an AI DDoS problem here, which may need a completely new pathway for PRs or something. Maybe Nostr based so PRs can be validated in a WOT?

duxup 1y ago
I see it on forums now too. On Reddit, midsized subs that get a mild amount of traffic get these brand new accounts that post what reads like an amalgam of past posts. Often in help forums where people ask questions. They have that uncanny thing where yes it's on topic, but also not how a human would likely ask exactly AND they always let slip in just a hint of human drama that really draws in other users... They almost never respond to comments, when they do it's pretty clear they're AI (much like the response in this story). I've unsubscribed from a good half dozen subs in the past few months because of it.
- andrewflnr 1y ago
  > they always let slip in just a hint of human drama I haven't seen this so it's hard to visualize, but that seems potentially kind of tricky to do via AI. Is it actually tricky, are they donw in a way where AI could conceivably do it on its own, or are those hints easy to drop in without disturbing the bulk of the slop?
  - duxup 1y ago
    I think the AI might pick it up from the most popular / engaged with posts anyway.
  - dmiracle 1y ago
    The ones I have seen are like "my wife thinks I just need to blah but I think ..." or something

Miraltar 1y ago

This example is much worse: https://hackerone.com/reports/2298307 https://hackerone.com/reports/2298307

jordigh 1y ago
That's interesting. Was AI slop harder to spot in 2023? I can't remember anymore when did everything really start getting flooded with it.
volkk 1y ago
wow this is infuriating--from 2023 so i guess the proliferation of chatgpt's vernacular wasn't yet carved into the curl dev
gigatree 1y ago
Wow that’s infuriating. Fascinating watching the maintainer respond in good faith.
- philipwhiuk 1y ago
  bagder is both extremely grumpy about the state of it and fascinatingly patient. He's like 80% wise old barn owl.
  - hersko 1y ago
    He's a pillar of the community. When i was starting out i made a basic PR to cURL to fix some typos and he was kind enough to engage and walk me through some other related changes i could add to the PR. I think he's a genuinely nice person.
  - genter 1y ago
    Here's a list of AI slop reports: https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s... I've read all of them. It's interesting how over the last 2 years badger moved from being polite to zero fucks given.

ilaksh 1y ago

I wonder if there could be some kind of platform where you have to pay a $5 deposit or something to be able to post bugs. If you waste people's time with total nonsense then you lose the $5 and can no longer report. If it's less egregious than this, like they at least made a human effort, then maybe you keep some of the deposit. Although maybe $10 or $50 would be better.

zipy124 1y ago

It is quite clear from this that a major implication of LLM's in today's society is making spam much much more difficult to discern from actual content. I empathize with any website or project popular enough to draw this kind of attention, as it must be exhausting to deal with. I wonder if burnout rates in open source will drive even higher.

AtNightWeCode 1y ago

Fun fact. I posted that post into Claude and asked if it was AI. Claude totally trashed the post.

rchaud 1y ago
Perhaps for your next post you could ask Claude for the definition of a "fun fact".
- AtNightWeCode 1y ago
  Typical Claude answer to not see the irony and whine about it.

nicklevin 1y ago

It’s really quite disappointing to see how fast just copy/pasting AI responses has proliferated, even into things that don’t benefit the copy/pasters. I’m doing an online course currently that has absolutely no benefit outside of learning the content (i.e. the certificate or whatever you get for completing means nothing) - yet classmates are very clearly just copying/pasting in responses for the exercises. How does that benefit them? More than any slop I’ve experienced thus far, this instance has made me the most worried/sad/pessimistic to see. If even people who are supposedly motivated to learn (why else would you pay for this course?) just revert to the easiest AI slop path, what hope do we have for avoiding it in stuff that more resembles “work”?

2OEH8eoCRo0 1y ago

Doing this should be a stain on your career. Since anons can't be named and shamed or have careers when do we start ignoring anons? Also, if AI were so great we could trust it to review and test these CVE reports autonomously.

moomoo11 1y ago

I’ve never read something that made my blood boil and blood pressure go through the roof before lol. Fuck!! Off!!! What a professional interaction by badger. Kudos to him.

VladVladikoff 1y ago

If I was running hackerone I would add a grey list filter for any submission with emojis in it.

spacecow 1y ago

Lord, did anyone else click through and read the actual attached "POC"? It's (for now) hilariously obviously doing nothing interesting at all, but my blood runs cold at AI potentially being able to generate more plausible-looking POC code in the future to waste even more dev time...

ramon156 1y ago

On another note, I actually received a clearly GPT generated GitHub PR but eventually merged it. The changes were just doc changes but they seemed okay enough to add. I feel like the goal is to get your name on a project, but I don't really lose anything from contributions like this

deleted 1y ago

[deleted]

jmuguy 1y ago

This is essentially what teachers are dealing with every day, across the majority of their students, for every subject where its even remotely possible to use AI.

mock-possum 1y ago
Why not deal with it the same way teachers have always felt with students breaking the rules?
- jmuguy 1y ago
  Wife is a high school history teacher - she would have to flunk 75% of her students. That is after proving they used AI, which would be extremely time consuming. Its very demoralizing for her, she has to spend a lot of time reading written essays generated by AI. I think given time educators will adapt. Unless they get burnt out first. She could also just not give a shit and they let go on to be some college professor's problem, who could also not give a shit, and then they become our problem when they enter the workforce.
  - simulator5g 1y ago
    Maybe this is the answer to the fermi paradox. Intelligent life eventually invents the LLM, education collapses, dumb people empowered by technology destroy the environment.
  - criddell 1y ago
    The educators will adapt. They might use AIs to grade papers written by AIs. Or, do what my kids' school did for some classes. Instead of teaching in class and then assigning homework, the homework will be reading a text book and classroom time will be spent writing essays by hand, doing exercises, answering questions, etc...
    - EasyMark 1y ago
      How does grading with AI help when the content is generated by AI? might as well just take humans out of the loop, but I don't think we're in a post scarcity society yet...

progforlyfe 1y ago

and this fucking slop is going to further pollute search engine results and future LLM models as it gets scraped up. Bleak future!

nojs 1y ago

What is the motivation for people submitting these?

smusamashah 1y ago

Once you are sure, these users should be shadow banned and an AI clone should keep them engaged. There isn't a way around it, no one deserves wasting their time on this spam.

wnevets 1y ago

Say what you want about AI but it has undeniably made aspects of life worse. Unfortunately I foresee effective bug bounty programs that are open to the public going away because of the sheer amount of spam like this.

rasz 1y ago

>printf("<unicode icon that HN seems to remove> hello LLM

rikschennink 1y ago

Recently a customer pasted a complete ChatGPT chat in the support system and then wrote “it doesn’t work” as subject. I kindly declined. I’ve also received tickets where the code snippets contained API calls that I never added to the API. A real “am I crazy” situation where I started to doubt I added it and had to double check. On top of that you get “may I get a refund” emails but expanded to four paragraphs by our friend Chat. It’s getting kinda ridiculous. Overall it’s been a huge additional time drain. I think it may be time to update the “what’s included in support” section of my softwares license agreement.

otikik 1y ago

More than half of the ads I get on Youtube these days are shovel-sellers with messages like "We have reached a point where anyone can build an app without knowing how to code". So obviously this kind of thing is going to happen. People are being encouraged by misleading marketing.

benbojangles 1y ago

I wonder how many university degrees have been passed using ai?

raffraffraff 1y ago

Verification Status: CONFIRMED bullet points Pity HN doesn't support all of those green checkboxes and bold bullet points. Every time I see these in supposedly humans generated documents and pull requests I laugh.

bogwog 1y ago

It's funny to think that the criminal underworld that trades in zerodays also has to deal with AI spam like this.

rdtsc 1y ago

I wonder what's going on in the minds of these people. I would just be terribly embarrassed and not be able to look at myself in the mirror if I did shit like this. > batuhanilgarr posted a comment (6 days ago) Thanks for the quick review. You’re right ... On one hand, it's sort of surprising that they double down, copy and paste the response to the llm prompt, paste back that response and hope for the best. But, of course it shouldn't be surprising. This is not just a mistake, it's deliberate lying and manipulating.

strbean 1y ago
This one is fun: https://hackerone.com/reports/2981245 https://hackerone.com/reports/2981245 > submitter: After thinking it through, I’m really sad to say that I’m not comfortable with disclosing the report . I’d prefer to keep it private . I hope this doesn’t cause any issues, and I appreciate your understanding." > bagder: I am willing to give you some time to think about your life choices, but I am going to disclose this report later. For human kind, for research, for everyone to learn. Including you. > submitter: After thinking it over, I’ve decided I’m okay with disclosing the report. Honestly, the best way for me and others to learn is by learning from our mistakes, and I think sharing this will help .
- rdtsc 1y ago
  A good one! I like how Daniel pretended like not disclosing it was an option just to show their reaction. > "the best way for me and others to learn is by learning from our mistakes, and I think sharing this will help" I guess it worked, that's their only hackerone report they made from that account. Well, in reality the probably abandoned it, created another account and continued on with the script.
bombcar 1y ago
It’s a game to them, they don’t care. They likely live somewhere where a $50 beg bounty would be half a year’s work. How do you feel about pixels in a video game? That’s all the maintainer is to them.
simulator5g 1y ago

hintymad 1y ago

This reminded me of an interview I listened to by a startup founder talking about how his company integrates AI into all of its workflows. During the Q&A, he said that they could tackle any challenge simply by iteratively constructing better contexts for the AI. At first this sounded optimistic, but then it struck me that it was actually the ultimate pessimistic view of what current AI can do. His assumption seemed to be that software engineers have already implemented all the primitives humans will ever need. If that’s true, then the only task left is to phrase our instructions in the right way so the model can stitch those primitives together into a production system.

rvz 1y ago

The emoji usage was another dead giveaway that this was done by an AI.

clearleaf 1y ago

There's a phenomenon of fraudulent "security researchers" which has sprung out of the AI world. I became aware of it when someone on discord posted a video covering an "ACE exploit" against users of a particular AI coding assistant. The exploit was this: 1. You accidentally grab a malicious config file for the assistant 2. For some reason, you would pipe this entire file into curl and then into bash 3. This results in downloading and running a script that sets up malware. It didn't make sense at any point but I was gripped by a need to know the intention such a worthless video. It made sense when the host started shilling his online course about how to be a "security researcher" like him. Not only that, paying members get premium first access to the latest "disclosures" that professional engineers are afraid to admit exist. It's likely that the creator of this bug report is building up their own repertoire of exploits that have been ignored. Or perhaps they're trying to put their course knowledge to use.

zahlman 1y ago

When I view this page without JavaScript (on my current small monitor), there is a micro-scroll vertically down to a banner which reads > It looks like your JavaScript is disabled. To use HackerOne, enable JavaScript in your browser and refresh this page. on a rgba(206, 0, 0, 0.3) background (this apparently interpolates onto pure white, so it's actually something like (240, 178, 178) ), and otherwise nothing but blank white. I know I've complained about lack of "graceful degradation" before, but this seems like a new level.

snoozeZzz 1y ago

Time pressures during sprints have started to change, and it's forcing many people to use AI for everything. So when they interview for their next role they are rusty for some tasks

tootyskooty 1y ago

I've been getting a lot of vulnerability "spam mail" recently that's clearly AI-generated. It's a surprise every public bounty program isn't completely buried in automatic reports by now, but it likely won't take long.

userbinator 1y ago

IMHO the first reply looks very automated and may even encourage them to do stuff like this, as this should've been a "fuck off" after a quick glance at the "Verified POC Code".

stuaxo 1y ago

This should be a t-shirt.

moktonar 1y ago

Given the stubbornness with which slop continued in the replies, I’m starting to doubt that this is actually part of an ongoing experiment with AI in vulnerability r&d.