10 ms·
Curl: We still have not seen a valid security report done with AI help
- ivape 1y ago[flagged]
- _se 1y agoFortunately, they can beat an automated idiot agent, which is much closer to what we have today.
- ivape 1y ago[flagged]
- jsheard 1y agoThis very thread is about LLMs hallucinating too much today. The curl dev is reporting a zero percent hit rate on automated bug reports in TYOOL 2025. You could say that's an example of the toupee fallacy, but nobody is showing up to defend the AI tools they used for valid curl bug reports either.
- surgical_fire 1y agoThey still hallucinate (generate bullshit) too much. Nothing much to be done there, it is part of what they are. They are still useful as code assistant. But the sort of overhype that you push for is actually detrimental to its healthy development.
- alabastervlog 1y agoI'm still waiting to see an LLM that is actually helpful versus the alternatives, outside a very small set of tasks that's basically the same as "2 years ago".
- HPsquared 1y agoThe word "intelligent" is doing a lot of heavy lifting here. Theoretically yes, but maybe we're not there yet.
- yoyohello13 1y agoDid you actually read the objection? They are basically getting DDoS by spurious vuln reports. How do you "get over" that.
- deleted 1y ago[deleted]
- ivape 1y ago[flagged]
- deleted 1y ago[deleted]
- gipp 1y agoYou're moving into delusional territory here. He is not being "exhaustively probed," he is being "constantly spammed by confidently incorrect nonsense."
- vel0city 1y agoClearly the answer is to have an AI assistant handle all the reports from bogus AI auditors. It's all turtles, all the way down.
- ivape 1y ago[flagged]
- surgical_fire 1y agoNothing intelligent about that loop. But a loop it is.
- vel0city 1y agoCool, so I'll just script something that costs you $0.03/submission but costs me $$0.000001 per submission. We'll see who wins at the end! > It's an intelligent loop Highly debatable if most submissions are bogus bullshit.
- devmor 1y agoYour statement is proven wrong by the article you are replying to.
- jacksnipe 1y agoSomething that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!
- x3n0ph3n3 1y agoThanks for this. It's a great response I intend to use going forward.
- deleted 1y ago[deleted]
- esafak 1y agoI had to deal with someone who tried to check in hallucinated code with the defense "I checked it with chatGPT!" If you're just parroting what you read, what is it that you do here?!
- unsnap_biceps 1y agoFor those of you who don't want to click into linked in, https://hackerone.com/reports/3125832 https://hackerone.com/reports/3125832 is the latest example of a invalid curl report
- nneonneo 1y agoGood god did they hallucinate the segmentation fault and the resulting GDB trace too? Given that the diffs don’t even apply and the functions don’t even exist, I guess the answer is yes - in which case, this is truly a new low for AI slop bug reports.
- bluGill 1y agoAn real report would have a GDB trace that looks like that, so it isn't hard to create such a trace. Many of us could create a real looking GDB trace just as well by hand - it would be tedious, boring, and pointless but we could.
- nneonneo 1y agoOh, I'm fully aware an LLM can hallucinate a GDB trace just fine. My complaint is: if you're trying to use an AI to help you find bugs, you'd sincerely hope that they would have *some* attempt to actually run the exploit. Having the LLM invent fake evidence that you have done so, when you haven't, is just evil, and should be resulting in these people being kicked straight off H1 completely.
- bluGill 1y agoThat means doing work. I can get a llm to write up a bugus report in minutes and then whatever value comes frome it. Checking the report is real would take time.
- terom 1y agoThe git commit hashes in the diff are interesting: 1a2b3c4..d4e5f6a I think my wetware pattern-matching brain spots a pattern there.
- parliament32 1y agoDidn't even have to click through to the report in question to know it would be all hallucinations -- both the original patchfile and the segfault ("ngtcp2_http3_handle_priority_frame".. "There is no function named like this in current ngtcp2 or nghttp3.") I guess these guys don't bother to verify, they just blast out AI slop and hope one of them hits?
- indigodaddy 1y agoReminds me of when some LLM (might have been Deepseek) told me I could add wasm_mode=True in my FastHTML python code which would allow me to compile it to WebAssembly, when of course there is no such feature in FastHTML. This was even when I had provided it full llms-ctx.txt
- alabastervlog 1y agoI had Google's in-search "AI" invent a command line switch that would have been very helpful... if it existed. Complete with usage caveats and warnings! This was like two weeks ago. These things suck.
- j_w 1y agoMy favorite is when their in search "AI answer" hallucinates on the Golang standard lib. Always makes me happy to see.
- hedora 1y agoYou think that's funny? Try using AI help button in Google's office suite the next time you're trying to track down the right button to press.
- sidewndr46 1y agoIsn't there a website that builds git man pages this way? By just stringing together random concepts into sentences that seem vaguely like something Git would implement. I thought it was silly and potentially harmful the first time I saw it. Apparently, it may have just been ahead of the curve.
- hx8 1y agoIt's probably a net positive that ChatGPT isn't going around detecting zero day vulnerabilities. We should really be saving those for the state actors to find.
- vessenes 1y agoReading the straw that broke the camel's back commit illustrates the problem really well: https://hackerone.com/reports/3125832 https://hackerone.com/reports/3125832 . This shit must be infuriating to dig through. I wonder if reputation systems might work here - you could give anyone who id's with an AML/KYC provider some reputation, enough for two or three reports, let people earn reputation digging through zero rep submissions and give someone like 10,000 reputation for each accurate vulnerability found, and 100s for any accurate promoted vulnerabilities. This would let people interact anonymously if they want to edit, quickly if they found something important and are willing to AML/KYC, and privilege quality people. Either way, AI is definitely changing economics of this stuff, in this case enshittifying first.
- emushack 1y agoReputation systems for this kind of thing sounds like rubbing some anti-itch cream on bullet wound. I feel like the problem seems to me to be behavior, not a technology issue. Personally I can't imagine how miserable it would be for my hard-earned expertise to be relegated to sifting through SLOP where maybe 1 in hundreds or even thousands of inquiries is worth any time at all. But it also doesn't seem prudent to just ignore them. I don't think better ML/AI technology or better information systems will make a significant difference on this issue. It's fundamentally about trust in people.
- delusional 1y agoI consider myself a left leaning soyboy, but this could be the outcome of too "nice" of a discourse. I won't advocate for toxicity, but I am considering if we bolster the self-image of idiots when we refuse to call them idiots. Because you're right, this is fundamentally a people problem, specifically we need people to filter this themselves. I don't know where the limit would go.
- orthecreedence 1y agoShame is a useful social tool. It can be overused or underused, but it's still a tool and people like this should be made to publicly answer for their obnoxious and destructive behavior.
- deleted 1y ago[deleted]
- uludag 1y agoI can imagine that most LLMs, if you ask it to find a security vulnerability in a given piece of code, will make something up completely out of the air. I've (mistakenly) sent valid code with an unrelated error and to this day I get nonsense "fixes" for these errors. This alignment problem between responding with what the user wants (e.g. a security report, flattering responses) and going against the user seems a major problem limiting the effectiveness of such systems.
- rdtsc 1y ago> evilginx updated the severity from none to high Well the reporter in the report that stated it that they are open for employment https://hackerone.com/reports/3125832 https://hackerone.com/reports/3125832 Anyone want to hire them? They can play with ChatGPT all day and spam random projects with the AI slop.
- bogwog 1y agoIf I wanted to slip a vulnerability into a major open source project with a lot of eyes on it, using AI to DDOS their vulnerability reports so they're less likely to find a real report from someone who caught me seems like an obvious (and easy) step. Looking at one of the bogus reports, it doesn't even seem like a real person. Why do this if you're not trying to gain recognition?
- jsheard 1y ago> Why do this if you're not trying to gain recognition? They're doing it for money, a handful of their reports did result in payouts. Those reports aren't public though, so there's no way to know if they actually found real bugs or the reviewer rubber-stamped them without doing their due diligence.
- 0x500x79 1y agoIt should be called "Denial of Attention" attack!
- zulban 1y agoShame they need to put up with that spam. However, every big open source project has by now had good contributions with "AI help". Many millions of developers are using AI a little as a tool, like Google.
- eestrada 1y agoAnd that increase in LLM usage has resulted in an enormous increase of code duplications and code churn in said open source projects. Any benefit from new features implemented by LLMs is being offset by the tech debt caused by duplication and the maintenance burden of constantly reverting bad code (i.e. churn). https://arc.dev/talent-blog/impact-of-ai-on-code/ https://arc.dev/talent-blog/impact-of-ai-on-code/
- zulban 1y agoYes. The internet has also created a ton of email spam but I wouldn't say "we've never seen a single valid contribution to our project that had internet help". Many millions of developers are using AI. Sometimes in a good way. When that results in a good MR, they likely don't even mention they used Google, or stackoverflow, or AI, they just submit.
- Analemma_ 1y agoI mean, I certainly would say “I’ve never seen a single commercial email that was valid and useful to me as a customer”, and this is entirely because of spam. Any unsolicited email with commercial intent goes instantly, reflexively, to the trash (plus whatever my spam filters prevent me from ever seeing to begin with). This presumably has cost me the opportunity to purchase things I genuinely would’ve found useful, and reduced the effectiveness of well-meaning people doing cold outreach for actually-good products, but spam has left me no choice. In that sense, it has destroyed actual value as the noise crowds out the signal. AI could easily do the same to, like, all Internet communication.
- marcosdumay 1y agoIf they never got a valid contribution to their project through the internet, yes, they would say exactly that. They don't say it because the internet provides actual value.
- molticrystal 1y agoThere is or at various times was, nitter for twitter, Invidious for youtube, Imginn for instagram, and even many variations of ones for hackernews like hckrnews.com & ones that are lighter, work better in terminals, etc. Anything for linkedin, a light interface that doesn't required logging in? I pretty much stopped going to linkedin years ago because they started aggressively directing a person to login. I was shocked this post works without login. I don't know if that is how it has always been, or if that is a recent change, or what. It would be nice to have alternative interfaces. In case some people are getting gated here is their post: === Daniel Stenberg curl CEO. Code Emitting Organism That's it. I've had it. I'm putting my foot down on this craziness. 1. Every reporter submitting security reports on #Hackerone for #curl now needs to answer this question: "Did you use an AI to find the problem or generate this submission?" (and if they do select it, they can expect a stream of proof of actual intelligence follow-up questions) 2. We now ban every reporter INSTANTLY who submits reports we deem AI slop. A threshold has been reached. We are effectively being DDoSed. If we could, we would charge them for this waste of our time. We still have not seen a single valid security report done with AI help. --- This is the latest one that really pushed me over the limit: https://hackerone.com/reports/3125832 https://hackerone.com/reports/3125832 ===
- perching_aix 1y ago> Anything for linkedin, a light interface that doesn't required logging in? I just opened the site with JS off on mobile. No issues.
- Marsymars 1y agoI don’t think there exists any alternative frontend for LinkedIn. LinkedIn actually just lack week started demanding I upload ID to be able to log in…... which I’m not going to do, so LinkedIn content is effectively inaccessible to me even with an account.
- ianbutler 1y agoCounterpoint we have a CVE attributable to ours and I suspect the difference is my co-founder was an offensive kernel researcher so our system is tuned for this in a way your average...ambulance chaser is unable to do. https://blog.bismuth.sh/blog/bismuth-found-the-atop-bug https://blog.bismuth.sh/blog/bismuth-found-the-atop-bug https://www.cve.org/CVERecord?id=CVE-2025-31160 https://www.cve.org/CVERecord?id=CVE-2025-31160 The amount of bad reports curl in particular has gotten is staggering and it's all from people who have no background just latching onto a tool that won't elevate them. Edit: Also shoutout to one of our old professors Brendan Dolan-Gavitt who now works on offensive security agents who has a highly ranked vulnerability agent XBOW. https://hackerone.com/xbow?type=user https://hackerone.com/xbow?type=user So these tools are there and doing real work its just there are so many people looking for a quick buck that you really have to tease the noise from the bs.
- pizzalife 1y agoI would try to find a better example than CVE-2025-31160. If you ask me, this kind of 'vulnerability' is CVE spam.
- ianbutler 1y agoExcept if you read the blog post we helped a very confused maintainer when they had this dropped on them with no explanation on hacker news except "oooh potential scary heap vuln"
- danielvf 1y agoI handle reports for a one million dollar bug bounty program. AI spam is bad. We've also never had a valid report from an by an LLM (that we could tell). People using them will take any being told why a bug report is not valid, questions, or asks for clarification and run them back through the same confused LLM. The second pass through generates even deeper nonsense. It's making even responding with anything but "closed as spam" not worth the time. I believe that one day there will be great code examining security tools. But people believe in their hearts that that day is today, and that they are riding the backs of fire breathing hack dragons. It's the people that concern me. They cannot tell the difference between truth and garbage.
- VladVladikoff 1y agoThis sounds more like an influx of scammers than security researchers leaning too hard on AI tools. The main problem is the bounty structure. And I don’t think these influx of low quality reports will go away, or even get any less aggressive as long as there is money to attract the scammers. Perhaps these bug bounty programs need to develop an automatic pass/fail tester of all submitted bug code, to ensure the reporter really found a bug, before the report is submitted to the vendor.
- holuponemoment 1y agoSimply charge a fee to submit a report. At 1% of the payment for low bounties it's perfectly valid. Maybe progressively scale that down a bit as the bounty goes up. But still for a $50k bounty you know is correct it's only $500.
- justsid 1y agoCould also be made refundable when the bug report is found to be valid. Although of course the problem then becomes some kid somewhere who is into computers and hacking find something but can’t easily report it because the barrier to entry is too high now. I don’t think there is a good solution unfortunately.
- meindnoch 1y agoThe solution is simple. Before submitting a security report, the reporter must escrow $10 which is awarded to the reviewer if the submission turns out to be AI slop.
- GuB-42 1y agoI am more interested about the why than on bashing AI-based code analyzers. Without checking, I am sure that AI will be able to find all sorts of vulnerabilities in my untested, unpublished week-end projects. What in curl makes AI-based analysis completely ineffective? The more positive take, and I think the biggest reason is that curl is just well made. But along the way, it most likely uses plenty of code analysis tools: static analysis, testing, coverage, fuzzing,... the classic. And I am sure these tools catch bugs before they are published. Is there an overlap between one of these tools and AI, can one substitute for the other? Another possibility is that curl is "weird" enough to throw off AI-based code analysis. We won't change curl for that reason, but it may be good to know. And yeah, it may just be that AI just sucks but only looking at one side of the equation is not very productive I think. The article mentions spam and AI slop, it is a problem for sure, but the claim here is much stronger than "stop spamming me", it is "AI never worked". And I find it a bit surprising, because when I introduce an new category of tool on some code base I work with, AI or not, I almost always find at least a problem or two.
- CGamesPlay 1y agoI'm pretty sure it's your "more positive take". It's just a mature project which many, many competent eyeballs analyzing and securing it, and correspondingly many, many more incompetent eyeballs looking to make a quick bug bounty. > Is there an overlap between one of these tools and AI, can one substitute for the other? AI is a crude facsimile of any tool, which is both why it's useful and why it's ineffective. In the case linked from the post, it's hallucinating function names and likely hallucinating the entire patch. This hallucination would be an annoyance for the submitter using an AI tool to discover potential security vulnerabilities, and is both an annoyance and waste of time for the maintainer who was given the hallucination in bad faith.
- kazinator 1y agoA prominent project in which people have a stake in seeing bugs fixed can afford to charge a refundable deposit against reporters. Say, $100. If your report is true, or even if it is incorrect but honestly mistaken, you get your $100 back. If it is time-wasting slop with hallucinated gdb crash traces, then you don't get your money back (and so you don't pay the deposit in the first place, and don't send such a report, unless you're completely stupid, or too rich to care about $100). If AI slopsters have to pay to play, with bad odds and no upside, they will go elsewhere.
- prettyblocks 1y agoOf course you can say that when people are obviously copy and pasting AI slop, but if someone used AI in order to find a valid security issue, and reported it effectively, how would they know that AI was even involved?
- deleted 1y ago[deleted]