9 ms·
SQLite Critical CVEs or LLM Slop?
- dvh 2mo ago[flagged]
- mike_hock 2mo agoIt's not actually vulnerable. You're absolutely right. I made a critical error. It's NOT vulnerable. It' actually vulnerable. You're absolutely right. I made a critical error. It IS vulnerable. It's not actually vulnerable. You're absolutely right. I made a critical error. It's NOT vulnerable.
- deleted 2mo ago[deleted]
- inigyou 2mo agoThis is going to be fun for organizations that are mandated to patch all CVEs, isn't it?
- deleted 2mo ago[deleted]
- ymir_e 2mo agoThis was my first thought, this could be terrible if used offensively. The best defense I can imagine is to have an agent reproduce the issues before a human sees it, but even that will cost money.
- inigyou 2mo agoIf by "terrible" you mean "amazing". Imagine being able to tear down an entire Jenga tower of bullshit corpospeak, and replace it with actual security.
- lucideer 2mo agoI'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).
- SirFatty 2mo agoITAR
- lucideer 2mo agoITAR has no such hard requirements. Might be some orgs that tell themselves they're attempting this under ITAR but they're not doing it in any comprehensive way. The only thing within ITAR that I'm aware of concerning itself with software supply chain is SP 800-218 requirements & that's just a load of open-to-interpretation weasel words about having CVE detection & automations in place & some defined plans for reducing the number of vulns. Pretty sure that component of it is even eligible for self-assessment.
- bluGill 2mo agoYes and no. ITAR (and other laws like it) are self assessment and don't specifically say thing thing. However your interpretation / self-assessment is subject to various reviews/audits. These days the reviewers are not going to be kind to someone who just says "not an issue", they will demand strong justification. Most organizations take the view that is is easier to fix all CVEs than try to pass audits. Thus by the letter of the law you are correct. However to meet the letter of the law without fixing CVEs is generally seen as harder than thus fixing CVEs. So the effect is ITAR (and similar laws) force you to fix CVEs.
- SirFatty 2mo agoWhatever you say, chief. I worked in that environment for quite a while, and maybe on a technicality you're right, the effect outcome is that you will do it if not for all your customers that will require it.
- 2mo ago
- whatevaa 2mo agoThose organizations will have to adapt to new reality, ie, that some CVEs are not real.
- afarah1 2mo agoNothing new about that
- ape4 2mo agoCreate the referenced but non-existent file and then fix it /s
- cleansy 2mo agoAll organisations also have exceptions to policies. This one would be one
- smitty1110 2mo agoIt’s honestly not great. The security guys are completely exasperated at my job, we’re wasting time having with these. You take the scam really, investigate for a bit, write up a DNF with justification, they go and up date records, and we all just kinda hope that someone updates the scans so it stops showing up. Something is going to give, and I suspect that the optimistic open filling is going to get canceled.
- lelanthran 2mo agoNot really. LLMs can hallucinate the patches too :-)
- dns_snek 2mo agoSteps to patch a hallucinated CVE: git stash -m "sigh" git commit --allow-empty -m "Patch: CVE-2026-51302" git stash pop
- dgellow 2mo agoOne can hope that will put pressure on the industry to design a better system than CVEs. The signal noise ratio was already terrible before LLMs, I cannot imagine that will still be a meaningful system in 10y. But I’m too cynical to not consider all the middlemen who benefit from the status quo
- pixl97 2mo agoIt's going to be very very difficult to build this system as the exploitability of any particular CVE can massively vary depending on your system configuration. There are a lot of things that are bugs, but cannot be exploited in a standard configuration, most people would wonder why this is even a CVE. But then you have those users that would have the application, then something like a report module that was imported, and another 3rd party module that imports images, and maybe another one that arranges the images in a chart. And suddenly that isolated CVE is now exploitable by user written data because of the original bug, it just required a complex and unexpected chain to get there.
- dns_snek 2mo ago> There are a lot of things that are bugs, but cannot be exploited in a standard configuration And this describes the vast majority of CVEs you see when you scan your dependencies. Typical case: Your regex library has a denial of service vulnerability for crafted regular expressions but your program never allows users to have any influence over the regex.
- ChrisMarshallNY 2mo agoThe problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.
- ymir_e 2mo agoEdits for clarity: seems like this was a controversial take, which I find pretty interesting. I believe people took this comment as LLMs being better than security aware engineers who have the time to spend building solid systems. This wasn't the point, the reality outside "established" tech companies is that software security can be lackluster. There often simply isn't enough resources to check old software for basic vulnerabilities, outdated packages with known issues, there might be a manager who insists on a certain solution. Or that certain services are on "maintenance mode", but rarely get checked since they're on a certain part of the internal network. If they are able to properly scan their full software stack for CVE issues, they cannot deal with a flood of CVEs. ---- We're in a transition period where AI will eventually make software much more secure than it ever was. These noisy CVEs will probably lead to agents verifying vulnerabilities before humans review them. The problem with agent reviews from what I can think of is: - cost to use LLMs to review things - not necessarily easy to plug-and-play in repos: (domain knowledge + vulnerability knowledge) - especially with anthropic: able to use models defensively, without hitting guardrails The last one is the most interesting one to me. How does the AI providers know if you're a "good or bad" guy? And does it matter if open source models is catching up? We're in a kind of cyber arms race wether we like it or not.
- rghammt 2mo agoCurrently we either get AI promo vulnerability dumps like from Chrome with pretty graphs that no one checks or false positives. Where is this one now that was hyped everywhere? https://news.ycombinator.com/item?id=49133889 https://news.ycombinator.com/item?id=49133889 The GitHub submitter could no longer reproduce the issue and the LKML post has no replies: https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-ZHY5T_=4QY5+Fn8BM4A@mail.gmail.com/ https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...
- BedVibe_Studios 2mo ago[flagged]
- mlvljr 2mo agoHonest take, this is a critical CVE.
- awkwardpotato 2mo agoHow so? All 6 of the CVEs covered in the article did not actually exist when investigated
- mlvljr 2mo agoYou're absolutely right, I have hallucinated this. Would you like to find some real CVEs next?
- mahogany 2mo agoThe comment is making fun of a Claude-ism where it becomes super “honest” about stuff. It’s a joke but there is an underlying real effect where this type of language is psychologically manipulative and I would guess makes people believe LLMs output more than if it didn’t use “honest” (or “load bearing” or whatever super serious important sounding word).
- ptx 2mo agoOr maybe they didn't train it that way to be manipulative (although it's certainly a plausible explanation) but simply as an accidental artifact of trying to make it give honest answers? LLM-generated images sometimes includes text from the prompt as literal text in the image, so perhaps this is the same sort of artifact? If they've told it to be honest, it responds by talking about being honest instead of actually being honest, because it has no actual understanding of anything.
- inigyou 2mo agoThe whole RLHF process is structured to train models to be manipulative, no matter what you thought you were training them for.
- jimktrains2 2mo ago
- Spide_r 2mo agoSomewhat related: https://sqlite.org/cves.html https://sqlite.org/cves.html
- Ekaros 2mo agoNot validating submissions seems like avenue for massive attack. Flood the whole system with endless false reports. Thus making it significantly less reliable.
- insanitybit 2mo agoThis is what the Linux kernel is currently attempting since becoming a CNA. edit: "This" being to sabotage the CVE system via DoS, to be clear.
- inigyou 2mo agoWhat do you mean? It isn't a DoS to assign every single bug fix a CVE! Every single bug is making someone vulnerable in some way.
- worthless-trash 2mo ago> What do you mean? No analysis is being done in the linux kernel to assess vulnerability. > It isn't a DoS to assign every single bug fix a CVE! On people who care about this, it is, not in the project itself though. > Every single bug is making someone vulnerable in some way. Not every bug is making someone vulnerable. (docs bugs, test bugs) behavioral changes, performance improvements, the list goes on.
- insanitybit 2mo agoThat's nonsense. The goal is and always has been to undermine the CVE system because Greg does not believe in the system. The kernel is the only project that marks CVEs to fixes, it's the only project that considers every bug to be a "Security" bug, and it has a multi-decades long history of telling the security world to fuck off - all consistent with what I've said. No one else has the process that the kernel has, despite plenty of people having software that's deployed in very similar ways. There's zero question - this is ideologically motivated, not a genuine good-faith attempt to leverage the system.
- 2mo ago
- progval 2mo ago> All advisories in this repo seem AI generated when testing them with Gptzero I pasted this blog post from "Analysis Matrix" to the end in Gptzero, and it also says the blog post was AI-generated (71% chance of AI, 29% chance of AI-Human mix).
- sabot90260 2mo agoA critical rating on a vuln that doesn't exist is wild. The triage cost still lands on the maintainers.
- throwa356262 2mo agoIn the mean time, my very real vulnerability reports are not even acknowledged because maintainers are busy handling this kind of nonsense.
- chrisjj 2mo agoDoes author rep not count for something?
- trueno 2mo agojust ran this article that ran a cve through gptzero.. through gptzero -GPTZero AI Detection -Model 4.8b -We are moderately confident this text is a mix of AI and human -63/88 Sentences likely AI generated
- nearlyepic 2mo agoI mean you didn’t even have to do that, you could tell just by the AI generated header image.
- vips7L 2mo agoSad days ahead.
- account42 2mo agoThe irony of writing an article about slop reports and then defacing it with a giant unrelated slop image at the top.
- pelasaco 2mo agoI had the impression that the code review done and text formatting by the jfrog was mostly driven by Claude too..
- trashb 2mo agoCould be just me but I would be quite careful running these PoC's, especially since the CVE's mostly seem nonsensical. I suppose these CVE's could be targeting some kind of PoC execution instead of actual bug reporting? (I didn't look at the code in depth). Why is the repo even mixing CVE's for "schreibfaul1 ESP32-audioI2S" and "SQLite"? Is mixing CVE's for different products in one repo common practice?
- inigyou 2mo agoThere is no specific practice for a repo full of bug reports. Having a repo full of bug reports is already unusual. Normally you report a bug against the software that has a bug, not by creating a file in your own repo.
- _flux 2mo agoI've created bug reports to open source projects that have related files in my repo. Then I can just list the reproduction instructions like: 1) arrange xxx, such that yyy.. or git clone this repo where this is set up. 2) .. I thought that might have been quite helpful not just to the person I'm sending the bug report to, but also to myself when I need to evaluate if the bug has been fixed.
- effnorwood 2mo ago[dead]
- kasperset 2mo agoAs a non-expert in this field, I feel we have introduced more “stochasticity”. Before, We had to deal with some human error/ misunderstanding for most part but now we have to deal with power of condensed human expertise along with homogenized flaws.
- gortok 2mo agoWe can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credibility massively when your LLM-based “AI” gets the probabilities wrong at best, or in this case, claims a line of code generates a vulnerability when it is, in fact, a code comment. LLMs are text-prediction engines. They are not Artificial Intelligence, and shouldn’t not be treated in any form or fashion as if they possess intelligence. What bothers me about this entire situation is that presumably the folks that relied on the LLM-based “AI” to generate these vulnerabilities knew (or should have known) enough about their tool to know this would happen, but did not. Now, we all pay the consequence, to the tune of hundreds of thousands if not millions of dollars of wasted productivity from teams that have to deal with the resulting fall-out of this usage of “AI”. A human must verify everything an LLM presents as fact. Everything. If you don’t, we all pay the price. LLMs do not remove the onus of responsibility on the human being, if anything they amplify it because LLMs can generate lots more output more quickly that needs to be verified than humans can.
- geraneum 2mo agoUnfortunately people sometimes get defensive against this take. But I think treating the LLM as you described can make you a better LLM user and help get better output. It helps understand the failure modes better, and moderate one’s reliance on them. Just like how we should do for every tool we work with.
- gr_norm 2mo agoYes, I've found that reminding yourself of how they actually work helps keep you on guard against LLM-patterned mistakes. Especially things like carefully considering what parts of the current task likely fall outside the distribution of corpus + RL data (as much as that can be guessed).
- palmotea 2mo ago
- firesteelrain 2mo ago> This incident demonstrates a systemic issue with automated vulnerability ingestion But they make X-Ray which does automated vulnerability indexing and matching dependencies to CVEs.
- fl1pper 2mo agoGreat article! We need to further emphasize the importance of responsibility when using LLM tools to produce output for others. It's great to use them for refactoring and bug discovery, but keep in mind that it's your responsibility to analyze it and iterate on it with AI. It makes your code better and develops technical expertise. The "Hey, analyze that codebase, find all dangerous CVEs, and write a README for the PoCs, so I can post it online for others to analyze, and if I'm lucky, I'll get a paycheck or a title to add to my resumé" approach might work in 0.1% of cases, but it will generate a ton of slop for the community to drown in. I also think GPTZero and other AI detectors have far more false positives than correct guesses. I tried it on several texts & messages I wrote before 2019, and it flagged them as 80% AI-generated. In that case, it's reasonable to assume that AI also generated the README text for each discovered CVE. In other cases, however, we should be more cautious.
- gste 2mo ago> The cited code didn't even exist in those versions or referenced unrelated logic. > When testing the PoC payloads they didn’t work (not triggering any crash). I think the future is pretty obvious, if this isn't being done on projects already: you need to automate these checks and reject automatically
- tantalor 2mo agoGreat idea! We can use LLM to do this. LLM: I ran the check and it repro'd > Did you really? LLM: You're absolutely right. I didn't actually run the check. Good catch! One sec let me do that now... yep I ran the check and it definitely repro'd this time. > I hate my job
- timedude 2mo agoThat is if it didn't completely delete the repo in the first place. Consider yourself lucky
- luciana1u 2mo ago[flagged]
- r_lee 2mo agocan anyone explain what value does the ai slop image at the top add to this blog post?
- wang_li 2mo agoThere should be some sort of defamation lawsuit possible when people publicly publish trash repos or reports like this. If you were writing a competing product in a safe language you’d love to see such FUD.
- inigyou 2mo agoIt might be possible, but we'll only find out if a lawsuit actually takes place.
- leviyi 2mo ago[flagged]
- myshapeprotocol 2mo ago[flagged]
- myshapeprotocol 2mo ago[flagged]
- taf2 2mo agoLlm are great at things that are repeatable. Otherwise they are just like us - great at believable bullshit… a sign of true intelligence
- boundsconf 2mo ago[flagged]
- oxydite 2mo agoDamn, dunno why I've always assumed that if something gets a CVE, then some authority has reproduced and verified it. Shouldn't it be CNA's job? Why would anything get a number if it hadn't been verified?
- insanitybit 2mo agoThe current system is radically overwhelmed and has been seeing cuts over the last few years. > then some authority has reproduced and verified it. No, the system puts a lot of trust into the submitter.
- inigyou 2mo agoIt's really just a number allocation and that's all. You can get a number. That doesn't mean the number refers to something useful.
- mavleop 2mo agoThankfully I’m seeing other people mention that this article itself seems AI-generated. While there's less proof, i suspect that the actual analysis was done by an llm as well. So while i like to dunk on hallucinations as much as the next guy, who's to say this analysis doesnt have them too? (for the record, i think the cve's are probably bogus, i just think there's an inherent problem with debunking slop with slop)
- BigTTYGothGF 2mo agoThey couldn't even be bothered to use a non-AI image.
- lexicality 2mo agothat's fine, I'm not sure they bothered to actually write the article either
- cocorebop 2mo ago> Over the past few days, a newly created GitHub repo (programmervuln/cveadvisory-) published a batch of SQLite vulnerability advisories (as part of other 50+ CVEs which we believe are also LLM slop except from one). NVD quickly flagged these as critical, and CISA's ADP agreed. These two sentences seem incompatible to me. A brand new actor can just say some stuff and cause major disruptions in this way?
- minraws 2mo agoI think this will give a lot of ammunition for people on both sides, but the thing is we have made software so complex, ofc a lot of it is inherent, that we can't even tell if the vulnerability is hallucinated or not. I find it getting closer and closer to the nonsensical 50 page mathematical proofs built with computer assited tools, that been driving reason for abstract proof systems so that we can break down such complicated examples into something simpler that we can feed into a system to help us verify. I feel like for some reason we have built the output machine before we could even build the actual output verification machine. And aren't ready to accept that we should all move towards building some kind of system to better help verify the "truth" produced by these AI systems. I am now very convinced that human built and proven to work verification systems for LLM outputs are more important than ever, but I just don't see any company or team working towards it. Trillion dollar systems shipping to production code that is hardly understood by any one without any verification what really could go wrong. Honestly I really truly hope nothing goes wrong. To note I am not anti-AI but I want to be able to have my LLM outputs be verified by more than just me under a tighter and tighter deadlines, accepting worse and worse slop just because it would less cost to me to accept the PR and then fix it in prod. LLMs aren't sustainable software development, they are delusional slop factories if we aren't able to actually make sure the outputs have any amount of correctness. With humans it worked because all 10 of us could in some ways verify each others outputs, and had trust in the fact that our timelines included the time for reviews.
- _superposition_ 2mo agoTaking security theatre to a whole new level.
- jedbrooke 2mo agononexistent functions being referenced is pretty bad, I haven’t had that issue since like opus 4. Shows either complete lack of oversight and/or using a bargain basement model (so no even having financial skin in the game). Most likely both
- linuxhansl 2mo agoI almost feel that we have a new generation of the "script-kiddie". People without much (or any?) knowledge of software engineering using outside tooling to do "stuff" they wouldn't be able to do themselves. Maybe it's a not a perfect analogy - in this case the intentions seemed to be laudable - but we will see much more of this, including from malicious actors.
- jokoon 2mo agocould this be used to create some panic and make some money through trading or something? no idea who submitted all of those, but it sounds like malicious intent
- umarcyber 2mo agoIt really is a shame that funding cuts at NIST has led directly to this. I don't know if there is an incentive for a non-profit to come out of the woods to do the validation and enrichment process, but boy do we need it. Like gortok has said, LLMs are just probabilistic text-prediction engines. They cannot automate the vulnerability validation lifecycle because they cannot account for improbable edge cases. They may not even get the version of the software right...
- wildfireday2 2mo agoFunding cuts at NIST did not lead directly to this. You must be new here or haven’t spent a lot of time directly interfaced with NIST on cybersecurity in decades past. And NIST’s role with CVE’s has always been purely ministerial/clerical. It has never been their job to do even a cursory investigation into the vulnerability itself.
- jrochkind1 2mo agoOP suggests something at NIST changed in 2024, where they stopped doing as much verification as they did before. To be sure, the suggestion is not funding cuts, but an increase in workload with same funding. Here is the 2024 NIST announcement OP links to: https://nvd.nist.gov/general/news/nvd-program-transition-announcement https://nvd.nist.gov/general/news/nvd-program-transition-ann... > Currently, we are prioritizing analysis of the most significant vulnerabilities. In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well. The OP described this as "NIST effectively hit pause on deep analysis. " It does sound like they stopped doing something they used to do in 2024. I personally have definitely not spent a lot of time directly interfaced with NIST on cybersecurity in decades past, I know nothing about it, just what I read in OP. Are you saying the OP was wrong to call what NIST used to do "deep analysis", and/or that the thing NIST stopped doing was "purely ministerial/clerical" in a way that it would not have caught fake reports anyway, contradicting the OP? Or other? Again, to be sure, the OP's suggestion was not that this was caused by NIST funding cuts, but by "a massive surge in vulnerability reports,"
- hnjbx769kd 2mo ago[dead]
- nirbendavid 2mo ago[flagged]
- wbsun 2mo agoSo a so-called vulnerability has: - cited code not exist, and - PoC not work. How would any serious system allow it become a big deal?
- 1970-01-01 2mo agoExactly. CVEs shouldn't be allowed to be anywhere near critical without a working PoC or other proof.. Trust+verify.
- jrochkind1 2mo agoThe federal government ought to be funding NIST sufficiently to actually do sufficient analysis. Do we care about funding civil "cyber" defense or not? (Obviously the answer is not, we only care about funding offensive capabiltiies).
- inigyou 2mo agoThe federal government defunded the CVE program last year, I think, because it was woke.
- Plutoberth 2mo agoThis is obviously impractical. With the volume of bug reports that are generated and such a wide breadth of software no single agency will be able to handle all reports. And honestly? It doesn't matter, even today. CVE should serve as a reasonably deduped identifier of specific vulnerabilities. It was never interesting and practical to care about 100% of vulnerabilities in a specific deployment, and it's not interesting today.
- dpacmittal 2mo agoThe duality of LLMs is real. On one hand, they are solving unsolved math problems, on the other hand they get fooled by the strawberry test or the car wash test. We see this all the time on reddit and HN, where some users are building crazy things with LLM and others are swearing by how useless LLMs are. Some people claim SWE will be dead in a few years, and others are claiming the AI bubble is going to burst anytime now.
- GeorgeWBasic 2mo agoThe "AI bubble" refers to the massive overinvestment in AI, and doesn't imply that LLMs are going to go away after it pops.
- bilater 2mo agoThis isn't great but a negative side effect of an otherwise positive change. I know a lot of comments will try to make this into a gotcha but its really not. Decreased friction leads to misuse. Just like before we'll figure out ways (reputation, light filter pass) to deal with it.
- rib3ye 2mo ago> Because no step in today's system actually requires a proof-of-concept or bug reproduction, a plausible-sounding fake advisory can slide right through the pipeline and end up in GHSA, downstream databases, and enterprise scanners. I have no experience in the realm of security, but why isn't the submission process similar to any normal software shop (big or small) in terms of required bug repro steps?
- Joel_Mckay 2mo agoSue them for defamation. Seriously, the hubris on these "AI" sloppy clowns needs consequences. =3
- Bitu79 2mo ago[dead]
- securitycipher 2mo ago[dead]
- mpfh 2mo ago[dead]
- ghostnshells 2mo agoI think this is expected with the increased usage of AI on both fronts (finding vulns and automating their remediation). Yes, there are and will be companies that help other companies validate CVEs before they raise any alarms, but that noise/signal battle has always been there. I work as a sec eng at a biotech firm and have a tool I built that pulls our ITIL and runs a search on NVD, GHSA, etc. for enrichment and classification. But the number of CVEs I see for products we house is astronomical compared to the actual version of that product we are running (and that's only one value). But that's also a challenge when working with so many sources and having a small team try to tackle them. The age-old validate, remediate, verify proves itself useful once again!
- beyondscaletech 2mo ago[flagged]
- Sattyamjjain 2mo ago[flagged]