12 ms·

Hardening Firefox with Claude Mythos Preview

https://arstechnica.com/information-technology/2026/05/mozilla-says-271-vulnerabilities-found-by-mythos-have-almost-no-false-positives/ https://arstechnica.com/information-technology/2026/05/mozil...

Diti 5mo ago
I hope to see the day when (or if) the LLMs get so good at spotting and fixing bugs that all that’s left for the Firefox engineers to do is to focus on adding new features. This isn’t sarcasm. Firefox deserves to be used more. Most people I know don’t use it because “Chrome does almost everything better”, and Firefox can’t compete with the other browsers’ roadmaps.
- greggoB 5mo ago
  > Firefox deserves to be used more Totally agree. I even go as far as choosing which website I make purchases on depending if they work on FF, or writing to support occasionally to tell them it's not supported or a feature isn't working properly and this would be appreciated. I know it pretty much always goes nowhere, but I feel it's what I can do to keep the browser somehow on the radar.
- nubinetwork 5mo ago
  > Firefox deserves to be used more Part of the problem is, when they stop working on fixing bugs, they start doing Mr Robot things... We just want a web browser. Nobody asked for pocket, or AI... If they use AI to fix all the bugs, then what else is for them to do, other than maintain syntax compatibility with the various languages they build with? They're just going to go back to making the browser trash again.
  - FeepingCreature 5mo ago
    I have an old apk of Firefox pinned on mobile. I do this because I genuinely believe that for my very limited usecases, the browser has become actively worse. (Don't worry- I use the system browser for any site I don't fully trust.)
  - cobalt60 5mo ago
    when was the last time you made a contribution financially or physically? Those features were economically tied, who's giving the dough?
- kakacik 5mo ago
  Chrome ain't better in any meaningful way for >99% of use cases. Heck, I am a dev and I use FF with ublock origin only, for past... 10 years? Same with my wife, after I've explained things to her and she understood how different internet experience can be thats the primary browser. So please don't put the argument like 'here is crappy underdog but please use it because monopoly is bad and google is a bit evil', its first class experience in everything I have ever thrown at it. Tripple that on mobile, by far the best mobile and useful mobile experience, bar none.
  - freedomben 5mo ago
    Agreed, though many websites only test on Chrome and are unusable on Firefox. Ramp.com and mailgun come immediately to mind. Zoom also won't let you join with browser on Firefox. There's enough that I have to keep a chrome available for those types of sites. It shouldn't be this way, but it is
    - mnicky 5mo ago
      > Zoom also won't let you join with browser on Firefox. FF works for me.
      - freedomben 5mo ago
        do you hack your user agent or anything to get it to show you the option?
        sfink 5mo ago
        You haven't needed to in quite a while. Well, assuming you're on desktop? I guess I've never tried it on mobile. That feels like asking for trouble. I ran Zoom in my Firefox desktop browser for a while, but it tended to overheat my laptop. Other things overheat it too, so I don't know how much was specific to Zoom on Firefox. I just checked. Still gives me the option ("Join from browser" in a less highlighted option, trying to drive you to their native client I guess.)
- IshKebab 5mo ago
  Unfortunately we're probably still quite far from that. This is the best case for LLMs still - the quality of their output didn't matter as long as it worked, and there was a near-perfect oracle for checking if their output worked. That's a really good use case for LLMs. It also applies to things like finding proofs in Lean and creating test stimulus. In both cases you know automatically whether the output is good, and it doesn't really matter if it isn't. That isn't the case for most bugs, and definitely isn't the case for actually fixing bugs.
- kgwxd 5mo ago
  Browser haven't needed new features in a very long time. Extensions were supposed to be the solution to that.
- _heimdall 5mo ago
  Wouldn't that quality and availability just allow Chrome to pull ahead of Firefox that much faster? If Mozilla created some proprietary LLM or harness that they used internally to outpace Chrome that may be a different story, though I also don't see that happening.
ChrisArchitect 5mo ago
Related: The zero-days are numbered https://news.ycombinator.com/item?id=47853277 https://news.ycombinator.com/item?id=47853277
lschueller 5mo ago
Let's see, how this will improve the daily soc work. I still don't see, what's the big difference between Mythos and Opus, security wise. I'm confident, that this kind of vul detection is a long-term improvement. But does specifically Mythos makes such a big difference to "normal" models? I would love to see, what's the actual difference.
- JoshTriplett 5mo ago
  Among other things, Mythos seems better at "let me find, weaponize, and stack vulnerabilities until I get end-to-end from untrusted content to root", rather than just finding one thing in a specific identified area.
- mccr8 5mo ago
  Quantifying the abilities of an LLM is a hard research problem, so I'm not sure if I can describe it in any great way, but Mythos did seem to be fairly clever about putting together things from different domains to find problems. For instance, in one of the included bugs (2022034) it figured out that a floating point value being sent over IPC could be modified by an attacker in such a way that it would be interpreted by the JS engine as an arbitrary pointer, due to the way the JS engine uses a clever representation of values called NaN-boxing. This is not beyond the realm of a human researcher to find, but it did nicely combine different domains of security. As the person responsible for accidentally introducing that security problem (and then fixing it after the Mythos report), while I am aware of NaN-boxing (despite not being a JS engine expert), I was focused more on the other more complex parts of this IPC deserialization code so I hadn't really thought about the potential problems in this context. It is just a floating point value, what could go wrong?
  - lschueller 5mo ago
    Okay, so far it makes sense to me. But is the deal with JS and floating point values, which isn't soemthing super special super rare stuff, only detected and identfied by Mythos while Opus wouldn't get to this point?
- IainIreland 5mo ago
input_sh 5mo ago
Original source: https://news.ycombinator.com/item?id=48051079 https://news.ycombinator.com/item?id=48051079 It's better because it actually lists a sample of Bugzilla reports that were made public. This topic was discussed previously (36 comments two weeks ago: https://news.ycombinator.com/item?id=47885042 https://news.ycombinator.com/item?id=47885042), but the part about bug reports being made public is brand new.
xacky 5mo ago
I just hope they don't start ignoring human created bug reports, as there are still many that haven't been fixed for years.
MetaverseClub 5mo ago
I'm curious about how did Mozilla do bug finding before Mythos? Did they use any non-AI bug finding tools?
- mccr8 5mo ago
  The usual sorts of fuzzing and static analyses, using AddressSanitizer and ThreadSanitizer. Also, with a bug bounty program to try to encourage external researchers to report issues. (I work on Firefox security; also I fixed 2 of the bugs linked in the blog post.)
- canucker2016 5mo ago
  Coverity (similar to lint) scans various open source software products for vulnerabilities. see https://www.blackduck.com/static-analysis-tools-sast/coverity.html https://www.blackduck.com/static-analysis-tools-sast/coverit... and for Firefox-related alleged defects, see https://scan.coverity.com/projects/firefox https://scan.coverity.com/projects/firefox You have to create an account to view the actual reported defects. There are just over 5000 reported defects still outstanding. I don't know how many overlap with the reported 271 Mythos-reported defects.
  - rockdoe 5mo ago
    How many of those are false positives though? Probably just over 5000? You get bug bounties if you report the kind of bugs Mythos identified. There's a reason no-one collected bounties from the "5000 defects" Coverity identified. The Mythos reports have several examples of chaining a whole bunch of logic in different parts of the program together to exploit something very subtle. The Coverity reports aren't anything like that. These tools aren't remotely in the same league or even universe.
    - IainIreland 5mo ago
      Yeah, fuzzing, sanitizers, and bug bounties were our main pre-AI tools for finding bugs.
    - 5mo ago
jerrythegerbil 5mo ago
Again, and this is important: A bug is a bug. A “potential vulnerability” is a bug. A vulnerability is verifiable as having security implications with a proof of concept or other substantial evidence. Words matter. Bugs matter. It’s important to fix large amounts of bugs, just as it always has been, and has been done. Let that be impressive on its own, because it IS impressive. Mythos didn’t write 271 PoC for vulnerabilities and demonstrate code path reachability with security implications. Mythos found 271 valid bugs. Let that be enough.
- epistasis 5mo ago
  I was a bit confused by your definitions, but here's how Mozilla broke out [1] the 271, um, things: > As additional context, we apply security severity ratings from critical to low to indicate the urgency of a bug: > * sec-critical and sec-high are assigned to vulnerabilities that can be triggered with normal user behavior, like browsing to a web page. We make no technical difference between these, but sec-critical bugs are reserved for issues that are publicly disclosed or known to be exploited in the wild. > * sec-moderate is assigned to vulnerabilities that would otherwise be rated sec-high but require unusual and complex steps from the victim. > * sec-low is assigned to bugs that are annoying but far from causing user harm (e.g, a safe crash). > Of the 271 bugs we announced for Firefox 150: 180 were sec-high, 80 were sec-moderate, and 11 were sec-low. Mozilla uses the term "vulnerability" for even sec-high, even though they say right below that it doesn't mean the same thing as a practical exploit. And on their definitional page, they classify even sec-low as "vulnerabilities" [2]. Words are tools, that get their utility from collective meaning. I'd be interested where you recieved your semantics from and if they match up or disagree with Mozilla. [1] https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/ https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin... [2] https://wiki.mozilla.org/Security_Severity_Ratings/Client https://wiki.mozilla.org/Security_Severity_Ratings/Client
  - Gregaros 5mo ago
    > Mozilla uses the term "vulnerability" for even sec-high, even though they say right below that it doesn't mean the same thing as a practical exploit. That’s not evident in what you pastedat all. What you pasted says > sec-critical and sec-high are assigned to vulnerabilities that can be triggered with normal user behavior […] We make no technical difference between these […] sec-critical bugs are reserved for issues that are publicly disclosed or known to be exploited in the wild. > sec-low is assigned to bugs that are annoying but far from causing user harm (e.g, a safe crash). From this one infers that the "180 were sec-high" bugs found are actually exploitsble but known to have been found in the wild, and are NOT mere annoying bugs. The difference between 180 and 270 does nothing to deflate the signicance, or lack there of, of the implication re: Mythos.
crummy 5mo ago
Curious if people think LLMs will lead to more secure or less secure software in five years.
- int32_64 5mo ago
  Both. The skilled will use them to find problems, the unskilled will use them to slopcode insecure software the skilled will have to fix.
- deleted 5mo ago
  [deleted]
- stavros 5mo ago
  That depends on which side has more money.
- bawolff 5mo ago
  One of the biggest issues in security historically imo is vendors who think, well nobody will ever find this bug so we can deprioritize fixing it. LLMs will prevent vendors lying to themselves which will lead to more secure software.
- UltraSane 5mo ago
  In 5 years attackers have an advantage but in the long run I think more secure if developers use LLMs on software to find and fix all of the worse remotely exploitable bugs before release. LLMs are going to force devs to be much more security conscious.
  - canucker2016 5mo ago
    I think it'll be a war of who has the better LLMs-as-security-scanner. Ideally, you'd do a comprehensive all-source-code scan, (and the LLM-scanner finds everything during those scans), and fix all the reported defects. Afterwards, any dev that commits code will run the LLM-scanner on the modified code (and affected areas) and fix any reported defects. So the black-hat hacker would be shut out unless they get access to an LLM-scanner with better analysis than what the target project is using. Major LLM-scanners could give priority access for new versions of LLM-scanners to major projects to find any defects in the current source code before any other party could use the reported defects against the project or their users. So black-hat hackers would be left with developing their own LLM-scanner better/more efficient than existing major LLM-scanners. Given enough incentive, they might develop such a tool. Look at the market for zero-day vulnerabilities for smartphones, esp iPhones.
deferredgrant 5mo ago
[flagged]
mmooss 5mo ago
> “That’s the key thing that has unlocked our ability to operate at the scale we’ve been operating at now,” he said. “It gives the engineer a crank they can pull that says: ‘Yep, this has the problem,’ and then you can iterate on the code and know clearly when you’ve fixed it and eventually land the test case in the tree such that you don’t regress it.” I don't understand much of this paragraph: * "a crank they can pull that says: ‘Yep, this has the problem,’": as in, ring an alarm? Does the LLM ring th alarm? * "you can iterate on the code and know clearly when you’ve fixed it": Isn't that true of most bugs, assuming you do the normal thing and generate a test case? And I thought the LLM output test cases itself: "It will craft test cases. We have our existing fuzzing systems and tools to be able to run those tests" And are they claiming the LLM facilitates iterating? * "and eventually land the test case in the tree": Don't you create the test case before the fix? And just a few words earlier they seemed to be working on the fix, not the test case. And see the prior point about test cases. * "such that you don’t regress it.”: How is the LLM helping here? Maybe I'm missing some fundamental unwritten assumption?
- mccr8 5mo ago
  Mostly I think this just means that having a test case makes it easier to fix and verify. You can't actually take for granted having a test case when fixing a security bug. Sometimes you only have a crash stack or maybe a vague and hypothetical static analysis result. > eventually land the test case This is just a reference to the fact that we don't land test cases for security bugs immediately in the public repository, to make it harder for attackers. You are right that the LLM only helps with creating the initial test case. Things like running the test case in automation is part of the standard development process.
  - mmooss 5mo ago
    Thank you; that makes sense.
rem1099 5mo ago
I don't find that number very high. In a project of the size of Firefox, a new version of a compiler with stricter warnings or a draconian interpretation of the C standard can easily find 200 new bugs. New tools find new bugs, but the oligarchy newspapers report on Mythos and not on clang-22.0.
- sfink 5mo ago
  The raw number of things found by Claude (Opus or Mythos) was much higher and would be more comparable to a new clang warning. I vaguely remember seeing a number early on in this process that was in the mid-thousands. The 271 is a small, validated subset of that. None of the 271 were deemed false positives iiuc. Most instances of a new clang warning will be false positives. (Same as most of the raw problems reported by the AI.) It is still unclear and open for speculation as to what percentage of all security bugs in Firefox today are being found by the AIs (as opposed to not being found at all). It might be that AI is very good at certain types of problems, even if we can't put our finger on what those types are, and that after the initial wave of bug reports the AI findings will slow to a trickle even while many many other bugs remain in the codebase. Or it might be that AI really does detect most instances of some class of problems and all those bugs will now be gone forever, never to return as long as Mozilla keeps paying the token monster. This is closely related to the oft-asked question "are we better or worse off after both attackers and defenders have access to this new capability?"
tialaramex 5mo ago
They've only linked a few tickets, so of course maybe when we see all 271 actual distinct things the insight won't apply but all those I examined ended up as some C++ code with a nasty bug in it. Firefox is written in several languages, only about 25% of it is in C++ but every single one of these issues seems to touch the C++.
- mccr8 5mo ago
  A general limitation of this approach is that it is only as good as your validator, and there's nothing easier to validate than a test case that creates, say, an AddressSanitizer use-after-free. For subtler issues will we have to more specific validators or will the LLM become better at coming up with other dangerous conditions it will verify? We'll see.
  - tialaramex 5mo ago
    > A general limitation of this approach is that it is only as good as your validator, and there's nothing easier to validate than a test case that creates, say, an AddressSanitizer use-after-free Sure, but, surely AddressSanitizer would also detect the same problem in the C or Rust which together also make up about 25% of Firefox so... ?
- jeroenhd 5mo ago
  It's possible Mythos is a lot better at finding vulnerabilities in C++ code than it is for other languages. After all, these models are also based on pre-existing security analysis. From what I can tell, a lot of these bugs were hardly C++-specific, they just happened in C++ code. Even the most secure Rust can't magically catch things like TOCTOU issues.
  - tialaramex 5mo ago
    > Even the most secure Rust can't magically catch things like TOCTOU issues I suppose it depends what the word "magically" means. A ToCToU race is because you imagined things wouldn't change but they did and in Rust you actually do write fewer patterns with this mistake because of the Mutable xor Aliased rule. If we have at least one immutable reference to a Goose then Rust isn't OK with anybody mutating the Goose, your safe Rust can't do that and unsafe Rust mustn't do that. So the ToCToU race caused by "Oops I forgot somebody else might change the Goose" is less likely because you were made to wrestle with this problem during design - the safe Rust where you just forgot about this doesn't compile.
delichon 5mo ago
In the latest Mission Impossible, saving the world depends on recovering the original software of an escaped superhuman AGI from a sunken Russian submarine. Luther writes a "poison pill" that given the original source will instantly one-shot the AI. We were left to wonder how this magical code could have been written, but now we know. Luthor just wrote a Mythos prompt that handed it the source code and asked for an immutable critical exploit.
gnabgib 5mo ago
16 day old story Wired: Mozilla Used Anthropic's Mythos to Find and Fix 271 Bugs in Firefox (41 points, 18 comments) https://news.ycombinator.com/item?id=47853649 https://news.ycombinator.com/item?id=47853649 Ars: Mozilla: Anthropic's Mythos found 271 security vulnerabilities in Firefox 150 (33 points, 8 comments)https://news.ycombinator.com/item?id=47855384 https://news.ycombinator.com/item?id=47855384
- mozdeco 5mo ago
  No, it's a new post, see also https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/ https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin...
  - gnabgib 5mo ago
    That's this post.. and while it's more detail on the same headline (271 bugs) these discussions look the same as 2 weeks ago (and the same as all the bloggers and podcasters discussed)
OhMeadhbh 5mo ago
When I was at PalmSource, I tried to get budget for CoVerity or Fortify (static code analysis tools.). "Too expensive," my management chain said. I spent another year putting together a deal for a lower cost but limited to scanning the network stack. "No, it's based on BSD and BSD is inherently secure," my management chain said (neither is true, btw.) I eventually left and wound up at Mozilla where there were a number of /* flawfinder ignore */ comments scattered throughout the code. My guess is that Mythos just ignored the "flawfinder ignore" directives and reported the known vulnerabilities in the code.
- AndrewDucker 5mo ago
  The code is open. If you can prove that's the case you'll have a real news story...
- bvisness 5mo ago
  Even a quick glance at the bugs revealed in the blog post would quickly disprove your theory.
nnm 5mo ago
I still don't know the exploit count for Mythos. Is it zero, one, or more?
- sfink 5mo ago
  More, many more. See the bug reports linked in the post. I checked a few, and all of them had an exploit in them, and there are definitely more than 1 bugs listed. Well, depending on how you define "exploit"; some might only read arbitrary pointers or just out of bounds. Those would be useful primitives in a chain of vulnerabilities, not exploits themselves. You'll have to read through the first comments yourself, but if you're hoping that this is all nonsense and ignorable hype, you're going to be disappointed.
kajman 5mo ago
I dismissed the earlier non-technical blog post as shameless product boosterism for Anthropic. The linked hacks blog (which is a better source than this article) is a welcome release. It's hard to deny there's something real to this now, I think. Mozilla's internal definition of a "vulnerability" is also probably more widely applied than what many would intuit, but it is good that these issues are being taken seriously and fixed.
- apexalpha 5mo ago
  At the same time other companies like AISLE are matching Mythos on vulnerabilities using older models but their own harnass: https://aisle.com/blog/aisle-matches-anthropic-mythos-on-freebsd-zero-days https://aisle.com/blog/aisle-matches-anthropic-mythos-on-fre... So while Mythos certainly is real I think you could do the same with Deepseek pro, GPT 5.5 etc...
  - kajman 5mo ago
    Agreed. The earlier blog post did not explicitly claim this, but I think casual viewers were prompted to believe that the Magic of Mythos (TM) went and found (and fixed??) a bunch of vulnerabilities with minimal human guidance, and even contrasted this with their fuzzing infrastructure and made it sound (to me) like it was casting shade on it. This new post makes it pretty clear that this was all bolted on-top of their existing fuzzing infrastructure, and really just used to get more and better initial hits that a very skilled team is looking at. I assume Anthropic was giving them a very good deal on inference for the positive PR, but I believe these other reports and suspect Mozilla did not really need them.
  - yorwba 5mo ago
    As the Hacks.Mozilla article notes: "We began with small-scale experiments prompting the harness to look for sandbox escapes with Claude Opus 4.6. Even with this model, we identified an impressive amount of previously-unknown vulnerabilities which required complex reasoning over multiprocess browser engine code."
  - jonfw 5mo ago
fg137 5mo ago
What are people's thoughts on how this could affect static analysis tools? I know they are very different beats but often they achieve the same goal. Static analysis tools can be slow, and they report lots of false positives. I wonder if these models will get good + cheap enough so that people rarely reach for static analysis.
- sfink 5mo ago
  I've been thinking about this. Static analysis tools can also be much faster and most are fully deterministic, so including them in CI can catch bugs or latent bugs before they have a chance to land. I maintain a static analysis tool using in Firefox's CI. False positives have to be fixed or annotated as non-problems in order for you to land a patch in our tree. That means permitting zero positives (false or true), which is a strict threshold. This is a conscious tradeoff; it requires weakening the analysis and getting some false negatives (missed bugs) in order to keep the signal-to-noise ratio high enough that people don't just ignore it and annotate everything away, or stop running it. Nearly all static analysis tools have to do this balancing act. AI, as commonly used, is given more leeway. It's kind of fundamental that it must be allowed to hallucinate false positives; that's the source of much of its power. Which means you need layers of verification and validation on top of it. It can be slow, you'll never be able to say "it catches 100% of the errors of this particular form: ...", and yet it catches so much stuff. Data point: my analysis didn't cover one case that I erroneously thought was unlikely to produce true positives (real bugs), and was more complex to implement than seemed worth the trouble. Opus or Mythos, I'm not sure which, started reporting vulnerabilities stemming from that case, so I scrambled and extended the analysis to cover the gap. It took me long enough to implement that by the time I had a full scan of the source tree, Claude had found every important problem that it reported. The static analysis found several others, and I still honestly don't know whether any of them could ever be triggered in practice. I still think there's value in the static analysis. Some of those occurrences of the problematic pattern might be reachable now through paths too tricky for the AI to construct. Some of them might turn into real problems when other code changes. It seems worth having fixes for all of them now for both possibilities, and also for the lesser reason of not wanting the AI to waste time trying to exploit them. At the same time, clearly the cost/benefit balance has shifted. They could also team up: if I relax my standards and allow my analysis to write an additional warnings report of suspected problems, with the clear expectation that they might be false alarms, then I could feed that list to an AI to validate them. Essentially, feed slop to the slop machine and have it nondeterministically filter out the diamonds in the rough. Food for thought...
benced 5mo ago
Reading this article in the context of the Zig folks refusing to even consider LLM-generated bugs certainly shapes my perspective on what technologies will be in my toolchain.
- grumbelbart2 5mo ago
  Both are right and it depends on which model you use and who submits those bugs. The capabilities of leading models went from 99% noise to 99% valid bugs in essentially a few months. Some projects are flooded with the former and need to take precautions to avoid essential DoS attacks on the maintainers.
  - benced 5mo ago
    Yeah but Zig essentially has till Mythos releases (and arguably less time) to fix their policy. OSS that doesn't take LLM security reports at that point is a liability.
- tkel 5mo ago
  Zig devs can run Mythos same as anyone else can. I think you are failing to understand the reasoning behind their decision. It's about contributors, not contributions. https://kristoff.it/blog/contributor-poker-and-ai/ https://kristoff.it/blog/contributor-poker-and-ai/
  - benced 5mo ago
    Quoting from https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-policy https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-pol...: > Strict No LLM / No AI Policy > No LLMs for issues. > No LLMs for pull requests. > No LLMs for comments on the bug tracker, including translation. English is encouraged, but not required. You are welcome to post in your native language and rely on others to have their own translation tools of choice to interpret your words. If they would accept issues filed by AI or written by AI, they should edit their policy to say that.
    - 5mo ago
danieltanfh95 5mo ago
Really it was not the issue that Opus could not do all these, there was just no incentive to fix bugs. Mythos represented a real marketing use case, so yes thanks for spending money to fix this, but this is not sustainable.
jwr 5mo ago
> Anyone building software can start using a harness with a modern model to find bugs and harden their code today. We recommend getting started now. From what I understand, that is a recipe for getting quickly banned by commercial LLM providers?
Worf 5mo ago
Maybe if Mozilla focused less on new useless features and redesigns, they would be able to focus more on writing secure and bug-free code. I'm not only talking about big things like * Pocket, * several major UI redesigns and * the offline translations, but even tiny useless things like * browser.urlbar.trimURLs, * putting the search query in the URL bar instead of the URL after searching from the URL bar, * messing with the Edit and Resend feature for no reason (the good one that updates the content length is still available at devtools.netmonitor.features.newEditAndResend) and * probably thousands of little shit like this that took a bunch of developer hours to implement. All of the above should've been add-ons. And of course, we know Mozilla spends a lot of money on things unrelated to Firefox at all. It's amazing Firefox is somewhat secure and stable compared to Chrome, which is backed by Google with their infinitely deep pockets. This is a web browser, after all. Something most people use all the time. Something that accepts untrusted input from thousands of sources every day. People use it pretty much every aspect of their lives - banking, personal communication, porn, expressing political opinions. It's used for viewing PDFs, playing media files, for interacting with a whole bunch of APIs (that IMO shouldn't be part of the web, but they are). Security should be top priority.
- mplanchard 5mo ago
  It would be amazing if we didn’t have to have this conversation on every single thread about anything related to Firefox. Firefox/Mozilla tries literally anything to expand their feature set, customer base, or revenue stream? They need to stop spending money on that and instead spend money on the free product of theirs that I care about, in exactly the way I want. Google surveilles the entire world, spends huge amounts on lobbying, degrades their own websites on other browsers? Not a peep, usually. For my part, I pay mozilla for their VPN service, which I’m sure many here would decry as useless spending that should be going to firefox instead.
  - Worf 5mo ago
    > It would be amazing if we didn’t have to have this conversation on every single thread about anything related to Firefox. If Firefox starts acting maturely, we can stop having these conversations. Until then we see useless crap in every update while most bugs don't get any meaningful attention. Some changes even made things worse than they were before, for example the new Edit and Resend (not so "new" anymore). If Mozilla starts acting the best interest of the user, stops with the ad BS and doesn't try to be everything all at once and actually focuses on Firefox, I would donate. And so would others. If I donate now, I doubt even 1% of my money would go to anything meaningful, like bug fixing. > Not a peep, usually. No, fuck Google and Chrome and even anything Chromium-based. Here's the peep from me. > For my part, I pay mozilla for their VPN service, which I’m sure many here would decry as useless spending that should be going to firefox instead. Does the profit from the VPN service go to Firefox? If not, what's the point of having a VPN service.
qsera 5mo ago
The flipside of this is that with AI and prompt injection attacks, you don't need a browser vulnerability to be pwned!
legacynl 5mo ago
Mozilla is always looking for new revenue, how likely is it that Anthropic payed for this article?
kittikitti 5mo ago
This is great, and it reflects some of the changes I've seen in the changelogs of Firefox and many others that have utilize Mythos. I'm closely watching a supposed data wall for AI models and this is a clear indicator that AI capabilities can still become much more advanced even at this point in time. It makes me enthusiastic about future releases and optimizations. Thanks for sharing.
languagehacker 5mo ago
I'm having more problems with Firefox 150 than I have had with any other browser update in years. I think I might be the only one though?
- isatis 5mo ago
  On both desktop and Android, I've been getting HTTPS errors that require two Refresh clicks before the actual page loads.
londons_explore 5mo ago
> We fixed a total of 423 security bugs in releases in April. And any one or two of them could have led to a random web ad stealing your ssh keys and installing a keylogger to get into your bank account. I often wonder if we're taking the right route with computer security. Would we be better off having a whole virtual machine for every web page, application or service? Or even physically separate hardware you just vnc into?

https://arstechnica.com/information-technology/2026/05/mozilla-says-271-vulnerabilities-found-by-mythos-have-almost-no-false-positives/ https://arstechnica.com/information-technology/2026/05/mozil...

Diti 5mo ago

I hope to see the day when (or if) the LLMs get so good at spotting and fixing bugs that all that’s left for the Firefox engineers to do is to focus on adding new features. This isn’t sarcasm. Firefox deserves to be used more. Most people I know don’t use it because “Chrome does almost everything better”, and Firefox can’t compete with the other browsers’ roadmaps.

greggoB 5mo ago
> Firefox deserves to be used more Totally agree. I even go as far as choosing which website I make purchases on depending if they work on FF, or writing to support occasionally to tell them it's not supported or a feature isn't working properly and this would be appreciated. I know it pretty much always goes nowhere, but I feel it's what I can do to keep the browser somehow on the radar.
nubinetwork 5mo ago
> Firefox deserves to be used more Part of the problem is, when they stop working on fixing bugs, they start doing Mr Robot things... We just want a web browser. Nobody asked for pocket, or AI... If they use AI to fix all the bugs, then what else is for them to do, other than maintain syntax compatibility with the various languages they build with? They're just going to go back to making the browser trash again.
- FeepingCreature 5mo ago
  I have an old apk of Firefox pinned on mobile. I do this because I genuinely believe that for my very limited usecases, the browser has become actively worse. (Don't worry- I use the system browser for any site I don't fully trust.)
- cobalt60 5mo ago
  when was the last time you made a contribution financially or physically? Those features were economically tied, who's giving the dough?
kakacik 5mo ago
Chrome ain't better in any meaningful way for >99% of use cases. Heck, I am a dev and I use FF with ublock origin only, for past... 10 years? Same with my wife, after I've explained things to her and she understood how different internet experience can be thats the primary browser. So please don't put the argument like 'here is crappy underdog but please use it because monopoly is bad and google is a bit evil', its first class experience in everything I have ever thrown at it. Tripple that on mobile, by far the best mobile and useful mobile experience, bar none.
- freedomben 5mo ago
  Agreed, though many websites only test on Chrome and are unusable on Firefox. Ramp.com and mailgun come immediately to mind. Zoom also won't let you join with browser on Firefox. There's enough that I have to keep a chrome available for those types of sites. It shouldn't be this way, but it is
  - mnicky 5mo ago
    > Zoom also won't let you join with browser on Firefox. FF works for me.
    - freedomben 5mo ago
      do you hack your user agent or anything to get it to show you the option?
      - sfink 5mo ago
        You haven't needed to in quite a while. Well, assuming you're on desktop? I guess I've never tried it on mobile. That feels like asking for trouble. I ran Zoom in my Firefox desktop browser for a while, but it tended to overheat my laptop. Other things overheat it too, so I don't know how much was specific to Zoom on Firefox. I just checked. Still gives me the option ("Join from browser" in a less highlighted option, trying to drive you to their native client I guess.)
IshKebab 5mo ago
Unfortunately we're probably still quite far from that. This is the best case for LLMs still - the quality of their output didn't matter as long as it worked, and there was a near-perfect oracle for checking if their output worked. That's a really good use case for LLMs. It also applies to things like finding proofs in Lean and creating test stimulus. In both cases you know automatically whether the output is good, and it doesn't really matter if it isn't. That isn't the case for most bugs, and definitely isn't the case for actually fixing bugs.
kgwxd 5mo ago
Browser haven't needed new features in a very long time. Extensions were supposed to be the solution to that.
_heimdall 5mo ago
Wouldn't that quality and availability just allow Chrome to pull ahead of Firefox that much faster? If Mozilla created some proprietary LLM or harness that they used internally to outpace Chrome that may be a different story, though I also don't see that happening.

ChrisArchitect 5mo ago

Related: The zero-days are numbered https://news.ycombinator.com/item?id=47853277 https://news.ycombinator.com/item?id=47853277

lschueller 5mo ago

Let's see, how this will improve the daily soc work. I still don't see, what's the big difference between Mythos and Opus, security wise. I'm confident, that this kind of vul detection is a long-term improvement. But does specifically Mythos makes such a big difference to "normal" models? I would love to see, what's the actual difference.

JoshTriplett 5mo ago
Among other things, Mythos seems better at "let me find, weaponize, and stack vulnerabilities until I get end-to-end from untrusted content to root", rather than just finding one thing in a specific identified area.
mccr8 5mo ago
Quantifying the abilities of an LLM is a hard research problem, so I'm not sure if I can describe it in any great way, but Mythos did seem to be fairly clever about putting together things from different domains to find problems. For instance, in one of the included bugs (2022034) it figured out that a floating point value being sent over IPC could be modified by an attacker in such a way that it would be interpreted by the JS engine as an arbitrary pointer, due to the way the JS engine uses a clever representation of values called NaN-boxing. This is not beyond the realm of a human researcher to find, but it did nicely combine different domains of security. As the person responsible for accidentally introducing that security problem (and then fixing it after the Mythos report), while I am aware of NaN-boxing (despite not being a JS engine expert), I was focused more on the other more complex parts of this IPC deserialization code so I hadn't really thought about the potential problems in this context. It is just a floating point value, what could go wrong?
- lschueller 5mo ago
  Okay, so far it makes sense to me. But is the deal with JS and floating point values, which isn't soemthing super special super rare stuff, only detected and identfied by Mythos while Opus wouldn't get to this point?
IainIreland 5mo ago

input_sh 5mo ago

Original source: https://news.ycombinator.com/item?id=48051079 https://news.ycombinator.com/item?id=48051079 It's better because it actually lists a sample of Bugzilla reports that were made public. This topic was discussed previously (36 comments two weeks ago: https://news.ycombinator.com/item?id=47885042 https://news.ycombinator.com/item?id=47885042), but the part about bug reports being made public is brand new.

xacky 5mo ago

I just hope they don't start ignoring human created bug reports, as there are still many that haven't been fixed for years.

MetaverseClub 5mo ago

I'm curious about how did Mozilla do bug finding before Mythos? Did they use any non-AI bug finding tools?

mccr8 5mo ago
The usual sorts of fuzzing and static analyses, using AddressSanitizer and ThreadSanitizer. Also, with a bug bounty program to try to encourage external researchers to report issues. (I work on Firefox security; also I fixed 2 of the bugs linked in the blog post.)
canucker2016 5mo ago
Coverity (similar to lint) scans various open source software products for vulnerabilities. see https://www.blackduck.com/static-analysis-tools-sast/coverity.html https://www.blackduck.com/static-analysis-tools-sast/coverit... and for Firefox-related alleged defects, see https://scan.coverity.com/projects/firefox https://scan.coverity.com/projects/firefox You have to create an account to view the actual reported defects. There are just over 5000 reported defects still outstanding. I don't know how many overlap with the reported 271 Mythos-reported defects.
- rockdoe 5mo ago
  How many of those are false positives though? Probably just over 5000? You get bug bounties if you report the kind of bugs Mythos identified. There's a reason no-one collected bounties from the "5000 defects" Coverity identified. The Mythos reports have several examples of chaining a whole bunch of logic in different parts of the program together to exploit something very subtle. The Coverity reports aren't anything like that. These tools aren't remotely in the same league or even universe.
  - IainIreland 5mo ago
    Yeah, fuzzing, sanitizers, and bug bounties were our main pre-AI tools for finding bugs.
  - 5mo ago

jerrythegerbil 5mo ago

Again, and this is important: A bug is a bug. A “potential vulnerability” is a bug. A vulnerability is verifiable as having security implications with a proof of concept or other substantial evidence. Words matter. Bugs matter. It’s important to fix large amounts of bugs, just as it always has been, and has been done. Let that be impressive on its own, because it IS impressive. Mythos didn’t write 271 PoC for vulnerabilities and demonstrate code path reachability with security implications. Mythos found 271 valid bugs. Let that be enough.

epistasis 5mo ago
I was a bit confused by your definitions, but here's how Mozilla broke out [1] the 271, um, things: > As additional context, we apply security severity ratings from critical to low to indicate the urgency of a bug: > * sec-critical and sec-high are assigned to vulnerabilities that can be triggered with normal user behavior, like browsing to a web page. We make no technical difference between these, but sec-critical bugs are reserved for issues that are publicly disclosed or known to be exploited in the wild. > * sec-moderate is assigned to vulnerabilities that would otherwise be rated sec-high but require unusual and complex steps from the victim. > * sec-low is assigned to bugs that are annoying but far from causing user harm (e.g, a safe crash). > Of the 271 bugs we announced for Firefox 150: 180 were sec-high, 80 were sec-moderate, and 11 were sec-low. Mozilla uses the term "vulnerability" for even sec-high, even though they say right below that it doesn't mean the same thing as a practical exploit. And on their definitional page, they classify even sec-low as "vulnerabilities" [2]. Words are tools, that get their utility from collective meaning. I'd be interested where you recieved your semantics from and if they match up or disagree with Mozilla. [1] https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/ https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin... [2] https://wiki.mozilla.org/Security_Severity_Ratings/Client https://wiki.mozilla.org/Security_Severity_Ratings/Client
- Gregaros 5mo ago
  > Mozilla uses the term "vulnerability" for even sec-high, even though they say right below that it doesn't mean the same thing as a practical exploit. That’s not evident in what you pastedat all. What you pasted says > sec-critical and sec-high are assigned to vulnerabilities that can be triggered with normal user behavior […] We make no technical difference between these […] sec-critical bugs are reserved for issues that are publicly disclosed or known to be exploited in the wild. > sec-low is assigned to bugs that are annoying but far from causing user harm (e.g, a safe crash). From this one infers that the "180 were sec-high" bugs found are actually exploitsble but known to have been found in the wild, and are NOT mere annoying bugs. The difference between 180 and 270 does nothing to deflate the signicance, or lack there of, of the implication re: Mythos.

crummy 5mo ago

Curious if people think LLMs will lead to more secure or less secure software in five years.

int32_64 5mo ago
Both. The skilled will use them to find problems, the unskilled will use them to slopcode insecure software the skilled will have to fix.
deleted 5mo ago
[deleted]
stavros 5mo ago
That depends on which side has more money.
bawolff 5mo ago
One of the biggest issues in security historically imo is vendors who think, well nobody will ever find this bug so we can deprioritize fixing it. LLMs will prevent vendors lying to themselves which will lead to more secure software.
UltraSane 5mo ago
In 5 years attackers have an advantage but in the long run I think more secure if developers use LLMs on software to find and fix all of the worse remotely exploitable bugs before release. LLMs are going to force devs to be much more security conscious.
- canucker2016 5mo ago
  I think it'll be a war of who has the better LLMs-as-security-scanner. Ideally, you'd do a comprehensive all-source-code scan, (and the LLM-scanner finds everything during those scans), and fix all the reported defects. Afterwards, any dev that commits code will run the LLM-scanner on the modified code (and affected areas) and fix any reported defects. So the black-hat hacker would be shut out unless they get access to an LLM-scanner with better analysis than what the target project is using. Major LLM-scanners could give priority access for new versions of LLM-scanners to major projects to find any defects in the current source code before any other party could use the reported defects against the project or their users. So black-hat hackers would be left with developing their own LLM-scanner better/more efficient than existing major LLM-scanners. Given enough incentive, they might develop such a tool. Look at the market for zero-day vulnerabilities for smartphones, esp iPhones.

deferredgrant 5mo ago

[flagged]

mmooss 5mo ago

> “That’s the key thing that has unlocked our ability to operate at the scale we’ve been operating at now,” he said. “It gives the engineer a crank they can pull that says: ‘Yep, this has the problem,’ and then you can iterate on the code and know clearly when you’ve fixed it and eventually land the test case in the tree such that you don’t regress it.” I don't understand much of this paragraph: * "a crank they can pull that says: ‘Yep, this has the problem,’": as in, ring an alarm? Does the LLM ring th alarm? * "you can iterate on the code and know clearly when you’ve fixed it": Isn't that true of most bugs, assuming you do the normal thing and generate a test case? And I thought the LLM output test cases itself: "It will craft test cases. We have our existing fuzzing systems and tools to be able to run those tests" And are they claiming the LLM facilitates iterating? * "and eventually land the test case in the tree": Don't you create the test case before the fix? And just a few words earlier they seemed to be working on the fix, not the test case. And see the prior point about test cases. * "such that you don’t regress it.”: How is the LLM helping here? Maybe I'm missing some fundamental unwritten assumption?

mccr8 5mo ago
Mostly I think this just means that having a test case makes it easier to fix and verify. You can't actually take for granted having a test case when fixing a security bug. Sometimes you only have a crash stack or maybe a vague and hypothetical static analysis result. > eventually land the test case This is just a reference to the fact that we don't land test cases for security bugs immediately in the public repository, to make it harder for attackers. You are right that the LLM only helps with creating the initial test case. Things like running the test case in automation is part of the standard development process.
- mmooss 5mo ago
  Thank you; that makes sense.

rem1099 5mo ago

I don't find that number very high. In a project of the size of Firefox, a new version of a compiler with stricter warnings or a draconian interpretation of the C standard can easily find 200 new bugs. New tools find new bugs, but the oligarchy newspapers report on Mythos and not on clang-22.0.

sfink 5mo ago
The raw number of things found by Claude (Opus or Mythos) was much higher and would be more comparable to a new clang warning. I vaguely remember seeing a number early on in this process that was in the mid-thousands. The 271 is a small, validated subset of that. None of the 271 were deemed false positives iiuc. Most instances of a new clang warning will be false positives. (Same as most of the raw problems reported by the AI.) It is still unclear and open for speculation as to what percentage of all security bugs in Firefox today are being found by the AIs (as opposed to not being found at all). It might be that AI is very good at certain types of problems, even if we can't put our finger on what those types are, and that after the initial wave of bug reports the AI findings will slow to a trickle even while many many other bugs remain in the codebase. Or it might be that AI really does detect most instances of some class of problems and all those bugs will now be gone forever, never to return as long as Mozilla keeps paying the token monster. This is closely related to the oft-asked question "are we better or worse off after both attackers and defenders have access to this new capability?"

tialaramex 5mo ago

They've only linked a few tickets, so of course maybe when we see all 271 actual distinct things the insight won't apply but all those I examined ended up as some C++ code with a nasty bug in it. Firefox is written in several languages, only about 25% of it is in C++ but every single one of these issues seems to touch the C++.

mccr8 5mo ago
A general limitation of this approach is that it is only as good as your validator, and there's nothing easier to validate than a test case that creates, say, an AddressSanitizer use-after-free. For subtler issues will we have to more specific validators or will the LLM become better at coming up with other dangerous conditions it will verify? We'll see.
- tialaramex 5mo ago
  > A general limitation of this approach is that it is only as good as your validator, and there's nothing easier to validate than a test case that creates, say, an AddressSanitizer use-after-free Sure, but, surely AddressSanitizer would also detect the same problem in the C or Rust which together also make up about 25% of Firefox so... ?
jeroenhd 5mo ago
It's possible Mythos is a lot better at finding vulnerabilities in C++ code than it is for other languages. After all, these models are also based on pre-existing security analysis. From what I can tell, a lot of these bugs were hardly C++-specific, they just happened in C++ code. Even the most secure Rust can't magically catch things like TOCTOU issues.
- tialaramex 5mo ago
  > Even the most secure Rust can't magically catch things like TOCTOU issues I suppose it depends what the word "magically" means. A ToCToU race is because you imagined things wouldn't change but they did and in Rust you actually do write fewer patterns with this mistake because of the Mutable xor Aliased rule. If we have at least one immutable reference to a Goose then Rust isn't OK with anybody mutating the Goose, your safe Rust can't do that and unsafe Rust mustn't do that. So the ToCToU race caused by "Oops I forgot somebody else might change the Goose" is less likely because you were made to wrestle with this problem during design - the safe Rust where you just forgot about this doesn't compile.

delichon 5mo ago

In the latest Mission Impossible, saving the world depends on recovering the original software of an escaped superhuman AGI from a sunken Russian submarine. Luther writes a "poison pill" that given the original source will instantly one-shot the AI. We were left to wonder how this magical code could have been written, but now we know. Luthor just wrote a Mythos prompt that handed it the source code and asked for an immutable critical exploit.

gnabgib 5mo ago

16 day old story Wired: Mozilla Used Anthropic's Mythos to Find and Fix 271 Bugs in Firefox (41 points, 18 comments) https://news.ycombinator.com/item?id=47853649 https://news.ycombinator.com/item?id=47853649 Ars: Mozilla: Anthropic's Mythos found 271 security vulnerabilities in Firefox 150 (33 points, 8 comments)https://news.ycombinator.com/item?id=47855384 https://news.ycombinator.com/item?id=47855384

mozdeco 5mo ago
No, it's a new post, see also https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/ https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin...
- gnabgib 5mo ago
  That's this post.. and while it's more detail on the same headline (271 bugs) these discussions look the same as 2 weeks ago (and the same as all the bloggers and podcasters discussed)

OhMeadhbh 5mo ago

When I was at PalmSource, I tried to get budget for CoVerity or Fortify (static code analysis tools.). "Too expensive," my management chain said. I spent another year putting together a deal for a lower cost but limited to scanning the network stack. "No, it's based on BSD and BSD is inherently secure," my management chain said (neither is true, btw.) I eventually left and wound up at Mozilla where there were a number of /* flawfinder ignore */ comments scattered throughout the code. My guess is that Mythos just ignored the "flawfinder ignore" directives and reported the known vulnerabilities in the code.

AndrewDucker 5mo ago
The code is open. If you can prove that's the case you'll have a real news story...
bvisness 5mo ago
Even a quick glance at the bugs revealed in the blog post would quickly disprove your theory.

nnm 5mo ago

I still don't know the exploit count for Mythos. Is it zero, one, or more?

sfink 5mo ago
More, many more. See the bug reports linked in the post. I checked a few, and all of them had an exploit in them, and there are definitely more than 1 bugs listed. Well, depending on how you define "exploit"; some might only read arbitrary pointers or just out of bounds. Those would be useful primitives in a chain of vulnerabilities, not exploits themselves. You'll have to read through the first comments yourself, but if you're hoping that this is all nonsense and ignorable hype, you're going to be disappointed.

kajman 5mo ago

I dismissed the earlier non-technical blog post as shameless product boosterism for Anthropic. The linked hacks blog (which is a better source than this article) is a welcome release. It's hard to deny there's something real to this now, I think. Mozilla's internal definition of a "vulnerability" is also probably more widely applied than what many would intuit, but it is good that these issues are being taken seriously and fixed.

apexalpha 5mo ago
At the same time other companies like AISLE are matching Mythos on vulnerabilities using older models but their own harnass: https://aisle.com/blog/aisle-matches-anthropic-mythos-on-freebsd-zero-days https://aisle.com/blog/aisle-matches-anthropic-mythos-on-fre... So while Mythos certainly is real I think you could do the same with Deepseek pro, GPT 5.5 etc...
- kajman 5mo ago
  Agreed. The earlier blog post did not explicitly claim this, but I think casual viewers were prompted to believe that the Magic of Mythos (TM) went and found (and fixed??) a bunch of vulnerabilities with minimal human guidance, and even contrasted this with their fuzzing infrastructure and made it sound (to me) like it was casting shade on it. This new post makes it pretty clear that this was all bolted on-top of their existing fuzzing infrastructure, and really just used to get more and better initial hits that a very skilled team is looking at. I assume Anthropic was giving them a very good deal on inference for the positive PR, but I believe these other reports and suspect Mozilla did not really need them.
- yorwba 5mo ago
  As the Hacks.Mozilla article notes: "We began with small-scale experiments prompting the harness to look for sandbox escapes with Claude Opus 4.6. Even with this model, we identified an impressive amount of previously-unknown vulnerabilities which required complex reasoning over multiprocess browser engine code."
- jonfw 5mo ago

fg137 5mo ago

What are people's thoughts on how this could affect static analysis tools? I know they are very different beats but often they achieve the same goal. Static analysis tools can be slow, and they report lots of false positives. I wonder if these models will get good + cheap enough so that people rarely reach for static analysis.

sfink 5mo ago
I've been thinking about this. Static analysis tools can also be much faster and most are fully deterministic, so including them in CI can catch bugs or latent bugs before they have a chance to land. I maintain a static analysis tool using in Firefox's CI. False positives have to be fixed or annotated as non-problems in order for you to land a patch in our tree. That means permitting zero positives (false or true), which is a strict threshold. This is a conscious tradeoff; it requires weakening the analysis and getting some false negatives (missed bugs) in order to keep the signal-to-noise ratio high enough that people don't just ignore it and annotate everything away, or stop running it. Nearly all static analysis tools have to do this balancing act. AI, as commonly used, is given more leeway. It's kind of fundamental that it must be allowed to hallucinate false positives; that's the source of much of its power. Which means you need layers of verification and validation on top of it. It can be slow, you'll never be able to say "it catches 100% of the errors of this particular form: ...", and yet it catches so much stuff. Data point: my analysis didn't cover one case that I erroneously thought was unlikely to produce true positives (real bugs), and was more complex to implement than seemed worth the trouble. Opus or Mythos, I'm not sure which, started reporting vulnerabilities stemming from that case, so I scrambled and extended the analysis to cover the gap. It took me long enough to implement that by the time I had a full scan of the source tree, Claude had found every important problem that it reported. The static analysis found several others, and I still honestly don't know whether any of them could ever be triggered in practice. I still think there's value in the static analysis. Some of those occurrences of the problematic pattern might be reachable now through paths too tricky for the AI to construct. Some of them might turn into real problems when other code changes. It seems worth having fixes for all of them now for both possibilities, and also for the lesser reason of not wanting the AI to waste time trying to exploit them. At the same time, clearly the cost/benefit balance has shifted. They could also team up: if I relax my standards and allow my analysis to write an additional warnings report of suspected problems, with the clear expectation that they might be false alarms, then I could feed that list to an AI to validate them. Essentially, feed slop to the slop machine and have it nondeterministically filter out the diamonds in the rough. Food for thought...

benced 5mo ago

Reading this article in the context of the Zig folks refusing to even consider LLM-generated bugs certainly shapes my perspective on what technologies will be in my toolchain.

grumbelbart2 5mo ago
Both are right and it depends on which model you use and who submits those bugs. The capabilities of leading models went from 99% noise to 99% valid bugs in essentially a few months. Some projects are flooded with the former and need to take precautions to avoid essential DoS attacks on the maintainers.
- benced 5mo ago
  Yeah but Zig essentially has till Mythos releases (and arguably less time) to fix their policy. OSS that doesn't take LLM security reports at that point is a liability.
tkel 5mo ago
Zig devs can run Mythos same as anyone else can. I think you are failing to understand the reasoning behind their decision. It's about contributors, not contributions. https://kristoff.it/blog/contributor-poker-and-ai/ https://kristoff.it/blog/contributor-poker-and-ai/
- benced 5mo ago
  Quoting from https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-policy https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-pol...: > Strict No LLM / No AI Policy > No LLMs for issues. > No LLMs for pull requests. > No LLMs for comments on the bug tracker, including translation. English is encouraged, but not required. You are welcome to post in your native language and rely on others to have their own translation tools of choice to interpret your words. If they would accept issues filed by AI or written by AI, they should edit their policy to say that.
  - 5mo ago

danieltanfh95 5mo ago

Really it was not the issue that Opus could not do all these, there was just no incentive to fix bugs. Mythos represented a real marketing use case, so yes thanks for spending money to fix this, but this is not sustainable.

jwr 5mo ago

> Anyone building software can start using a harness with a modern model to find bugs and harden their code today. We recommend getting started now. From what I understand, that is a recipe for getting quickly banned by commercial LLM providers?

Worf 5mo ago

Maybe if Mozilla focused less on new useless features and redesigns, they would be able to focus more on writing secure and bug-free code. I'm not only talking about big things like * Pocket, * several major UI redesigns and * the offline translations, but even tiny useless things like * browser.urlbar.trimURLs, * putting the search query in the URL bar instead of the URL after searching from the URL bar, * messing with the Edit and Resend feature for no reason (the good one that updates the content length is still available at devtools.netmonitor.features.newEditAndResend) and * probably thousands of little shit like this that took a bunch of developer hours to implement. All of the above should've been add-ons. And of course, we know Mozilla spends a lot of money on things unrelated to Firefox at all. It's amazing Firefox is somewhat secure and stable compared to Chrome, which is backed by Google with their infinitely deep pockets. This is a web browser, after all. Something most people use all the time. Something that accepts untrusted input from thousands of sources every day. People use it pretty much every aspect of their lives - banking, personal communication, porn, expressing political opinions. It's used for viewing PDFs, playing media files, for interacting with a whole bunch of APIs (that IMO shouldn't be part of the web, but they are). Security should be top priority.

mplanchard 5mo ago
It would be amazing if we didn’t have to have this conversation on every single thread about anything related to Firefox. Firefox/Mozilla tries literally anything to expand their feature set, customer base, or revenue stream? They need to stop spending money on that and instead spend money on the free product of theirs that I care about, in exactly the way I want. Google surveilles the entire world, spends huge amounts on lobbying, degrades their own websites on other browsers? Not a peep, usually. For my part, I pay mozilla for their VPN service, which I’m sure many here would decry as useless spending that should be going to firefox instead.
- Worf 5mo ago
  > It would be amazing if we didn’t have to have this conversation on every single thread about anything related to Firefox. If Firefox starts acting maturely, we can stop having these conversations. Until then we see useless crap in every update while most bugs don't get any meaningful attention. Some changes even made things worse than they were before, for example the new Edit and Resend (not so "new" anymore). If Mozilla starts acting the best interest of the user, stops with the ad BS and doesn't try to be everything all at once and actually focuses on Firefox, I would donate. And so would others. If I donate now, I doubt even 1% of my money would go to anything meaningful, like bug fixing. > Not a peep, usually. No, fuck Google and Chrome and even anything Chromium-based. Here's the peep from me. > For my part, I pay mozilla for their VPN service, which I’m sure many here would decry as useless spending that should be going to firefox instead. Does the profit from the VPN service go to Firefox? If not, what's the point of having a VPN service.

qsera 5mo ago

The flipside of this is that with AI and prompt injection attacks, you don't need a browser vulnerability to be pwned!

legacynl 5mo ago

Mozilla is always looking for new revenue, how likely is it that Anthropic payed for this article?

kittikitti 5mo ago

This is great, and it reflects some of the changes I've seen in the changelogs of Firefox and many others that have utilize Mythos. I'm closely watching a supposed data wall for AI models and this is a clear indicator that AI capabilities can still become much more advanced even at this point in time. It makes me enthusiastic about future releases and optimizations. Thanks for sharing.

languagehacker 5mo ago

I'm having more problems with Firefox 150 than I have had with any other browser update in years. I think I might be the only one though?

isatis 5mo ago
On both desktop and Android, I've been getting HTTPS errors that require two Refresh clicks before the actual page loads.

londons_explore 5mo ago

> We fixed a total of 423 security bugs in releases in April. And any one or two of them could have led to a random web ad stealing your ssh keys and installing a keylogger to get into your bank account. I often wonder if we're taking the right route with computer security. Would we be better off having a whole virtual machine for every web page, application or service? Or even physically separate hardware you just vnc into?