21 ms·
Improving Firefox stability on Windows by retrying failed memory allocation
- dunno7456 4y agoDoes the issue with giant vsize+wasm-guard-pages (from about:memory) when using uBlock origin causing this behavior of OOM Killer?
- errantmind 4y agoOnly somewhat related but what I'd really like to see is an investigation as to why Firefox takes so long to acquire a connection on Linux. I suppose it could be something particular to my setup but it is very slow in comparison to 'ungoogled-chromium' which I keep around as a second browser. On the network tab I routinely find Firefox in a 'blocked' state for 100ms. On chrome this rarely exceeds 1ms. I've tried messing around with various configuration options and haven't found an answer. The end result is Chrome feeling much snappier.
- 4ensic 4y agoTL;DR. The latest version (105) of FireFox implemented the trick. Users don't need to do anything other than update.
- red_trumpet 4y agoThe trick was implemented in 105, but the latest stable version is 107 already.
- hadrien01 4y ago2023-07-04/115 for ESR users
- zmk5 4y agoAll these hacks blog posts are really interesting insights by Firefox engineers. Browsers in general are such a complicated system, so it's pretty remarkable what they come up with and what they learn when attempting to update Firefox.
- avian 4y agoClick bait title. On Windows, if they get an error on memory allocation, they wait a bit and try again. This gives the OS time to resize the swap file which means that the second try often succeeds.
- Vinnl 4y ago...which is a pretty weird trick, at least for someone like me who has no idea how memory allocation works.
- pmoriarty 4y agoWhy don't more applications use this "weird trick" of just checking to see if memory allocation succeeded and waiting and retrying if it didn't, instead of unceremoniously crashing? It's not like this hasn't been possible to do for many decades already.
- avian 4y agoEven better question is, if it is this simple, why didn't Microsoft implement their malloc or whatever like this in the first case. If the OS needs to resize the page file, why don't they just delay the return of the function until that is done? I'm guessing the answer is something along the lines of "it makes applications freeze for <amount of time> and users don't like that".
- jraph 4y agoSure, they prefer their applications to just crash :-) Bu yes, the problem is probably not totally simple. adding delay in malloc while application might expect an immediate return could cause bugs. But swapping is a thing anyway and introduces delays so I don't know why they would not do this.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- smodo 4y agoI've been a linux/osx user for 20+ years so I'm not familiar with Windows memory management. It'd be interesting to know why MS chose this approach and if it has any benefits? Why not just let userspace request whatever it wants?
- jraph 4y agoWell, over-committing is an arguable choice. "Here's the memory you requested! I don't know if I have it available, but anyway. Here it is!" It turns out it probably works well/better in many cases, because apps don't actually use all the memory they request (up front), and for other reasons, but it's not the obvious choice to make. I would intuitively expect my OS to fail malloc if it does not have any enough memory available if I didn't know better. I would expect an OS capable of expending its swap file to try doing it before failing my malloc call though.
- mariusmg 4y ago>I would expect an OS capable of expending its swap file to try doing it before failing my malloc call though. It takes A LOT of time to expand the swap file. So failing malloc immeditately seems, to me, the right way to handle it. Maybe adding an optional callback to malloc to be notified when further allocations are possible would be a better way to handle this.
- jraph 4y ago> It takes A LOT of time to expand the swap file. So failing malloc immediately seems, to me, the right way to handle it. Maybe it'd be possible to check if expanding the swap file is possible, return and then actually expand the swap file when convenient. (I like being an armchair kernel developer :-))
- bogwog 4y agoBut 99.99999% of the time, when a program calls malloc, it needs it to succeed. So if there is a callback or something to notify that a wait is required or whatever, then 99.99999% of programs are going to do it. That means the high cost of expanding the swap file will be incurred basically every single time...so why not make that the default? In the rare case where a program wants to handle a failed allocation differently, then they should use a native system call that provides a more detailed interface than standard malloc. It doesn't matter if it's not portable since this is really a Windows-only thing. Crashing is not good for anyone. A temporary freeze sucks, sure, but that's what you get for not having enough memory. ...plus, it's not like random freezing is a foreign concept to Windows users.
- skozharinov 4y ago> trimming down Firefox memory consumption Of that Firefox that can fill 16 Gb zram with lz4 compression and all remaining physical RAM with less than 100 tabs loaded?
- phkahler 4y agoWhy do people keep so many tabs open? I usually top out around 5. Bookmarks are a thing, I wonder if just leaving pages open is the new bookmark? But the URL is gone when a tab is closed. IMHO people using so many tabs seems to suggest a shortcoming somewhere else.
- elcomet 4y agoBecause tabs and bookmarks are not the same thing. It's harder to find something in a bookmark, easier to just go through the opened tabs quickly if you know it's there but don't remember the name
- projektfu 4y agoEver try to plan out an electronics project? I often have about 35 datasheets open with a half dozen different price lists.
- genewitch 4y agoweather, two work chats, gmail, hnews, personal mail, 2 fediverse accounts, current work tabs (currently at a low 8 for that), then the ones that change hourly, social media, youtube, wikipedia, searches, etc. Tabs are really starting to bug me, i've tried a few tree tab managers and the ones i tried just don't work. I don't have a better solution. multiple windows is potentially bad news, as a crash may lose tabs you had open, even with a tab session manager. That is if you load several tabs in a new window and the computer or firefox crashes / reboots, you could lose history, cookies, possibly your old window layouts, etc. Tab session managers help a bit, by keeping consistently open tabs saved somewhere, but it doesn't help with new tabs and new windows if there's a crash.
- 4y ago
- the8472 4y agoAfaik memory-mapped files don't count towards the commit limit. Couldn't they roll a custom file-backed allocator (essentially manual swap) to get around the windows limit?
- 323 4y agoNot sure, but probably just the file-backed memory mapped files (as opposed to the anonymous ones) don't count.
- dr-detroit 4y ago
- dundarious 4y agoIt’s good to somewhat limit the amount of memory that gets paged to disk, as it is incredibly slow memory. There should be backpressure on memory allocation. Getting around the OS on this matter would be silly, IMO. We’re talking about an interactive UI environment where massive page files would be doubly bad, swallowing up surprising amounts of disk space in order to provide incredibly slow memory. However, the type of solution you’re talking about would apply well to a caching server, and Varnish (on FreeBSD I believe) uses it quite successfully.
- zyx321 4y agoPlease don't. I've set a maximum swap size per partition, and you are suggesting to bypass it by falsely declaring your swap as cache? Who died and made you NT_AUTHORITY\SYSTEM?
- justsomehnguy 4y agoReminds me how some folk (uTorrent team, AFAIR) thought they are smarter than those guys in Redmond and rolled out their own caching algorithm. Except they weren't in fact smarter and fucked up the memory usage by allocating all the availble memory for a disk cache... pushing everything else including OS to the swap.
- pas 4y ago
- hawski 4y agoFor a few months now I saw a huge improvement on Linux regarding the memory management of Firefox. Previously I had to run Firefox in a separate cgroup to limit its memory usage, because it could easily deplete my whole RAM. And if I did close most of my tabs it did not release the memory back to the system. Now I never actually get to the limit I've set before and also with Auto Tab Discard extension it is well managed. So kudos to the team for such improvements.
- marto1 4y agoCorrect me if I'm wrong, but the blog post is concerning a stability improvement on Windows specifically?
- kaladin_1 4y agoYeah the article is about Windows. Even though you can see here: https://hacks.mozilla.org/author/gsveltomozilla-com/ https://hacks.mozilla.org/author/gsveltomozilla-com/ that they also worked on MacOs and Linux stability this year. I had abandoned Firefox entirely on MacOS until sometime I decided to try again and it was no longer attempting to claim an entire 32GB memory for itself. So, I am back as a happy user :)
- hawski 4y agoIt is, but it just reminded me of this other bit. Sorry if I was a bit too off topic. I just see those improvements on the sidelines and I'm glad.
- Existenceblinks 4y agoMe too, on macOS, I notice memory is freed more.
- chris_wot 4y agoI'm exclusively using Firefox on MacOS now.
- gsvelto 4y ago
- ravenstine 4y agoThe author left off the part where it was invented by a mother and how dentists hate it! /s I suppose it technically improves stability, but the cause seems like a flaw in the Windows operating system, if I'm understanding correctly. > Stalling the main process led to a smaller increase in tab crashes – which are also unpleasant for the user even if not nearly as annoying as a full browser crash – so we’re cutting those down too. I wonder if this has anything to do with my recent experience with Firefox Nightly on macOS. In the last few weeks I started to experience spontaneous tab crashes that couldn't be explained by anything from what I could tell. I could open a new tab with the exact same page and it would manage to not crash. Then I noticed Firefox would sometimes prevent me from viewing tabs until I restarted to update the software. Haven't seen it in the last few days, but it was incredibly frustrating. IMO, Firefox should make a best effort to render a page and not have its core functionality stop working entirely until updating.
- deleted 4y ago[deleted]
- nwah1 4y agoOperating in a low memory environment is inherently about tradeoffs. Where is the blame? Maybe Firefox is too bloated or memory-inefficient. Maybe Mozilla didn't understand Windows's memory management strategy until now. Or maybe Windows is too bloated and memory-inefficient. Or maybe the memory management tradeoffs were suboptimal. Or maybe nobody is to blame, and they are taking advantage of something in a novel way that allows them to squeeze more juice out of the same fruit than others.
- narag 4y agoWhere is the blame? Not sure, but sound stopped working in VLC about one year ago and was restored this month at the same time that dropping files stopped working. Every time after a Windows update and with the same VLC version. I don't think Windows is too bloated or memory-inefficient, but I do believe that they're abusing the AV and "monitoring" stuff. It's a cat & mouse game, with me trying to disable crap and they re-enabling it with every update or simply making impossible to disable annoyances. Also I suspect they're trying to "fix" drivers that worked before the fixes.
- jefftk 4y agoGreat work! This is also a good example of the benefit of telemetry: that they have crash numbers coming back from the field lets them tell that this really did work in practice and get a sense of how much of the problem they've solved.
- 323 4y agoHN mantra: * telemetry is evil * if the product is free (Firefox), you are the product
- xeromal 4y agoSo say we all
- mcdonje 4y agoI get the saying. But in this case, Mozilla gets most of its money from search royalties, primarily from Google. We are Google's product, not Firefox's.
- pbhjpbhj 4y agoIt's both. Firefox sell you to Google, Google sell you to advertisers. It seems Firefox executives take a large chunk of money from Google; presumably to make sure Firefox doesn't do anything too wild that would reduce Googles income.
- derefr 4y ago> presumably to make sure Firefox doesn't do anything too wild I wonder how much of what the Tor Browser version of Firefox does, would be upstreamed to Firefox proper, if not for that deal? (I wonder how much the Firefox team considers Tor Browser to be "the real Firefox" / "Firefox the way we intended", with Firefox itself just being "the sell-out version of Firefox"?)
- Cerium 4y agoInstead of viewing the situation as Firefox selling me down the river to Google, I like to view it as an opportunity for collective bargaining. Firefox is my union representative. They are better able to fight for my interests against Google than should I go it alone and use Chrome.
- brailsafe 4y agoFirefox for macOS has been great for me, but Firefox for Android has been brutally unstable, crashing at least a few times per day.
- pitaj 4y agoYeah I'm very happy with Firefox desktop no matter the OS. The Android version of Firefox is not nearly as polished or capable at the moment. It's a shame because I really do still use it a ton. It has great potential.
- MayeulC 4y agoAh, very few crashes on Android (using it right now), but my biggest gripe is that the first page load can take a while to even start, after opening a new tab or typing something in the address bar. Tapping on stuff lyke the reply/comment button is pretty fast though.
- hinkley 4y agoI’m really not comfortable with people trying to normalize that headline style. This doesn’t work the way adopting kids’ slang “ruins” it and gets them to stop using it. Nobody doing it is worried about looking cool.
- dcow 4y agoI refuse to click the headline because of it.
- layer8 4y agoIt’s not trying to normalize anything, it’s just a tongue-in-cheek pop culture reference.
- alpaca128 4y agoIt's less boring this way and based on the article I think it would be difficult to include more information in the title anyway - the solution is simple but it follows multiple paragraphs describing the problem. Sure, it could also just be "Improving Firefox stability", but that doesn't tell me the nature or scale of the update.
- adql 4y ago> but what if we could recover from this situation instead? Windows automatically resizes the swap file when it’s almost full, increasing the amount of commit space available. Could we use this to our advantage? I expect followup by some annoyed blogger that notices swap file ate his system partition and he spent whole day debugging the reason of that...
- BlueTemplar 4y agoDoesn't Windows still limit swap by default to like maximum 20% of the disk ? And that issue of running out of disk space because swap ate it isn't new, it really came into prominence when we switched from slow but huge HDDs to fast but small SSDs...
- st3fan 4y agoFirefox stability is funny ... I was at Mozilla for 10+ years and used Nightly as my daily driver on my work Mac. I don't think I got more than one or two dozen crashes in total. A crash was a special occasion and I would walk to someone's desk to show an explore. It barely every happened. On Nightly. So much love for all the stability work that is happening.
- mmis1000 4y agoEven for Firefox nightly, it is probably already much more stable and less buggy than tons of games nowadays. We are now in an age that think software bug is a norm. It's respectable that mozilla still keep the standard as high as the past days.
- bigDinosaur 4y agoComparing to games isn't particularly great since games are ultimately low stakes and companies have learnt that gamers will tolerate pretty much any bug that doesn't literally stop the game from working. See: Bethesda games.
- capableweb 4y agoPersonal experiences are funny :) I've been using Firefox on Windows, Linux and Mac for as long as I can remember (first version was maybe Firefox 3 I think? The version that introduced being able to resume downloads I think), and it's the Mac version has always been the most unstable and worst performing one for me, while Linux one being the most stable and performant. I can't even remember the last time the Linux version crashed, while I had the Mac version crash last week. Windows one seems stable, but not as performant as the Linux version.
- julianlam 4y agoSays you! My FF on Linux crashes all the bloody time! It says it needs to RESTART in order to apply updates. Hmph! The audacity. But in all seriousness, I can't recall the last time FF crashed on me. My OS hard locks more often than FF crashes.
- moffkalast 4y ago> with this one weird trick Chromium developers hate him!
- josefresco 4y agoI can't remember the last time Firefox crashed and I've used it daily on Windows since ... the beginning. Are most issues related to stability more common to Linux/MacOS?
- skrowl 4y agoI'm in the same boat. Who are these people who are running out of memory? What on earth are you doing and how old is your machine?
- julienreszka 4y agothey probably have thousands of tabs open
- wkdneidbwf 4y agoand extensions to hell and back
- pdntspa 4y agoFirefox has been solid for me since the dawn of time, and I have or currently run it on Windows, Mac, Linux, and FreeBSD. For a while it used to be kind of slow on Mac and Linux, but I think that was slow graphics calls, which points to a possible issue with the graphics driver I was using. But the last time I checked (many months ago), it was much better.
- com2kid 4y agoI've seen plenty of webgl content that will hard crash FF 100% of the time. Perhaps not FFs fault, could be an underlying driver issue, but ideally web content is sandboxed enough that it can't bring down everything. I once saw a unity web game with a bug that crashed all major browsers across multiple OSes. Go go web tech!
- LoganDark 4y ago
- deleted 4y ago[deleted]
- tabtab 4y agoSounds like this feature should be built into the Windows releases of Firefox. Are there downsides? I almost never had Firefox crash for more than a decade of heavy usage, then starting roughly 2 years ago it got janky quite often. Something changed.
- WalterBright 4y agoThis is why I come to read HN.
- anonymousab 4y agoFor me, there's been some kind of weird instability or resource leak that resulted in Windows Firefox getting slower all around and less stable the longer the application is running. It's been around and 100% reproducible (over an active session of a few days) for a couple of years. The general problem used to feature some sort of bug where some window processes would completely fail to render/paint UI components - instead, rendering them as pure black. The rendering problem is gone, same with a correlated memory leak, but the complete performance slowdown that accompanied it is still there. One day I'll submit a bug report or profiler trace or something, but I find it odd every time I see a post about stability or performance fix, it never happens to be the big one that I run into, regardless of the window device or extensions. It makes me wonder if some users just have browsing habits that most others don't, so they hit obscure bugs more frequently. But since everyone has their own obscure habits, and thus bugs, there's a theoretical endless deluge of problems with no critical mass to justify prioritization or investigation.
- altairprime 4y agoDuly noted that about:crashes is a thing, though I don't know at all how it works or when it decides to collect a crash.
- anonymousab 4y agoOne thing I can say is that it is extremely rare that Firefox actually crashes on me. The instabilities are in the behaviors of the browser, its UI and the tabs/pages themselves. It can slow to a crawl, and even hourglass on me in an extreme case (I usually get fed up and just restart the browser and all the tabs at once to fix the issue before it gets that bad) but it manages to keep itself some, somehow. That said, I'll poke around in there next time anyways and see if anything stands out. Thanks!
- make3 4y agoI use firefox mobile and it has very strange behavior quite often. It's the only browser I would ever use because of the ad block. But sometimes tabs become blacked out and you can't refresh them, you have to close them and start new ones. And sometimes I have two different (consecutive?) youtube tabs, and the second tab basically fuses with the first tab, to where both tabs show the same content and the same video. I wonder if in both of these cases a tab process was killed. It's pretty common
- ecef9-8c0f-4374 4y agoEvery couple of weeks my firefox was crashing tabs one after another. And I always had to restart firefox. This weird trick fixed it. I made the entire firefox directory readonly.
- butz 4y agoThank you for providing more stable experience. Would be great if something could be done about some Unity web games, that sometimes crash whole browser. It is a bit peculiar, while each tab seemingly runs in separate process, but one website crash takes down whole browser.
- nurettin 4y agoHaha I put sleep-retry to every explicit memory allocation in every delphi program I wrote since windows xp and not one person acknowledged it as a good practice. This is just too good.
- MrStonedOne 4y ago
- nottorp 4y ago> The first computer I owned shipped with 128 KiB of RAM and to this day I’m still jarred by the idea that applications can run out of memory given that even 15-year-old machines often shipped with 4 GiB of memory. I think a safe number for the modern web is like 64 Gb. At least if you want to have anything productive open besides the browser...
- habibur 4y agoI assumed while(!ptr) ptr=malloc(size) had been the norm for applications for the last 10 years.
- tga_d 4y agoMaybe I'm misunderstanding your point, but that doesn't address the problem described in the post?
- LoganDark 4y agoHas Firefox stopped allocating so much RAM that it fills up all my 40GB? Overcommit isn't always enabled, because I'm not making a 64GB page file just to keep Firefox running for more than a week. (That is how large Windows tried to make it before I disabled the page file.) On my computer, Firefox running out of memory is its own damn fault. Retrying memory allocation won't fix anything! And running out of commit space crashes everything else too, so Firefox letting itself crash isn't going to help, because it still brings down half the system. (And it still, already crashes when it runs out of memory.)
- iamchp 4y agoAwesome to read about the memory management in Windows. (To me Windows seems to much of a black-box, but this is also because I only develop on Linux)