15 ms·

Firefox Takes the Performance Crown From Chrome

323 points by bpierre 13y ago

leeoniya 13y ago
i'm developing some js compute-heavy apps and Chrome still pulls ahead quite significantly (by as much as 5x) in many cases. i've filed js-perf bugs with mozilla that have been accepted but seem to be quite low on the priority list. artificial benchmarks don't always tell the full story :( if any mozillian js dev is lacking stuff to work on ;) https://bugzilla.mozilla.org/show_bug.cgi?id=879393 https://bugzilla.mozilla.org/show_bug.cgi?id=879393 https://bugzilla.mozilla.org/show_bug.cgi?id=858986 https://bugzilla.mozilla.org/show_bug.cgi?id=858986
3327 13y ago
I did not find this objective, the test scope was limited and the sample set of browsers tested were not the latest version. Not that I use IE but why use IE10 when IE11 is available for testing? If you are doing a performance benchmark browser vs browser you will not be taken seriously if your test is not objective. In this case I find the sample set skewed.
- msujaws 13y ago
  In all fairness, the sample set included all "release" builds of the browsers. The vast majority of users only use release builds, so it makes sense to test them when they get released. It also is important because it includes all performance fixes that will possibly go in to the released product.
- pix64 13y ago
  If I understand correctly, IE11 is only a preview version not the final release.
  - 3327 13y ago
    Correct it is a preview. If that is the case then I am wrong in my assessment and withdraw my statement.
Roboprog 13y ago
I suspect this was "by a nose". But that's good: I hope to see both browsers trading blows in this little war, leap frogging back and forth in the lead.
- zaph0d 13y ago
  Actually not; Firefox beat Chrome comprehensively in the overall test. Chrome actually came 3rd in the list - http://www.tomshardware.com/reviews/chrome-27-firefox-21-opera-next,3534-12.html http://www.tomshardware.com/reviews/chrome-27-firefox-21-ope...
  - magicalist 13y ago
    To be fair, this slashdot story covering the original story (presumably submitted because the original story was submitted and talked about yesterday) mentions the "performance crown", which was just "by a nose". You're right about the overall test, though.
zimbatm 13y ago
The issue with Firefox is that some times the whole browser is freezing. That's what keeps pushing me back to Chrome, not performance issues.
- mcdoh 13y ago
  Chrome freezing is why I'm considering switching back to Firefox.
  - cmpxchg8 13y ago
    Chrome also phones home all the time, which is why I stopped using it. I don't notice much performance difference between Firefox and Chrome anyway, and always use Noscript, Adblock+ and Disconnect.
- matthiasv 13y ago
  That is your issue with Firefox or your setup. I never had this for at least one or two years now, even with misbehaving scripts.
  - ksec 13y ago
    Nope, It is a common issues with Firefox although it has been greatly reduced in the last year or so. The project to fix all this is called e10s. Moving Content, and UI etc to seperate process as well as OMTx ( Off Main Thread Everything ) where Firefox Dev trys to do everything in async style and move them off Main Thread. Progress has been slow. But at least they are working on it
  - driverdan 13y ago
    It's a core problem with FF. Everything runs under the same process so a single site with slow / frozen JS will affect the whole browser.
  - thelukester 13y ago
    matthiasv, I don't know what your secret is, only 1 tab open in 1 window open at time, JS disabled, or what, but this is a very real issue. I have Firefox installed on all 4 of my machines, work desktop, home desktop, laptop, and a netbook. 2 of which are fresh installs. On all 4 when I visit multiple HTML5 heavy sites, FF's UI gets unresponsive. Chrome and newer versions of IE do not suffer from this, and I'm pretty sure it comes from FF running in a single process. Chrome and IE have their issues too, like thrashing memory a lot sooner than FF, which is why FF is still the default browser on my laptop and netbook despite its unresponsive UI under heavy load.
ebbv 13y ago
These types of headlines are so meaningless. On what benchmark on what platform? On OS X Safari outperforms both for many tasks (again, YMMV depending on what you're doing.) But that's meaningless for the billions of Windows and Linux users out there. I wish our industry would move past these kinds of silly headlines. When it first started when Chrome came out there was a massive difference between Chrome and the other browsers, now all of them are very capable and competitive for pretty much anything you want to do.
- mtgx 13y ago
  TomHardware's browser benchmarks are pretty comprehensive. However, what annoys me about this test and this headline, is that Chrome 28, with its new faster Blink core, is literally a week away from being released, which means Firefox will only have its 15 seconds of fame (or rather a week). Why did they make the test immediately after Firefox was out? Or do they repeat the test immediately after each browser version comes out? In that case I'm looking forward to the test with Chrome 28 included. I'm not saying Chrome 28 will necessarily win in the next one. I just find it a little strange that they did it without waiting a bit more for the next Chrome version, too, before writing a headline like that. It just reminds of me those polls who turn out a certain way depending on how you ask the question.
  - mariusmg 13y ago
    Most likely because it takes time to do those benchmarks. After Chrome 28, FF 23 should be out so why not wait for that either ?
    - mtgx 13y ago
      The difference between FF release and next Chrome release is 2 weeks at most. The different between the Chrome release and the next Firefox is 4 weeks. But whatever, I'm not too hung up on this. Good for Firefox, if they can maintain that lead.
  - deleted 13y ago
    [deleted]
evmar 13y ago
(Disclaimer: I used to work on Chrome. I'm more than a year out at this point but I still talk to Chrome developers.) Chrome wins some tests but loses the most on the start-up benchmarks, which is a real bummer. There was a time where we'd spend literally days figuring out why startup regressed by a handful of milliseconds, and this test shows an end-to-end test taking nearly six seconds. One idea I saw mentioned is that there may be a bug related to how Chrome reads the system proxy settings (something about an API change between Windows 7 and Windows 8). I think most of the Chrome developers are still on Windows 7; the bots that are tracking startup performance are definitely still Windows 7 as well. So maybe that number is more of a single bug than a systematic thing. All of the above is not intended to diminish Firefox's impressive results -- just thought I'd provide some background! What these sorts of tests show most is that competition pushes browsers ever faster. If the proxy theory above is right, then maybe this test will inspire Google to invest more into Windows 8 testing, which ultimately benefits users. (Edit: Firefox won other tests too, I also didn't mean to say this was the only factor.)
- _pmf_ 13y ago
  > Chrome wins some tests but loses the most on the start-up benchmarks Well, congratulations to FF for being the perfect browser for the 0.01 percent of people who don't keep their browser open all day.
  - mariusmg 13y ago
    Startup time matters (a lot). Even if i open it once a day i still want it to be fast. Nobody likes slow apps.
    - mcphilip 13y ago
      Agreed. A few years ago there were a lot of complaints about Firefox starting very slowly, especially on Windows machines [1]. For those of us that obsessively close apps that aren't being used, the startup time was a big turn off. [1]. Just google Firefox loads slow to see lots of results like http://support.mozilla.org/en-US/questions/762446 http://support.mozilla.org/en-US/questions/762446
driverdan 13y ago
Actual benchmark story from yesterday: https://news.ycombinator.com/item?id=5970429 https://news.ycombinator.com/item?id=5970429
ajays 13y ago
I wish they (FF) would also spend some time on the small annoyances: Lack of a restart button. Now when I upgrade FF, I have to "kill -9" it from my terminal to get it to restore windows upon restart Memory leaks. Leave yourself logged in to Facebook for a few days, and watch it take up 2GB of memory. No way to easily filter sites with cookies like you can in the next tab over, where you manage sites with saved passwords. Why is this? Does FF secretly want you to not muck with cookies? Now, I'm sure there are addons and plugins for the above. But I should _not_ need addons for basic UX. Save the addons for fancier stuff.
- padenot 13y ago
  * For the restart after upgrade, either it is a bug, or you have not seen the little "restore tabs from last time" button in the lower right part of the browser. * I have not experienced that, I don't use Facebook. * For the cookie stuff, you'll be pleased to learn about the about:permissions feature (type it in your address bar).
  - epmatsw 13y ago
    Whoa, about:permissions is awesome. Thanks for that!
  - Recoil42 13y ago
    I have not experienced that, I don't use Facebook. "Area Man Constantly Mentioning He Doesn't Use Facebook"
  - lazyjones 13y ago
    > about:permissions Interesting, thanks. But while it's open, the list of sites keeps growing for me, apparently various shady/malware sites get added at the bottom continuously. I haven't managed to find it why this happens, but I'm guessing that it may have something to do with the "safe browsing" feature, perhaps all sites in the blacklist get added to about:permissions for some obscure reason. Any ideas?
- epmatsw 13y ago
  If you go to Help > About Firefox when there's an update to be installed, there's a restart button that restores tabs. As for memory leaks, it's been a project for a while now: https://areweslimyet.com/ https://areweslimyet.com/ I'm not entirely sure what the third point is referring to (I just let the browser handle cookies), but in the Privacy tab of Options, you can view and remove individual cookies and search by site. So maybe that helps with what you want?
bd 13y ago
Ehm, at least one of the tests they used is wrong. They used this one as WebGL performance test: http://luic.github.io/WebGL-Performance-Benchmark/ http://luic.github.io/WebGL-Performance-Benchmark/ Despite the name this program in fact doesn't use WebGL, it runs using three.js 2D canvas renderer (just check the source). Additionally even the results they got there are suspicious, I got completely opposite results on my system (Windows 7), with Chrome being faster than Firefox by a large margin: http://www.tomshardware.com/reviews/chrome-27-firefox-21-opera-next,3534-8.html http://www.tomshardware.com/reviews/chrome-27-firefox-21-ope... They got 777 on Firefox vs 437 on Chrome, I got 290 on Firefox vs 441 on Chrome (this is fully in line with my everyday experience, doing browser graphics).
- glomph 13y ago
  Toms hardware seem to consistently have methodology problems in their tests. I don't understand why they are given so much credence.
- deleted 13y ago
  [deleted]
  - bd 13y ago
    Are you sure you are checking the right code? https://dl.dropboxusercontent.com/u/26786471/img/luic-cubes.jpg https://dl.dropboxusercontent.com/u/26786471/img/luic-cubes.... https://github.com/luic/WebGL-Performance-Benchmark/blob/master/benchmark/cubes.html#L138 https://github.com/luic/WebGL-Performance-Benchmark/blob/mas... Also it gets rendering artefacts of 2d canvas renderer, console log confirms it runs CanvasRenderer and performance numbers are not consistent with WebGLRenderer (should be in order of 5,000-7,000 thousands of cubes at 60 fps). Disclaimer: I did write most of WebGLRenderer's code.
    - simonster 13y ago
      Oops, I was looking at the fill rate benchmark, but I see that the cubes benchmark does in fact use the CanvasRenderer, and this is the benchmark Tom's Hardware reports to have used. Sorry for the noise.
brownbat 13y ago
What, no Lynx?
- Argorak 13y ago
  WebGL is not implemented, though 3d renderers to tty exist since the 90s, e.g. ttyquake! A scandal! http://web.archive.org/web/20021113144923/http://webpages.onvoy.com/bobz/ttyquake/ http://web.archive.org/web/20021113144923/http://webpages.on... (original site gone)
- brownbat 13y ago
  Oh, come on. It was a joke.
simonsarris 13y ago
Performance is nice to be sure, but I think other things are much more important in 2013, like consistent HTML Canvas implementations. For one small example the in-canvas kerning on both Firefox and Chrome are awful compared to IE10. From a while ago: http://i.imgur.com/62WBzVZ.png http://i.imgur.com/62WBzVZ.png You can see for yourself here: http://jsfiddle.net/vVC4s/ http://jsfiddle.net/vVC4s/ Notice also the difference between doubling the font and doubling the scale, esp where each line gets cut off: http://jsfiddle.net/jGcrL/ http://jsfiddle.net/jGcrL/ This makes animations rickety and un-smooth (text animation): http://jsfiddle.net/simonsarris/HZFcR/ http://jsfiddle.net/simonsarris/HZFcR/ Surprisingly IE10 handles all of these perfectly. Of course, IE10 has totally broken canvas clipping (non-rectangular clipping regions are impossible. They worked fine in IE9.) Firefox does render large text much better than Chrome, which is why I used it when taking screenshots for my book. But scaling the text (as opposed to setting a larger font) is a disaster. This is just an example. I could go on for days about canvas bugs. I wish there was a bigger push to fix those instead of eking out a performance advantage. To Firefox's huge credit, I've submitted a lot of bugs to Chromium and the FF team, and the FF team consistently gets back to me within a week and usually fixes the bug within a month. The bug reporting experience with Chrome on the other hand is rather disenchanting. For a cross-platform bug example, the context's miterLimit is just plain broken by default in Chrome. I reported this (with examples) back in April and have yet to receive any kind of reply. Thank god it's an easy workaround. https://code.google.com/p/chromium/issues/detail?id=225512 https://code.google.com/p/chromium/issues/detail?id=225512 If you want people to use your browser and develop with it in mind, working elements are more important than being slightly faster than the others. (Good developer tools come in close second I think, at least for this crowd).
- Aloisius 13y ago
  I'm using FireFox 23 on a Mac and the kerning looks just fine on that canvas page for the first example. Aliasing is a little ugly though.
  - kunai 13y ago
    That's odd, considering Firefox clones the antialiasing algorithm used by the host OS and uses that to render type. Are you sure that perhaps the OS font rendering settings are to blame, and not necessarily Firefox?
youngtaff 13y ago
Doing comparative tests of browsers is OK up to a point but in reality the performance problems on the web are due to the way we build and deploy sites.
deleted 13y ago
[deleted]
jumpbug 13y ago
In my daily use, I find Chrome to be overall faster and easier on the computer in general. I have recently run into a case during my development where Chrome actually has trouble compared to Firefox ANNNNNNND IE. It involves a large unordered list with several different divs, buttons, and links in it. There is some serious jumpy scrolling compared to both of the others, which surprisingly render smoothly. This is the only instance I've ever run into where it didn't quite measure up.
humanspecies 13y ago
Thanks Google, for investing on the best web browser out there!
visarga 13y ago
And that's the day I switched back. I abandoned after FF 3.x
jscheel 13y ago
Still has big issues with mouse events between iframes and parent documents. Been an issue since at least 2009. Sad. Edit: issue was reported in 2001. I can't even comprehend that.
mdhgriffiths 13y ago
Websites like "Tom's Hardware" really piss me off.
Pxtl 13y ago
I gave up on Firefox a year or two back because of performance concerns - not raw speed but ram consumption (leading to thrashing when tabbing between ram-intensive processes) and its poor single-threaded freezing problems. Nice to hear that FF is getting better and I'll be able to go back to it - its extension community is far better than Chrome's, and Firebug is without peer.
- krzyk 13y ago
  It's funny, I was using Chrome for about a month (when they finally released a Linux version) but was forced back to Firefox mainly because of massive memory consumption (because I keep a lot of open tabs). The other problem I had with it was lack of AdBlock Plus back then - without it the web is unusable.
- UberMouse 13y ago
  Frankly I think Firefox performs a lot better than Chrome for memory usage now. After a few days of having Chrome open it's usually using between 3 and 6GB of RAM (Usually Facebook is using a gigabyte so killing that tab frees up a lot) and this is with <50 tabs open. Friends who use Firefox end up with ~1GB of RAM usage in a similar timeframe.
speedyrev 13y ago
Go straight to article referenced in Slashdot http://www.tomshardware.com/reviews/chrome-27-firefox-21-opera-next,3534.html http://www.tomshardware.com/reviews/chrome-27-firefox-21-ope...
magicalist 13y ago
I'm really surprised that no one in these comments and the thread on the actual story[1] (instead of this...whatever the equivalent of blogspam is but for slashdot) hasn't mentioned the horrible averaging methodology of this benchmark suite. The individual tests aren't even a problem (though I would maybe pick some more and/or different ones, especially their somewhat odd benchmark choices in performance and graphics), but the averaging makes no sense at all. Averaging time-based benchmarks is problematic enough (it feels more right, but it still isn't a good idea), but how on earth do you convert a "number of times we had to refresh a page" result into a number that you can then average with a "standards compliance" count and a measure of memory efficiency?? Even if you normalize (which it doesn't seem like they did from the output numbers, but maybe they did), the numbers still aren't comparable because you've given no account for the relative magnitude of their effect. e.g. if you have a test of "does the browser have a konami code easter egg?", it doesn't matter if the geometric mean is less sensitive to outliers, because it still doesn't make any sense to take an average of that with "the playback framerate of an HD Trololo video" and then pretend that the average provides any insight. And it's even worse if you then compare that average to other crazy averages! At best you can look at relative ranking, which they actually mention but then proceed to give exact numbers for their relative ranking. There's no information about "betterness" in there, though, except if you then divide up the numbers again to say "these points came from the win in test A, and these points from test B"...at which point you just have the original tests again. Better to count win/no-wins and use that as your final result. Then at least it's obvious that if some tests are much more non-trivial than others that you'll have to give arbitrary weights for the final result, as opposed to having the arbitrary weights being implicit in the tests themselves. Sorry for the rant :) This is a good recognition of Mozilla's hard work, though notably they've been winning many of these tests for a while now (especially the memory ones), but it would be nice if tomshardware could drop the basically meaningless overall scores (or we could just collectively ignore them). [1] https://news.ycombinator.com/item?id=5970429 https://news.ycombinator.com/item?id=5970429
- bzbarsky 13y ago
  I agree the need to produce a single overall number is nuts. We need a better setup for comparing things, somehow... That said, if you _do_ need to produce one in a situation like this, then geometric mean is the least bad way to do it because it does not depend on the normalization of the individual tests at all: if you have 'k' tests and change the workload for one of them by a factor of N, then the average will change by a factor of N^{1/k} no matter what the actual numbers are. This is why it's used so commonly in benchmark that want to produce a single number. Of course this very property is what makes geometric-mean-based benchmarks so gameable: a 20% speedup of an already-fast operation (say eliminating a single instruction) is worth as much for purposes of the benchmark as a 20% speedup in something that's really slow. The result is that browsers keep micro-optimizing already-fast things to win the benchmark game....
Myrth 13y ago
I seriously can't understand how people can use it with its constant single threaded UI freezing. I've just tried nightly and still the same issue!
forgotAgain 13y ago
I just switched from Chrome to FF. The performance is close enough, and good enough for both, that it's a secondary consideration at this point. I switched because I got tired of hearing Chrome constantly accessing my hard drive. I wound up going through the list of Chrome switches here (http://src.chromium.org/svn/trunk/src/chrome/common/chrome_switches.cc http://src.chromium.org/svn/trunk/src/chrome/common/chrome_s...) to try to alleviate it. Some things helped but not to an acceptable level. I use W7. Using procmon I could see Chrome constantly re-reading keys from the registry and writing to temp files even though caching and pre-fetching were disabled. I was also was concerned for Chrome accessing my laptop SSD. Even though I couldn't hear it I could see the lifetime allotment of reads and writes being flushed.
- rjh29 13y ago
  On a similar note, my PC's hard drive starts making crazy noises as soon as Windows 7 goes into screensaver mode. I've turned off every disk-related option I can find and it's still doing... something. Maddeningly it stops as soon as the screensaver is interrupted. Linux is a lot more predictable in that regard...
  - ericabiz 13y ago
    Probably the Windows indexing service. It's configured to run when you're not using your computer. You can turn it off and see if that stops the hard drive noises: http://answers.microsoft.com/en-us/windows/forum/windows_7-files/disable-windows-search-indexer/914286d5-9485-49a9-a3df-f9516c9970cd http://answers.microsoft.com/en-us/windows/forum/windows_7-f...
lesslaw 13y ago
I switched to Chrome to see what the speed was like and I came to the conclusion it was a slicker browser for daily use. But then I reminded myself that some things are more important than raw speed and responsiveness. Google's interest is tracking and targeting. Chrome actively worked against me in this regard. Firefox, despite its financial ties to Google, put control and freedom onto my desktop. A bit of occasional sluggishness is a worthy price to pay. The beer is free at all the browser bars and while I was drinking it, I remembered I liked free as in speech.
- Karunamon 13y ago
  >Google's interest is tracking and targeting. Last I checked, every bit of this can be turned off in the browser settings if they really bother you that much. (Or heck, use Chromium.) In my experience, they offer quite a bit. Typo correction, instant and stupidly fast searches from the address bar, that kind of thing. I'll never understand the pathological hate-on that people have for advertising, I suppose.
  - lesslaw 13y ago
    Advertising is annoying and rude. Flashing away trying to steal my attention from the activity I am trying to pursue. "don't do that, come and do this instead" "skip this ad in 5 seconds" - no thanks The interesting thing about the New Albion was that it was so completely modern in spirit. There was hardly a soul in the firm who was not perfectly well aware that publicity--advertising--is the dirtiest ramp that capitalism has yet produced. In the red lead firm there had still lingered certain notions of commercial honour and usefulness. But such things would have been laughed at in the New Albion. Most of the employees were the hard-boiled, Americanized, go-getting type to whom nothing in the world is sacred, except money. They had their cynical code worked out. The public are swine; advertising is the rattling of a stick inside a swill-bucket. And yet beneath their cynicism there was the final naivete, the blind worship of the money-god. Gordon studied them unobtrusively. As before, he did his work passably well and his fellow-employees looked down on him. Nothing had changed in his inner mind. He still despised and repudiated the money-code. Somehow, sooner or later, he was going to escape from it; even now, after his first fiasco, he still plotted to escape. He was IN the money world, but not OF it. As for the types about him, the little bowler-hatted worms who never turned, and the go-getters, the American business-college gutter-crawlers, they rather amused him than not. He liked studying their slavish keep-your-job mentality. He was the chiel amang them takin' notes.
ekm2 13y ago
I just learned something my math teachers never told me from a tech blog: Geometric mean is useful for comparing when the expected range or units of values is different. For example, startup time is measured in seconds, but BrowsingBench numbers are things like the unitless 6646. The arithmetic mean would fail to "normalize" these values and give disproportionate weight to some over others; the geometric mean is one way of trying to account for this.
mrbill 13y ago
I stopped using Firefox and switched to Chrome when, even on an x86 Mac, I could see the browser's UI redraw itself. Not going back at this point.
deepblueq 13y ago
So much of any comparison like this is about performance, but I really don't think that's as important as it used to be, and that's coming from someone who generally uses slow computers and optimizes software until they run fast. I'm typing this on a bit of an exceptional example, a 2.6 GHz Northwood Pentium 4 with 1 GB of single-channel DDR-400 RAM. The one saving grace is that it has an SSD, but I put the swap file on the spinning drive (which is modern). It's running Linux Mint 14, Xfce edition, with a handful of minor OS-level optimizations. Firefox 21, with a fairly standard configuration, flawlessly handles a dozen or more tabs on a daily basis. It's even pretty snappy, more limited by my internet connection (1.5 Mb DSL) than by the hardware it's running on. If this sorry excuse for a computer does that well, are the relatively minor differences in performance between browsers going to be a big deal on modern hardware? There will be edge cases, such as the people who have hundreds of tabs open at a time, but for the average user I'm having trouble envisioning that. The things that make a difference anymore are very tough to quantify in tests like Tom's Hardware did. I will always have Firefox around because I think Mozilla actually cares about privacy. I use Presto on my phone because it's the only one I've found that renders things how I want. Many people are tied to a browser because of extensions. Standards support doesn't matter until you find a page where a browser doesn't work, and those pages will be different for different people. Browsers can be rock solid on one computer and worthlessly crashy on another. I don't think a round of benchmarks has meant anything to me in browser selection for a long time, and when it did I did them myself so as to account for the computer they were running on. I choose by trying to use a variety of them for a while, and a winner always emerges quite quickly.
Shivetya 13y ago
I just want to know one thing. When I have only two add ons in Firefox why does it take up nearly 200m of memory with no pages displayed? IE is taking 38m with this page displayed, FF has moved to 208m. Firefox is so damn quick to eat memory it causes my laptop to start caching which decreases performance and eats battery.
tommorris 13y ago
The reason I use Firefox is because it has a set of extensions I find useful, and because I like Firefox Sync (end-to-end encryption of bookmarks/history etc.). Whether the browsers shave a few microseconds off JavaScript performance is neither here nor there for me: but perhaps I'm a weirdo. Performance is nice, but not enough of a reason for me to switch browser.

323 points by bpierre 13y ago

i'm developing some js compute-heavy apps and Chrome still pulls ahead quite significantly (by as much as 5x) in many cases. i've filed js-perf bugs with mozilla that have been accepted but seem to be quite low on the priority list. artificial benchmarks don't always tell the full story :( if any mozillian js dev is lacking stuff to work on ;) https://bugzilla.mozilla.org/show_bug.cgi?id=879393 https://bugzilla.mozilla.org/show_bug.cgi?id=879393 https://bugzilla.mozilla.org/show_bug.cgi?id=858986 https://bugzilla.mozilla.org/show_bug.cgi?id=858986

3327 13y ago

I did not find this objective, the test scope was limited and the sample set of browsers tested were not the latest version. Not that I use IE but why use IE10 when IE11 is available for testing? If you are doing a performance benchmark browser vs browser you will not be taken seriously if your test is not objective. In this case I find the sample set skewed.

msujaws 13y ago
In all fairness, the sample set included all "release" builds of the browsers. The vast majority of users only use release builds, so it makes sense to test them when they get released. It also is important because it includes all performance fixes that will possibly go in to the released product.
pix64 13y ago
If I understand correctly, IE11 is only a preview version not the final release.
- 3327 13y ago
  Correct it is a preview. If that is the case then I am wrong in my assessment and withdraw my statement.

Roboprog 13y ago

I suspect this was "by a nose". But that's good: I hope to see both browsers trading blows in this little war, leap frogging back and forth in the lead.

zaph0d 13y ago
Actually not; Firefox beat Chrome comprehensively in the overall test. Chrome actually came 3rd in the list - http://www.tomshardware.com/reviews/chrome-27-firefox-21-opera-next,3534-12.html http://www.tomshardware.com/reviews/chrome-27-firefox-21-ope...
- magicalist 13y ago
  To be fair, this slashdot story covering the original story (presumably submitted because the original story was submitted and talked about yesterday) mentions the "performance crown", which was just "by a nose". You're right about the overall test, though.

zimbatm 13y ago

The issue with Firefox is that some times the whole browser is freezing. That's what keeps pushing me back to Chrome, not performance issues.

mcdoh 13y ago
Chrome freezing is why I'm considering switching back to Firefox.
- cmpxchg8 13y ago
  Chrome also phones home all the time, which is why I stopped using it. I don't notice much performance difference between Firefox and Chrome anyway, and always use Noscript, Adblock+ and Disconnect.
matthiasv 13y ago
That is your issue with Firefox or your setup. I never had this for at least one or two years now, even with misbehaving scripts.
- ksec 13y ago
  Nope, It is a common issues with Firefox although it has been greatly reduced in the last year or so. The project to fix all this is called e10s. Moving Content, and UI etc to seperate process as well as OMTx ( Off Main Thread Everything ) where Firefox Dev trys to do everything in async style and move them off Main Thread. Progress has been slow. But at least they are working on it
- driverdan 13y ago
  It's a core problem with FF. Everything runs under the same process so a single site with slow / frozen JS will affect the whole browser.
- thelukester 13y ago
  matthiasv, I don't know what your secret is, only 1 tab open in 1 window open at time, JS disabled, or what, but this is a very real issue. I have Firefox installed on all 4 of my machines, work desktop, home desktop, laptop, and a netbook. 2 of which are fresh installs. On all 4 when I visit multiple HTML5 heavy sites, FF's UI gets unresponsive. Chrome and newer versions of IE do not suffer from this, and I'm pretty sure it comes from FF running in a single process. Chrome and IE have their issues too, like thrashing memory a lot sooner than FF, which is why FF is still the default browser on my laptop and netbook despite its unresponsive UI under heavy load.

ebbv 13y ago

These types of headlines are so meaningless. On what benchmark on what platform? On OS X Safari outperforms both for many tasks (again, YMMV depending on what you're doing.) But that's meaningless for the billions of Windows and Linux users out there. I wish our industry would move past these kinds of silly headlines. When it first started when Chrome came out there was a massive difference between Chrome and the other browsers, now all of them are very capable and competitive for pretty much anything you want to do.

mtgx 13y ago
TomHardware's browser benchmarks are pretty comprehensive. However, what annoys me about this test and this headline, is that Chrome 28, with its new faster Blink core, is literally a week away from being released, which means Firefox will only have its 15 seconds of fame (or rather a week). Why did they make the test immediately after Firefox was out? Or do they repeat the test immediately after each browser version comes out? In that case I'm looking forward to the test with Chrome 28 included. I'm not saying Chrome 28 will necessarily win in the next one. I just find it a little strange that they did it without waiting a bit more for the next Chrome version, too, before writing a headline like that. It just reminds of me those polls who turn out a certain way depending on how you ask the question.
- mariusmg 13y ago
  Most likely because it takes time to do those benchmarks. After Chrome 28, FF 23 should be out so why not wait for that either ?
  - mtgx 13y ago
    The difference between FF release and next Chrome release is 2 weeks at most. The different between the Chrome release and the next Firefox is 4 weeks. But whatever, I'm not too hung up on this. Good for Firefox, if they can maintain that lead.
- deleted 13y ago
  [deleted]

evmar 13y ago

(Disclaimer: I used to work on Chrome. I'm more than a year out at this point but I still talk to Chrome developers.) Chrome wins some tests but loses the most on the start-up benchmarks, which is a real bummer. There was a time where we'd spend literally days figuring out why startup regressed by a handful of milliseconds, and this test shows an end-to-end test taking nearly six seconds. One idea I saw mentioned is that there may be a bug related to how Chrome reads the system proxy settings (something about an API change between Windows 7 and Windows 8). I think most of the Chrome developers are still on Windows 7; the bots that are tracking startup performance are definitely still Windows 7 as well. So maybe that number is more of a single bug than a systematic thing. All of the above is not intended to diminish Firefox's impressive results -- just thought I'd provide some background! What these sorts of tests show most is that competition pushes browsers ever faster. If the proxy theory above is right, then maybe this test will inspire Google to invest more into Windows 8 testing, which ultimately benefits users. (Edit: Firefox won other tests too, I also didn't mean to say this was the only factor.)

_pmf_ 13y ago
> Chrome wins some tests but loses the most on the start-up benchmarks Well, congratulations to FF for being the perfect browser for the 0.01 percent of people who don't keep their browser open all day.
- mariusmg 13y ago
  Startup time matters (a lot). Even if i open it once a day i still want it to be fast. Nobody likes slow apps.
  - mcphilip 13y ago
    Agreed. A few years ago there were a lot of complaints about Firefox starting very slowly, especially on Windows machines [1]. For those of us that obsessively close apps that aren't being used, the startup time was a big turn off. [1]. Just google Firefox loads slow to see lots of results like http://support.mozilla.org/en-US/questions/762446 http://support.mozilla.org/en-US/questions/762446

driverdan 13y ago

Actual benchmark story from yesterday: https://news.ycombinator.com/item?id=5970429 https://news.ycombinator.com/item?id=5970429

ajays 13y ago

I wish they (FF) would also spend some time on the small annoyances: Lack of a restart button. Now when I upgrade FF, I have to "kill -9" it from my terminal to get it to restore windows upon restart Memory leaks. Leave yourself logged in to Facebook for a few days, and watch it take up 2GB of memory. No way to easily filter sites with cookies like you can in the next tab over, where you manage sites with saved passwords. Why is this? Does FF secretly want you to not muck with cookies? Now, I'm sure there are addons and plugins for the above. But I should _not_ need addons for basic UX. Save the addons for fancier stuff.

padenot 13y ago
* For the restart after upgrade, either it is a bug, or you have not seen the little "restore tabs from last time" button in the lower right part of the browser. * I have not experienced that, I don't use Facebook. * For the cookie stuff, you'll be pleased to learn about the about:permissions feature (type it in your address bar).
- epmatsw 13y ago
  Whoa, about:permissions is awesome. Thanks for that!
- Recoil42 13y ago
  I have not experienced that, I don't use Facebook. "Area Man Constantly Mentioning He Doesn't Use Facebook"
- lazyjones 13y ago
  > about:permissions Interesting, thanks. But while it's open, the list of sites keeps growing for me, apparently various shady/malware sites get added at the bottom continuously. I haven't managed to find it why this happens, but I'm guessing that it may have something to do with the "safe browsing" feature, perhaps all sites in the blacklist get added to about:permissions for some obscure reason. Any ideas?
epmatsw 13y ago
If you go to Help > About Firefox when there's an update to be installed, there's a restart button that restores tabs. As for memory leaks, it's been a project for a while now: https://areweslimyet.com/ https://areweslimyet.com/ I'm not entirely sure what the third point is referring to (I just let the browser handle cookies), but in the Privacy tab of Options, you can view and remove individual cookies and search by site. So maybe that helps with what you want?

bd 13y ago

Ehm, at least one of the tests they used is wrong. They used this one as WebGL performance test: http://luic.github.io/WebGL-Performance-Benchmark/ http://luic.github.io/WebGL-Performance-Benchmark/ Despite the name this program in fact doesn't use WebGL, it runs using three.js 2D canvas renderer (just check the source). Additionally even the results they got there are suspicious, I got completely opposite results on my system (Windows 7), with Chrome being faster than Firefox by a large margin: http://www.tomshardware.com/reviews/chrome-27-firefox-21-opera-next,3534-8.html http://www.tomshardware.com/reviews/chrome-27-firefox-21-ope... They got 777 on Firefox vs 437 on Chrome, I got 290 on Firefox vs 441 on Chrome (this is fully in line with my everyday experience, doing browser graphics).

glomph 13y ago
Toms hardware seem to consistently have methodology problems in their tests. I don't understand why they are given so much credence.
deleted 13y ago
[deleted]
- bd 13y ago
  Are you sure you are checking the right code? https://dl.dropboxusercontent.com/u/26786471/img/luic-cubes.jpg https://dl.dropboxusercontent.com/u/26786471/img/luic-cubes.... https://github.com/luic/WebGL-Performance-Benchmark/blob/master/benchmark/cubes.html#L138 https://github.com/luic/WebGL-Performance-Benchmark/blob/mas... Also it gets rendering artefacts of 2d canvas renderer, console log confirms it runs CanvasRenderer and performance numbers are not consistent with WebGLRenderer (should be in order of 5,000-7,000 thousands of cubes at 60 fps). Disclaimer: I did write most of WebGLRenderer's code.
  - simonster 13y ago
    Oops, I was looking at the fill rate benchmark, but I see that the cubes benchmark does in fact use the CanvasRenderer, and this is the benchmark Tom's Hardware reports to have used. Sorry for the noise.

brownbat 13y ago

What, no Lynx?

Argorak 13y ago
WebGL is not implemented, though 3d renderers to tty exist since the 90s, e.g. ttyquake! A scandal! http://web.archive.org/web/20021113144923/http://webpages.onvoy.com/bobz/ttyquake/ http://web.archive.org/web/20021113144923/http://webpages.on... (original site gone)
brownbat 13y ago
Oh, come on. It was a joke.

simonsarris 13y ago

Performance is nice to be sure, but I think other things are much more important in 2013, like consistent HTML Canvas implementations. For one small example the in-canvas kerning on both Firefox and Chrome are awful compared to IE10. From a while ago: http://i.imgur.com/62WBzVZ.png http://i.imgur.com/62WBzVZ.png You can see for yourself here: http://jsfiddle.net/vVC4s/ http://jsfiddle.net/vVC4s/ Notice also the difference between doubling the font and doubling the scale, esp where each line gets cut off: http://jsfiddle.net/jGcrL/ http://jsfiddle.net/jGcrL/ This makes animations rickety and un-smooth (text animation): http://jsfiddle.net/simonsarris/HZFcR/ http://jsfiddle.net/simonsarris/HZFcR/ Surprisingly IE10 handles all of these perfectly. Of course, IE10 has totally broken canvas clipping (non-rectangular clipping regions are impossible. They worked fine in IE9.) Firefox does render large text much better than Chrome, which is why I used it when taking screenshots for my book. But scaling the text (as opposed to setting a larger font) is a disaster. This is just an example. I could go on for days about canvas bugs. I wish there was a bigger push to fix those instead of eking out a performance advantage. To Firefox's huge credit, I've submitted a lot of bugs to Chromium and the FF team, and the FF team consistently gets back to me within a week and usually fixes the bug within a month. The bug reporting experience with Chrome on the other hand is rather disenchanting. For a cross-platform bug example, the context's miterLimit is just plain broken by default in Chrome. I reported this (with examples) back in April and have yet to receive any kind of reply. Thank god it's an easy workaround. https://code.google.com/p/chromium/issues/detail?id=225512 https://code.google.com/p/chromium/issues/detail?id=225512 If you want people to use your browser and develop with it in mind, working elements are more important than being slightly faster than the others. (Good developer tools come in close second I think, at least for this crowd).

Aloisius 13y ago
I'm using FireFox 23 on a Mac and the kerning looks just fine on that canvas page for the first example. Aliasing is a little ugly though.
- kunai 13y ago
  That's odd, considering Firefox clones the antialiasing algorithm used by the host OS and uses that to render type. Are you sure that perhaps the OS font rendering settings are to blame, and not necessarily Firefox?

youngtaff 13y ago

Doing comparative tests of browsers is OK up to a point but in reality the performance problems on the web are due to the way we build and deploy sites.

deleted 13y ago

[deleted]

jumpbug 13y ago

In my daily use, I find Chrome to be overall faster and easier on the computer in general. I have recently run into a case during my development where Chrome actually has trouble compared to Firefox ANNNNNNND IE. It involves a large unordered list with several different divs, buttons, and links in it. There is some serious jumpy scrolling compared to both of the others, which surprisingly render smoothly. This is the only instance I've ever run into where it didn't quite measure up.

humanspecies 13y ago

Thanks Google, for investing on the best web browser out there!

visarga 13y ago

And that's the day I switched back. I abandoned after FF 3.x

jscheel 13y ago

Still has big issues with mouse events between iframes and parent documents. Been an issue since at least 2009. Sad. Edit: issue was reported in 2001. I can't even comprehend that.

mdhgriffiths 13y ago

Websites like "Tom's Hardware" really piss me off.

Pxtl 13y ago

I gave up on Firefox a year or two back because of performance concerns - not raw speed but ram consumption (leading to thrashing when tabbing between ram-intensive processes) and its poor single-threaded freezing problems. Nice to hear that FF is getting better and I'll be able to go back to it - its extension community is far better than Chrome's, and Firebug is without peer.

krzyk 13y ago
It's funny, I was using Chrome for about a month (when they finally released a Linux version) but was forced back to Firefox mainly because of massive memory consumption (because I keep a lot of open tabs). The other problem I had with it was lack of AdBlock Plus back then - without it the web is unusable.
UberMouse 13y ago
Frankly I think Firefox performs a lot better than Chrome for memory usage now. After a few days of having Chrome open it's usually using between 3 and 6GB of RAM (Usually Facebook is using a gigabyte so killing that tab frees up a lot) and this is with <50 tabs open. Friends who use Firefox end up with ~1GB of RAM usage in a similar timeframe.

speedyrev 13y ago

Go straight to article referenced in Slashdot http://www.tomshardware.com/reviews/chrome-27-firefox-21-opera-next,3534.html http://www.tomshardware.com/reviews/chrome-27-firefox-21-ope...

magicalist 13y ago

I'm really surprised that no one in these comments and the thread on the actual story[1] (instead of this...whatever the equivalent of blogspam is but for slashdot) hasn't mentioned the horrible averaging methodology of this benchmark suite. The individual tests aren't even a problem (though I would maybe pick some more and/or different ones, especially their somewhat odd benchmark choices in performance and graphics), but the averaging makes no sense at all. Averaging time-based benchmarks is problematic enough (it feels more right, but it still isn't a good idea), but how on earth do you convert a "number of times we had to refresh a page" result into a number that you can then average with a "standards compliance" count and a measure of memory efficiency?? Even if you normalize (which it doesn't seem like they did from the output numbers, but maybe they did), the numbers still aren't comparable because you've given no account for the relative magnitude of their effect. e.g. if you have a test of "does the browser have a konami code easter egg?", it doesn't matter if the geometric mean is less sensitive to outliers, because it still doesn't make any sense to take an average of that with "the playback framerate of an HD Trololo video" and then pretend that the average provides any insight. And it's even worse if you then compare that average to other crazy averages! At best you can look at relative ranking, which they actually mention but then proceed to give exact numbers for their relative ranking. There's no information about "betterness" in there, though, except if you then divide up the numbers again to say "these points came from the win in test A, and these points from test B"...at which point you just have the original tests again. Better to count win/no-wins and use that as your final result. Then at least it's obvious that if some tests are much more non-trivial than others that you'll have to give arbitrary weights for the final result, as opposed to having the arbitrary weights being implicit in the tests themselves. Sorry for the rant :) This is a good recognition of Mozilla's hard work, though notably they've been winning many of these tests for a while now (especially the memory ones), but it would be nice if tomshardware could drop the basically meaningless overall scores (or we could just collectively ignore them). [1] https://news.ycombinator.com/item?id=5970429 https://news.ycombinator.com/item?id=5970429

bzbarsky 13y ago
I agree the need to produce a single overall number is nuts. We need a better setup for comparing things, somehow... That said, if you _do_ need to produce one in a situation like this, then geometric mean is the least bad way to do it because it does not depend on the normalization of the individual tests at all: if you have 'k' tests and change the workload for one of them by a factor of N, then the average will change by a factor of N^{1/k} no matter what the actual numbers are. This is why it's used so commonly in benchmark that want to produce a single number. Of course this very property is what makes geometric-mean-based benchmarks so gameable: a 20% speedup of an already-fast operation (say eliminating a single instruction) is worth as much for purposes of the benchmark as a 20% speedup in something that's really slow. The result is that browsers keep micro-optimizing already-fast things to win the benchmark game....

Myrth 13y ago

I seriously can't understand how people can use it with its constant single threaded UI freezing. I've just tried nightly and still the same issue!

forgotAgain 13y ago

I just switched from Chrome to FF. The performance is close enough, and good enough for both, that it's a secondary consideration at this point. I switched because I got tired of hearing Chrome constantly accessing my hard drive. I wound up going through the list of Chrome switches here (http://src.chromium.org/svn/trunk/src/chrome/common/chrome_switches.cc http://src.chromium.org/svn/trunk/src/chrome/common/chrome_s...) to try to alleviate it. Some things helped but not to an acceptable level. I use W7. Using procmon I could see Chrome constantly re-reading keys from the registry and writing to temp files even though caching and pre-fetching were disabled. I was also was concerned for Chrome accessing my laptop SSD. Even though I couldn't hear it I could see the lifetime allotment of reads and writes being flushed.

rjh29 13y ago
On a similar note, my PC's hard drive starts making crazy noises as soon as Windows 7 goes into screensaver mode. I've turned off every disk-related option I can find and it's still doing... something. Maddeningly it stops as soon as the screensaver is interrupted. Linux is a lot more predictable in that regard...
- ericabiz 13y ago
  Probably the Windows indexing service. It's configured to run when you're not using your computer. You can turn it off and see if that stops the hard drive noises: http://answers.microsoft.com/en-us/windows/forum/windows_7-files/disable-windows-search-indexer/914286d5-9485-49a9-a3df-f9516c9970cd http://answers.microsoft.com/en-us/windows/forum/windows_7-f...

lesslaw 13y ago

I switched to Chrome to see what the speed was like and I came to the conclusion it was a slicker browser for daily use. But then I reminded myself that some things are more important than raw speed and responsiveness. Google's interest is tracking and targeting. Chrome actively worked against me in this regard. Firefox, despite its financial ties to Google, put control and freedom onto my desktop. A bit of occasional sluggishness is a worthy price to pay. The beer is free at all the browser bars and while I was drinking it, I remembered I liked free as in speech.

Karunamon 13y ago
>Google's interest is tracking and targeting. Last I checked, every bit of this can be turned off in the browser settings if they really bother you that much. (Or heck, use Chromium.) In my experience, they offer quite a bit. Typo correction, instant and stupidly fast searches from the address bar, that kind of thing. I'll never understand the pathological hate-on that people have for advertising, I suppose.
- lesslaw 13y ago
  Advertising is annoying and rude. Flashing away trying to steal my attention from the activity I am trying to pursue. "don't do that, come and do this instead" "skip this ad in 5 seconds" - no thanks The interesting thing about the New Albion was that it was so completely modern in spirit. There was hardly a soul in the firm who was not perfectly well aware that publicity--advertising--is the dirtiest ramp that capitalism has yet produced. In the red lead firm there had still lingered certain notions of commercial honour and usefulness. But such things would have been laughed at in the New Albion. Most of the employees were the hard-boiled, Americanized, go-getting type to whom nothing in the world is sacred, except money. They had their cynical code worked out. The public are swine; advertising is the rattling of a stick inside a swill-bucket. And yet beneath their cynicism there was the final naivete, the blind worship of the money-god. Gordon studied them unobtrusively. As before, he did his work passably well and his fellow-employees looked down on him. Nothing had changed in his inner mind. He still despised and repudiated the money-code. Somehow, sooner or later, he was going to escape from it; even now, after his first fiasco, he still plotted to escape. He was IN the money world, but not OF it. As for the types about him, the little bowler-hatted worms who never turned, and the go-getters, the American business-college gutter-crawlers, they rather amused him than not. He liked studying their slavish keep-your-job mentality. He was the chiel amang them takin' notes.

ekm2 13y ago

I just learned something my math teachers never told me from a tech blog: Geometric mean is useful for comparing when the expected range or units of values is different. For example, startup time is measured in seconds, but BrowsingBench numbers are things like the unitless 6646. The arithmetic mean would fail to "normalize" these values and give disproportionate weight to some over others; the geometric mean is one way of trying to account for this.

mrbill 13y ago

I stopped using Firefox and switched to Chrome when, even on an x86 Mac, I could see the browser's UI redraw itself. Not going back at this point.

deepblueq 13y ago

So much of any comparison like this is about performance, but I really don't think that's as important as it used to be, and that's coming from someone who generally uses slow computers and optimizes software until they run fast. I'm typing this on a bit of an exceptional example, a 2.6 GHz Northwood Pentium 4 with 1 GB of single-channel DDR-400 RAM. The one saving grace is that it has an SSD, but I put the swap file on the spinning drive (which is modern). It's running Linux Mint 14, Xfce edition, with a handful of minor OS-level optimizations. Firefox 21, with a fairly standard configuration, flawlessly handles a dozen or more tabs on a daily basis. It's even pretty snappy, more limited by my internet connection (1.5 Mb DSL) than by the hardware it's running on. If this sorry excuse for a computer does that well, are the relatively minor differences in performance between browsers going to be a big deal on modern hardware? There will be edge cases, such as the people who have hundreds of tabs open at a time, but for the average user I'm having trouble envisioning that. The things that make a difference anymore are very tough to quantify in tests like Tom's Hardware did. I will always have Firefox around because I think Mozilla actually cares about privacy. I use Presto on my phone because it's the only one I've found that renders things how I want. Many people are tied to a browser because of extensions. Standards support doesn't matter until you find a page where a browser doesn't work, and those pages will be different for different people. Browsers can be rock solid on one computer and worthlessly crashy on another. I don't think a round of benchmarks has meant anything to me in browser selection for a long time, and when it did I did them myself so as to account for the computer they were running on. I choose by trying to use a variety of them for a while, and a winner always emerges quite quickly.

Shivetya 13y ago

I just want to know one thing. When I have only two add ons in Firefox why does it take up nearly 200m of memory with no pages displayed? IE is taking 38m with this page displayed, FF has moved to 208m. Firefox is so damn quick to eat memory it causes my laptop to start caching which decreases performance and eats battery.

tommorris 13y ago

The reason I use Firefox is because it has a set of extensions I find useful, and because I like Firefox Sync (end-to-end encryption of bookmarks/history etc.). Whether the browsers shave a few microseconds off JavaScript performance is neither here nor there for me: but perhaps I'm a weirdo. Performance is nice, but not enough of a reason for me to switch browser.