8 ms·

Like many of the other Firefox memory changes, this one leaves me feeling a little ambiguous. On the one hand, obviously keeping memory usage down is good. On t

Like many of the other Firefox memory changes, this one leaves me feeling a little ambiguous. On the one hand, obviously keeping memory usage down is good. On the other hand, waiting to decompress/decode images until they are visible to the user means that there are obviously going to be performance trade-offs (and also, obviously, design/engineering trade-offs). Speaking just from my personal usage: I don't think there has ever been a time where delaying the image decompression on a page would ever have had any utility for me (at least from a memory standpoint). On the other hand, there have been plenty of times when eagerly performing image decompression would make my browsing a bit smoother. All in all, it seems (from a not-particularly-in-touch outsider perspective) as if Mozilla went from being totally dismissive of memory complaints to being hyper-responsive to memory complaints with no (publicly stated) rationale behind the change. It would be interesting to see the reasoning behind this (if only from an intellectual standpoint), particularly given that in many cases I tend to feel that it would have been better to simply be a bit more diplomatic when rejecting memory-based bug reports.

dbaupp 13y ago
I imagine it's good for image heavy sites like Pinterest or Tumblr.
- mikebonnell 13y ago
  Agreed, it's a huge win there.
daleharvey 13y ago
With the capbility being built, if there is a performance tradeoff (I didnt look through the bugs, but imagine it was already thought about), then its now possible to do 'hey you have a load of spare memory' pre decode all the things type optimisations
- mnarayan01 13y ago
  Good point. I imagine the issues would be more with when to unload an image, though depending on the implementation that might not be particularly problematic either.
sanxiyn 13y ago
https://wiki.mozilla.org/Performance/MemShrink https://wiki.mozilla.org/Performance/MemShrink has some reasons. One reason is explicitly "Fewer people will complain that Firefox is a memory hog and that Mozilla ignores memory usage". Even if complaints are technically nonsense, addressing complaints is still important.
- mnarayan01 13y ago
  Good link...thanks. > Even if complaints are technically nonsense, addressing complaints is still important. Totally true, but I don't have to like it...and I can fight against it by complaining about the addressing of the complaints!
  - badman_ting 13y ago
    And thank goodness for that.
AshleysBrain 13y ago
If image decoding is asynchronous (don't know - guessing it is already?) and you decode them when they get near to the viewport and not just when they become visible, then it should always be decoded by the time you scroll to it and yet never jank the page. Scrolling a long way really fast probably means there's a small delay while it decodes images, but surely that's worth it to save gigabytes of memory?
- mccr8 13y ago
  Image decoding in Firefox was recently made multithreaded.
- mnarayan01 13y ago
  > but surely that's worth it to save gigabytes of memory I'm not sure _save_ is the right word here...using available memory has no particular cost to me over not using it. Of course, the _available_ modifier is relevant, so I take your point.
  - scholia 13y ago
    It's a huge advance for those of us who usually have 50-60 tabs open for weeks at a time, where hitting a really heavy image page would tip Firefox over the top and crash it. I switched from Chrome to Firefox a couple of months ago for its ability to handle more tabs without slowing/crashing, and for its better performance with image-heavy pages. So, for me, it's not just "who cares?" it's a killer feature.
    - Touche 13y ago
      Tabs are not bookmarks. Use bookmarks.
      - dbaupp 13y ago
        Why should one use bookmarks rather than many tabs? I have many concurrent projects going, and things I refer to regularly, so I just leave them grouped in my tree of tabs on the sidebar. Much more efficient than digging through nested bookmark menus, or trying to make sure the bookmarks are properly classified. Basically, my tabs are bookmarks, except tabs are integrated into the web-browser better.
dkhenry 13y ago
I have been running aurora for a while so I already have this fix. I just tested it and there was no noticeable performance impact for the memory saved. Chrome on the other hand used all my memory and caused me to start swapping. Any website with lots of images loaded will be impacted by this ( facebook , G+ , tumblr , ... ) so I think its a rather important change
akjj 13y ago
This was a follow-up to an earlier change which also delayed image decompressing. In the description of that change the author wrote: "Andrew McCreight wrote a comment that I can’t find now, but basically he said that on one of his machines, that page was extremely slow and painful to scroll through before the change, and after the change it was smooth and pleasant." If you have any examples where the linked changes causes performance degradation, you should try reporting it to the developers.
- smackfu 13y ago
  I assume the old way choked very badly once you ran out of real RAM.
NathanKP 13y ago
We've been using stuff like Infinity.js (http://airbnb.github.io/infinity/ http://airbnb.github.io/infinity/) for quite a while now, and it's basic functionality is to improve long scrolling panes by dynamically adding and removing elements (including images) in real time as you scroll to keep the number of elements low and the page responsive. Sites that use this technique usually feel just as responsive as if the site really did have a page with 10,000 items on it instead of a dynamic viewpoint that only shows 100 items as a time. Personally I think it would be best if HTML developed a spec for an official UITableView style native element. That would provide a way to solve the memory issues and avoid the need for JavaScript hacks like Infinity.js But in the meantime browser workarounds like this one from Firefox will help avoid the huge memory usage in sites that don't use smart techniques like Infinity.js
- swift 13y ago
  I don't know that I'd characterize this as a workaround. Infinity.js is definitely a good approach, but you should expect good baseline performance from the browser even without more complicated techniques.
  - NathanKP 13y ago
    I expect good baseline performance from a combination of both the HTML and JavaScript code of the page, and the browser. Sure it's nice if the browser does some tricks to make a badly written page that has 2 GB of image data on it play somewhat more nicely on the system, but if that page loads badly its not the browsers fault, its the fault of the web developer who didn't paginate, or use Infinity.js or something like that. The reason why I call the Firefox fix a workaround is because the proper way to solve the problem of 2 GB of image data on a page is by using a UITableView style wrapper. But HTML 5 doesn't have such an element for us to make use of. The proper fix would be to add such an element to HTML itself, but obviously that's something that takes years to do and to get it adopted across all browsers. So in the meantime tricks like what Firefox is doing are nice to help prevent inexperienced web developers who create abominably inefficient webpages from consuming too much system resources when an unfortunate victim downloads the page.
driverdan 13y ago
The article states that it reduces initial page load time by about 4 seconds. Load time is far more important than a few ms here and there to load images on the fly when they are needed.
joosters 13y ago
Image decompression is pretty easy, CPU-wise. Well over a decade ago, computers were happily rendering JPEGs on-the-fly without storing a decompressed version (RISC OS apps did this). Your desktop/laptop/phone has plenty of ability to do this.
Lagged2Death 13y ago
...it seems ... as if Mozilla went from being totally dismissive of memory complaints to being hyper-responsive to memory complaints with no (publicly stated) rationale behind the change. I don't know what their internal rationale is. "Memory is cheap," we like to say, but it's really only true if you're not wasting it like crazy all the time. Ten years ago you could surf the web with 512MB of RAM; now we're getting to the point where 4GB isn't considered enough. If 4GB now costs the same as 512MB then, then in the sense of "dollars worth of memory required to perform function X acceptably," memory hasn't actually gotten any cheaper. If you can get performance that used to require 4GB down into 1GB, you've upgraded the performance of a bunch of older and low-budget devices from "barely usable" to "fine." Take the big-picture view: updates like this can actually result in a slower flow of "obsolete" computers and devices into the landfills. This is one of those bottlenecks where a little good engineering can have really disproportionate positive aggregate effects. This sort of thinking has been one of the selling points of desktop Linux for a long time. I wish Microsoft thought more like this.
- mindcrime 13y ago
  Right. Also, no matter how cheap memory is, you run into limits of things like maximum RAM capacity. I'm there now... I have 8GB of RAM in this laptop, and that's all it can hold. I can't spend more money to put more RAM in here even if I want too, as it just won't take it. I'd love to go up to 16GB, but that won't happen until I buy a whole new machine. Needless to say, I'm very happy about this firefox news. AS somebody who routinely runs a lot of apps (and more than a few memory hungry ones) I want every app to be a freaking memory frugal as it possibly can, and be a good citizen when RAM has to be shared between apps. Playing the whole "Unused RAM is wasted, take up as much RAM as you can" mindset is fine if you only run one app at a time, but, really, who the f%!# actually does that???
dragonwriter 13y ago
> All in all, it seems (from a not-particularly-in-touch outsider perspective) as if Mozilla went from being totally dismissive of memory complaints to being hyper-responsive to memory complaints with no (publicly stated) rationale behind the change. I suspect that being a company that wants to have a unified code base for their browser and that is now newly pushing a browser-centric OS for smartphones may explain this, at least in part.
- nnethercote 13y ago
  I first started working on improving memory consumption in early 2011, and that evolved into the MemShrink project (https://wiki.mozilla.org/Performance/MemShrink https://wiki.mozilla.org/Performance/MemShrink) a couple of months after that. Firefox OS came later. In fact, it's fortunate that MemShrink had been up and running for a while because it meant we had already fixed a lot of leaks and inefficiencies, and we had good tools for identifying memory problems in Firefox OS.
nnethercote 13y ago
Have you tried the new version? You should try it before you dismiss it due to theoretical concerns. As I wrote in the blog post, the time before the page displayed dropped from 5 seconds to 1 seconds. And scrolling performance is unaffected AFAIK; if you have evidence otherwise Timothy and I would love to hear about it.
- illumen 13y ago
  Firstly... thanks! I love firefox, and appreciate any memory savings. Did you do any tests for animation performance, and test that preloaders are not affected? Are these issues addressed? Hardware accelerated decoding(like IE, and image viewers use) would help. Much modern hardware can do the whole decode-resize step in dedicated hardware. For software only, an improvement is resizing in jpeg-space to the exact size required. It is much faster than decoding the entire jpeg (eg 5 megapixels 2MB jpeg, into 20MB of uncompressed image data) then only displaying it at 320x200 pixels(1MB at the 16bit needed for this YUV data). You can resize the smaller jpeg-space data, then not have to allocate that 20MB of data (perhaps never going back to the OS). As well, some images compress very nicely with hardware texture compression (but many look awful). There are heuristics that can be used to decide the best type of texture to use based on the image data. Why turn 6bit-16bit per pixel jpeg data into 32 bit per pixel data? It only takes more memory and is slower to draw. Surely jpeg usage is common enough to special case its handling? Good luck!

dbaupp 13y ago

I imagine it's good for image heavy sites like Pinterest or Tumblr.

mikebonnell 13y ago
Agreed, it's a huge win there.

daleharvey 13y ago

With the capbility being built, if there is a performance tradeoff (I didnt look through the bugs, but imagine it was already thought about), then its now possible to do 'hey you have a load of spare memory' pre decode all the things type optimisations

mnarayan01 13y ago
Good point. I imagine the issues would be more with when to unload an image, though depending on the implementation that might not be particularly problematic either.

sanxiyn 13y ago

https://wiki.mozilla.org/Performance/MemShrink https://wiki.mozilla.org/Performance/MemShrink has some reasons. One reason is explicitly "Fewer people will complain that Firefox is a memory hog and that Mozilla ignores memory usage". Even if complaints are technically nonsense, addressing complaints is still important.

mnarayan01 13y ago
Good link...thanks. > Even if complaints are technically nonsense, addressing complaints is still important. Totally true, but I don't have to like it...and I can fight against it by complaining about the addressing of the complaints!
- badman_ting 13y ago
  And thank goodness for that.

AshleysBrain 13y ago

If image decoding is asynchronous (don't know - guessing it is already?) and you decode them when they get near to the viewport and not just when they become visible, then it should always be decoded by the time you scroll to it and yet never jank the page. Scrolling a long way really fast probably means there's a small delay while it decodes images, but surely that's worth it to save gigabytes of memory?

mccr8 13y ago
Image decoding in Firefox was recently made multithreaded.
mnarayan01 13y ago
> but surely that's worth it to save gigabytes of memory I'm not sure _save_ is the right word here...using available memory has no particular cost to me over not using it. Of course, the _available_ modifier is relevant, so I take your point.
- scholia 13y ago
  It's a huge advance for those of us who usually have 50-60 tabs open for weeks at a time, where hitting a really heavy image page would tip Firefox over the top and crash it. I switched from Chrome to Firefox a couple of months ago for its ability to handle more tabs without slowing/crashing, and for its better performance with image-heavy pages. So, for me, it's not just "who cares?" it's a killer feature.
  - Touche 13y ago
    Tabs are not bookmarks. Use bookmarks.
    - dbaupp 13y ago
      Why should one use bookmarks rather than many tabs? I have many concurrent projects going, and things I refer to regularly, so I just leave them grouped in my tree of tabs on the sidebar. Much more efficient than digging through nested bookmark menus, or trying to make sure the bookmarks are properly classified. Basically, my tabs are bookmarks, except tabs are integrated into the web-browser better.

dkhenry 13y ago

I have been running aurora for a while so I already have this fix. I just tested it and there was no noticeable performance impact for the memory saved. Chrome on the other hand used all my memory and caused me to start swapping. Any website with lots of images loaded will be impacted by this ( facebook , G+ , tumblr , ... ) so I think its a rather important change

akjj 13y ago

This was a follow-up to an earlier change which also delayed image decompressing. In the description of that change the author wrote: "Andrew McCreight wrote a comment that I can’t find now, but basically he said that on one of his machines, that page was extremely slow and painful to scroll through before the change, and after the change it was smooth and pleasant." If you have any examples where the linked changes causes performance degradation, you should try reporting it to the developers.

smackfu 13y ago
I assume the old way choked very badly once you ran out of real RAM.

NathanKP 13y ago

We've been using stuff like Infinity.js (http://airbnb.github.io/infinity/ http://airbnb.github.io/infinity/) for quite a while now, and it's basic functionality is to improve long scrolling panes by dynamically adding and removing elements (including images) in real time as you scroll to keep the number of elements low and the page responsive. Sites that use this technique usually feel just as responsive as if the site really did have a page with 10,000 items on it instead of a dynamic viewpoint that only shows 100 items as a time. Personally I think it would be best if HTML developed a spec for an official UITableView style native element. That would provide a way to solve the memory issues and avoid the need for JavaScript hacks like Infinity.js But in the meantime browser workarounds like this one from Firefox will help avoid the huge memory usage in sites that don't use smart techniques like Infinity.js

swift 13y ago
I don't know that I'd characterize this as a workaround. Infinity.js is definitely a good approach, but you should expect good baseline performance from the browser even without more complicated techniques.
- NathanKP 13y ago
  I expect good baseline performance from a combination of both the HTML and JavaScript code of the page, and the browser. Sure it's nice if the browser does some tricks to make a badly written page that has 2 GB of image data on it play somewhat more nicely on the system, but if that page loads badly its not the browsers fault, its the fault of the web developer who didn't paginate, or use Infinity.js or something like that. The reason why I call the Firefox fix a workaround is because the proper way to solve the problem of 2 GB of image data on a page is by using a UITableView style wrapper. But HTML 5 doesn't have such an element for us to make use of. The proper fix would be to add such an element to HTML itself, but obviously that's something that takes years to do and to get it adopted across all browsers. So in the meantime tricks like what Firefox is doing are nice to help prevent inexperienced web developers who create abominably inefficient webpages from consuming too much system resources when an unfortunate victim downloads the page.

driverdan 13y ago

The article states that it reduces initial page load time by about 4 seconds. Load time is far more important than a few ms here and there to load images on the fly when they are needed.

joosters 13y ago

Image decompression is pretty easy, CPU-wise. Well over a decade ago, computers were happily rendering JPEGs on-the-fly without storing a decompressed version (RISC OS apps did this). Your desktop/laptop/phone has plenty of ability to do this.

Lagged2Death 13y ago

...it seems ... as if Mozilla went from being totally dismissive of memory complaints to being hyper-responsive to memory complaints with no (publicly stated) rationale behind the change. I don't know what their internal rationale is. "Memory is cheap," we like to say, but it's really only true if you're not wasting it like crazy all the time. Ten years ago you could surf the web with 512MB of RAM; now we're getting to the point where 4GB isn't considered enough. If 4GB now costs the same as 512MB then, then in the sense of "dollars worth of memory required to perform function X acceptably," memory hasn't actually gotten any cheaper. If you can get performance that used to require 4GB down into 1GB, you've upgraded the performance of a bunch of older and low-budget devices from "barely usable" to "fine." Take the big-picture view: updates like this can actually result in a slower flow of "obsolete" computers and devices into the landfills. This is one of those bottlenecks where a little good engineering can have really disproportionate positive aggregate effects. This sort of thinking has been one of the selling points of desktop Linux for a long time. I wish Microsoft thought more like this.

mindcrime 13y ago
Right. Also, no matter how cheap memory is, you run into limits of things like maximum RAM capacity. I'm there now... I have 8GB of RAM in this laptop, and that's all it can hold. I can't spend more money to put more RAM in here even if I want too, as it just won't take it. I'd love to go up to 16GB, but that won't happen until I buy a whole new machine. Needless to say, I'm very happy about this firefox news. AS somebody who routinely runs a lot of apps (and more than a few memory hungry ones) I want every app to be a freaking memory frugal as it possibly can, and be a good citizen when RAM has to be shared between apps. Playing the whole "Unused RAM is wasted, take up as much RAM as you can" mindset is fine if you only run one app at a time, but, really, who the f%!# actually does that???

dragonwriter 13y ago

> All in all, it seems (from a not-particularly-in-touch outsider perspective) as if Mozilla went from being totally dismissive of memory complaints to being hyper-responsive to memory complaints with no (publicly stated) rationale behind the change. I suspect that being a company that wants to have a unified code base for their browser and that is now newly pushing a browser-centric OS for smartphones may explain this, at least in part.

nnethercote 13y ago
I first started working on improving memory consumption in early 2011, and that evolved into the MemShrink project (https://wiki.mozilla.org/Performance/MemShrink https://wiki.mozilla.org/Performance/MemShrink) a couple of months after that. Firefox OS came later. In fact, it's fortunate that MemShrink had been up and running for a while because it meant we had already fixed a lot of leaks and inefficiencies, and we had good tools for identifying memory problems in Firefox OS.

nnethercote 13y ago

Have you tried the new version? You should try it before you dismiss it due to theoretical concerns. As I wrote in the blog post, the time before the page displayed dropped from 5 seconds to 1 seconds. And scrolling performance is unaffected AFAIK; if you have evidence otherwise Timothy and I would love to hear about it.

illumen 13y ago
Firstly... thanks! I love firefox, and appreciate any memory savings. Did you do any tests for animation performance, and test that preloaders are not affected? Are these issues addressed? Hardware accelerated decoding(like IE, and image viewers use) would help. Much modern hardware can do the whole decode-resize step in dedicated hardware. For software only, an improvement is resizing in jpeg-space to the exact size required. It is much faster than decoding the entire jpeg (eg 5 megapixels 2MB jpeg, into 20MB of uncompressed image data) then only displaying it at 320x200 pixels(1MB at the 16bit needed for this YUV data). You can resize the smaller jpeg-space data, then not have to allocate that 20MB of data (perhaps never going back to the OS). As well, some images compress very nicely with hardware texture compression (but many look awful). There are heuristics that can be used to decide the best type of texture to use based on the image data. Why turn 6bit-16bit per pixel jpeg data into 32 bit per pixel data? It only takes more memory and is slower to draw. Surely jpeg usage is common enough to special case its handling? Good luck!