7 ms·

The sad state of affairs is that papercuts are the death of us all. We've come to the point where we have "unlimited" resources that we just throw away memory l

The sad state of affairs is that papercuts are the death of us all. We've come to the point where we have "unlimited" resources that we just throw away memory like it's going out of style. I remember a profiling of a gnome startup. It touched something like insane like 40K files. Another profile I saw removed a few bytes (unneeded, it wasn't some complex optimization) from some commonly used datas-tructure for an 50%+ speed improvement. Simply we're being cut to death by bloat and pure, sheer lazyness. a web browser has absolutely no excuse for using 500megs of ram. None. Whatsoever. I expect this comment to fall on deaf ears.

mcpherrinm 15y ago
What if I have 10 tabs open, each with 5 images that are 10mb each? It's not an unreasonable workload. Yet there's 500mb, right there. Never? Whatsoever? I don't think so.
- spicyj 15y ago
  I seriously doubt you're opening many pages with 10 MB images; even 1 MB is considered pretty large for a web image.
  - mcpherrinm 15y ago
    Right: the example is contrived. My point is mainly that it is actually within reach, and that workload needs to be taken into consideration when making any statement about how much memory a browser should be using.
  - monkeypizza 15y ago
    It happens to me all the time - the "linked images" bookmarklet combined with EarthPorn reddit + autopager.
  - tsvk 15y ago
    Did you consider compressed JPG/PNG on disk versus uncompressed raw image data bitmap in RAM? A 1024x768 32-bit image is 1024x768x4 = 3.145.728 bytes when uncompressed.
  - masklinn 15y ago
    > even 1 MB is considered pretty large for a web image. Compressed. Remember that in order to display it, the browser has to unpack it to what amounts to a bitmap. And newer, better compression formats (e.g. PNG) make that even worse in compressing the same bitmap file to a much smaller transmission size.
ootachi 15y ago
"Simply we're being cut to death by bloat and pure, sheer lazyness." All these memory leaks being fixed are "pure, sheer lazyness"? https://bugzilla.mozilla.org/buglist.cgi?keywords=mlk%2C%20&keywords_type=allwords&resolution=FIXED&query_format=advanced&bug_status=RESOLVED https://bugzilla.mozilla.org/buglist.cgi?keywords=mlk%2C%20&...
- ak217 15y ago
  That's not what spitfire meant. He meant that the people who put the features responsible for these extra fields/leaks/fragmentation/etc. were not thoroughly performance tested or thought out with performance in mind, out of laziness.
- deleted 15y ago
  [deleted]
nnethercote 15y ago
The key word here is "workload". A web browser is a programming environment. It would be silly to say "no single native program should use 500MB of RAM", and it's equally silly to say that no website (and by extension, the browser) should use that much. That's not to say Firefox's memory usage is perfect, far from it. That's why we're making improvements like this one. See https://wiki.mozilla.org/Performance/MemShrink https://wiki.mozilla.org/Performance/MemShrink for more.
- saulrh 15y ago
  It's not just a programming environment, either; a web browser is an interpreter for some of the most poorly-written, highest-level code in existence. Not very conducive to good memory usage even theoretically.
  - mixmastamyk 15y ago
    Perhaps, but wouldn't you agree it should get freed when you close the tabs?
    - nnethercote 15y ago
      Yes, although it might not happen immediately. That's exactly what this fix helps with. And see https://bugzilla.mozilla.org/show_bug.cgi?id=668809 https://bugzilla.mozilla.org/show_bug.cgi?id=668809 for more about this.
    - saulrh 15y ago
      I believe that there are also some information-theoretical limits on the efficiency of garbage collection and memory allocation. Don't quote me on that, though.
    - ahpeeyem 15y ago
      But then the Ctrl+Shift+T feature to reopen closed tabs wouldn't work so well. Try it, you can reopen closed tabs as far back as you like; it's pretty awesome and has saved me more than once. It doesn't work in Private Mode (for good reason); I wonder if browsing in private mode consumes less memory?
gorbachev 15y ago
I agree on your principle, but not with the 500MB comment. You don't know how people use their browsers. For example, I have 8 tab groups open right now, each with 5 - 20 tabs. Let's say that's about 75 tabs. That's only 6.67MB each to fill 500MB. I'm regularly over a 1GB. I'm sure a significant part of that is actually memory leaks, but that's beside the point.
- michael_dorfman 15y ago
  Wait, you have 75 tabs open? Doesn't the time it takes to locate the tab you are looking for exceed the time it would take to launch a new tab, and load the desired page?
  - jpr 15y ago
    I presume that most people who have lots of tabs open, like I do, use addons that make finding tabs fast. I use Tab Kit to organize the tabs in trees, and Pentadactyl to give me fast search that is activated by pressing b. I can't imagine using a browser which doesn't have either or both of these features.
  - Argorak 15y ago
    The firefox awesome bar has an awesome (haha) feature for that: if you type an address, it will detect wether you have matching tabs open and give you the option to switch instead. It works over multiple tab groups and windows. So: locating a tab is quicker and it keeps the state of the page.
- gnosis 15y ago
  There's really no reason why browsers couldn't have an option to make tabs which aren't currently active (ie. being viewed) be treated as bookmarks, which take up virtually no memory, since all they are are a URL and a bit of metadata. This would, of course, slow down browsing when switching from tab to tab, since the entire page would need to be reloaded. But I would gladly give up some speed for memory on my old machine with only 3.7 gigs of RAM. Before anyone asks why don't I just use bookmarks in the first place, let me say that I prefer the organization that tabs give me for pages I am currently working on. I do move those tabs to bookmarks if I use them long enough. But in the relatively short term I prefer tabs. Another option that would be a compromise between the solution proposed above and keeping all tabs in memory would be to keep only the N most recently accessed tabs in memory, and treat the rest as bookmarks. That would make switching between those N tabs relatively responsive, but still use a lot less memory than having every page for every tab reside in memory. Especially for people like me, who regularly have over 100 tabs open. Of course, the value of N should be user-configurable. With a N=1, the second proposed solution would be equivalent to the first.
wheels 15y ago
Your comment is wrong on so many levels: • It's not even on topic. This post is not about general memory usage; it's about fragmentation in the javascript VM – i.e. that a system object existing in a general memory page precludes that page from being garbage collected / deallocated even though the lifetime of system objects is very different. This is not a trivial matter for which the only explanation is laziness. • It's spoken like someone who's never seriously worked on a large open source project. Memory profiling and leak tracking is done widely and often (using valgrind, mostly). • You speak as if you know how much memory a web browser should use. How do you know? Have you worked on modern web browsers? Or are you just, as I assume, by fiat deciding that 500 MB is too much? • There's very often a speed / memory usage tradeoff. At present, especially for web browsers, users tend to prefer speed. (i.e. how many rendered pages do you cache per tab so that clicking the back button is next to instant?) • The right time to profile and optimize is usually later in the development process. We all know the famous quote. Optimizing early tends to lead to ugly code that, amusingly, is harder to optimize later.
- z92 15y ago
  I remember sometime back probably around 2004. There was a lot of noise regarding memory leak in FF. And the expert sounding ones started to claim that there was no memory leak and the very large memory usage in FF was because of browser history and cache storage. Then one year later they all accepted that FF was actually leaking memory. Since then I stopped accepting expert opinion blindly.
  - wheels 15y ago
    Experts are sometimes wrong. That doesn't change the fact that they're right vastly more often than non-experts.
  - clobber 15y ago
    I could be wrong, but wasn't a large part of this due to Flash?
    - starwed 15y ago
demallien 15y ago
It's not laziness, it's all about avoiding premature optimisations. Personally I write the bulk of my code on embedded systems where we are severly constrained both in sheer CPU performance, but also in the amount of memory available. Nevertheless, I still start off writing code in what you might call a "lazy" style, but which I tend to think of as "the simplest, clearest way to write code to do the job". I assume I'll have the memory and the performance that I need. Having written the code, if that assumption turns out to be false, only then do I go back and try to improve performance characteristics - adding caching, optimising algorithms to avoid doing unnecessary calculations, and so on. The code is inevitably longer, and less flexible, after this work which means that I have probably added in bugs during the process. And that is why I don't start off doing things that way, I'm trying to keep the LOC count (and hence the bug count) down as much as possible. Memory and performance optimisations are done only when the is a demonstrated need for them, and I feel that this is the correct way to approach optimisation.
- JoeAltmaier 15y ago
  Yeah; sounds like a good process. But all too often the "try to improve performance characteristics" part is entirely skipped. It doesn't take much to run A/B tests and wonder why you are 5X over other similar apps. It was clearly never done, at least not by anybody in a position to help mitigate the issue. I use a similar version of your processes. adding: when code complete examine the footprint and speed, and diligently explain to myself where the time/memory went. If it was wasted, do something about it immediatley.
techdmn 15y ago
Obligatory Abrash quote: "In the long run, programs will be bigger and slower yet, but computers will be so fast and will have so much memory that no one will care." [1] I don't agree 100%, but he has a point. I'll say that resources are cheap enough they aren't always worth optimizing. On the other hand, it's easy to get caught on the wrong side of that. Always painful to discover that someone else's "plenty to spare" is your "just a bit short". [1] http://www.phatcode.net/res/224/files/html/ch39/39-01.html http://www.phatcode.net/res/224/files/html/ch39/39-01.html

mcpherrinm 15y ago

What if I have 10 tabs open, each with 5 images that are 10mb each? It's not an unreasonable workload. Yet there's 500mb, right there. Never? Whatsoever? I don't think so.

spicyj 15y ago
I seriously doubt you're opening many pages with 10 MB images; even 1 MB is considered pretty large for a web image.
- mcpherrinm 15y ago
  Right: the example is contrived. My point is mainly that it is actually within reach, and that workload needs to be taken into consideration when making any statement about how much memory a browser should be using.
- monkeypizza 15y ago
  It happens to me all the time - the "linked images" bookmarklet combined with EarthPorn reddit + autopager.
- tsvk 15y ago
  Did you consider compressed JPG/PNG on disk versus uncompressed raw image data bitmap in RAM? A 1024x768 32-bit image is 1024x768x4 = 3.145.728 bytes when uncompressed.
- masklinn 15y ago
  > even 1 MB is considered pretty large for a web image. Compressed. Remember that in order to display it, the browser has to unpack it to what amounts to a bitmap. And newer, better compression formats (e.g. PNG) make that even worse in compressing the same bitmap file to a much smaller transmission size.

ootachi 15y ago

"Simply we're being cut to death by bloat and pure, sheer lazyness." All these memory leaks being fixed are "pure, sheer lazyness"? https://bugzilla.mozilla.org/buglist.cgi?keywords=mlk%2C%20&keywords_type=allwords&resolution=FIXED&query_format=advanced&bug_status=RESOLVED https://bugzilla.mozilla.org/buglist.cgi?keywords=mlk%2C%20&...

ak217 15y ago
That's not what spitfire meant. He meant that the people who put the features responsible for these extra fields/leaks/fragmentation/etc. were not thoroughly performance tested or thought out with performance in mind, out of laziness.
deleted 15y ago
[deleted]

nnethercote 15y ago

The key word here is "workload". A web browser is a programming environment. It would be silly to say "no single native program should use 500MB of RAM", and it's equally silly to say that no website (and by extension, the browser) should use that much. That's not to say Firefox's memory usage is perfect, far from it. That's why we're making improvements like this one. See https://wiki.mozilla.org/Performance/MemShrink https://wiki.mozilla.org/Performance/MemShrink for more.

saulrh 15y ago
It's not just a programming environment, either; a web browser is an interpreter for some of the most poorly-written, highest-level code in existence. Not very conducive to good memory usage even theoretically.
- mixmastamyk 15y ago
  Perhaps, but wouldn't you agree it should get freed when you close the tabs?
  - nnethercote 15y ago
    Yes, although it might not happen immediately. That's exactly what this fix helps with. And see https://bugzilla.mozilla.org/show_bug.cgi?id=668809 https://bugzilla.mozilla.org/show_bug.cgi?id=668809 for more about this.
  - saulrh 15y ago
    I believe that there are also some information-theoretical limits on the efficiency of garbage collection and memory allocation. Don't quote me on that, though.
  - ahpeeyem 15y ago
    But then the Ctrl+Shift+T feature to reopen closed tabs wouldn't work so well. Try it, you can reopen closed tabs as far back as you like; it's pretty awesome and has saved me more than once. It doesn't work in Private Mode (for good reason); I wonder if browsing in private mode consumes less memory?

gorbachev 15y ago

I agree on your principle, but not with the 500MB comment. You don't know how people use their browsers. For example, I have 8 tab groups open right now, each with 5 - 20 tabs. Let's say that's about 75 tabs. That's only 6.67MB each to fill 500MB. I'm regularly over a 1GB. I'm sure a significant part of that is actually memory leaks, but that's beside the point.

michael_dorfman 15y ago
Wait, you have 75 tabs open? Doesn't the time it takes to locate the tab you are looking for exceed the time it would take to launch a new tab, and load the desired page?
- jpr 15y ago
  I presume that most people who have lots of tabs open, like I do, use addons that make finding tabs fast. I use Tab Kit to organize the tabs in trees, and Pentadactyl to give me fast search that is activated by pressing b. I can't imagine using a browser which doesn't have either or both of these features.
- Argorak 15y ago
  The firefox awesome bar has an awesome (haha) feature for that: if you type an address, it will detect wether you have matching tabs open and give you the option to switch instead. It works over multiple tab groups and windows. So: locating a tab is quicker and it keeps the state of the page.
gnosis 15y ago
There's really no reason why browsers couldn't have an option to make tabs which aren't currently active (ie. being viewed) be treated as bookmarks, which take up virtually no memory, since all they are are a URL and a bit of metadata. This would, of course, slow down browsing when switching from tab to tab, since the entire page would need to be reloaded. But I would gladly give up some speed for memory on my old machine with only 3.7 gigs of RAM. Before anyone asks why don't I just use bookmarks in the first place, let me say that I prefer the organization that tabs give me for pages I am currently working on. I do move those tabs to bookmarks if I use them long enough. But in the relatively short term I prefer tabs. Another option that would be a compromise between the solution proposed above and keeping all tabs in memory would be to keep only the N most recently accessed tabs in memory, and treat the rest as bookmarks. That would make switching between those N tabs relatively responsive, but still use a lot less memory than having every page for every tab reside in memory. Especially for people like me, who regularly have over 100 tabs open. Of course, the value of N should be user-configurable. With a N=1, the second proposed solution would be equivalent to the first.

wheels 15y ago

Your comment is wrong on so many levels: • It's not even on topic. This post is not about general memory usage; it's about fragmentation in the javascript VM – i.e. that a system object existing in a general memory page precludes that page from being garbage collected / deallocated even though the lifetime of system objects is very different. This is not a trivial matter for which the only explanation is laziness. • It's spoken like someone who's never seriously worked on a large open source project. Memory profiling and leak tracking is done widely and often (using valgrind, mostly). • You speak as if you know how much memory a web browser should use. How do you know? Have you worked on modern web browsers? Or are you just, as I assume, by fiat deciding that 500 MB is too much? • There's very often a speed / memory usage tradeoff. At present, especially for web browsers, users tend to prefer speed. (i.e. how many rendered pages do you cache per tab so that clicking the back button is next to instant?) • The right time to profile and optimize is usually later in the development process. We all know the famous quote. Optimizing early tends to lead to ugly code that, amusingly, is harder to optimize later.

z92 15y ago
I remember sometime back probably around 2004. There was a lot of noise regarding memory leak in FF. And the expert sounding ones started to claim that there was no memory leak and the very large memory usage in FF was because of browser history and cache storage. Then one year later they all accepted that FF was actually leaking memory. Since then I stopped accepting expert opinion blindly.
- wheels 15y ago
  Experts are sometimes wrong. That doesn't change the fact that they're right vastly more often than non-experts.
- clobber 15y ago
  I could be wrong, but wasn't a large part of this due to Flash?
  - starwed 15y ago

demallien 15y ago

It's not laziness, it's all about avoiding premature optimisations. Personally I write the bulk of my code on embedded systems where we are severly constrained both in sheer CPU performance, but also in the amount of memory available. Nevertheless, I still start off writing code in what you might call a "lazy" style, but which I tend to think of as "the simplest, clearest way to write code to do the job". I assume I'll have the memory and the performance that I need. Having written the code, if that assumption turns out to be false, only then do I go back and try to improve performance characteristics - adding caching, optimising algorithms to avoid doing unnecessary calculations, and so on. The code is inevitably longer, and less flexible, after this work which means that I have probably added in bugs during the process. And that is why I don't start off doing things that way, I'm trying to keep the LOC count (and hence the bug count) down as much as possible. Memory and performance optimisations are done only when the is a demonstrated need for them, and I feel that this is the correct way to approach optimisation.

JoeAltmaier 15y ago
Yeah; sounds like a good process. But all too often the "try to improve performance characteristics" part is entirely skipped. It doesn't take much to run A/B tests and wonder why you are 5X over other similar apps. It was clearly never done, at least not by anybody in a position to help mitigate the issue. I use a similar version of your processes. adding: when code complete examine the footprint and speed, and diligently explain to myself where the time/memory went. If it was wasted, do something about it immediatley.

techdmn 15y ago

Obligatory Abrash quote: "In the long run, programs will be bigger and slower yet, but computers will be so fast and will have so much memory that no one will care." [1] I don't agree 100%, but he has a point. I'll say that resources are cheap enough they aren't always worth optimizing. On the other hand, it's easy to get caught on the wrong side of that. Always painful to discover that someone else's "plenty to spare" is your "just a bit short". [1] http://www.phatcode.net/res/224/files/html/ch39/39-01.html http://www.phatcode.net/res/224/files/html/ch39/39-01.html