9 ms·

Click bait title. On Windows, if they get an error on memory allocation, they wait a bit and try again. This gives the OS time to resize the swap file which mea

by avian 4y ago

Click bait title. On Windows, if they get an error on memory allocation, they wait a bit and try again. This gives the OS time to resize the swap file which means that the second try often succeeds.

Vinnl 4y ago
...which is a pretty weird trick, at least for someone like me who has no idea how memory allocation works.
pmoriarty 4y ago
Why don't more applications use this "weird trick" of just checking to see if memory allocation succeeded and waiting and retrying if it didn't, instead of unceremoniously crashing? It's not like this hasn't been possible to do for many decades already.
- avian 4y ago
  Even better question is, if it is this simple, why didn't Microsoft implement their malloc or whatever like this in the first case. If the OS needs to resize the page file, why don't they just delay the return of the function until that is done? I'm guessing the answer is something along the lines of "it makes applications freeze for <amount of time> and users don't like that".
  - jraph 4y ago
    Sure, they prefer their applications to just crash :-) Bu yes, the problem is probably not totally simple. adding delay in malloc while application might expect an immediate return could cause bugs. But swapping is a thing anyway and introduces delays so I don't know why they would not do this.
  - deleted 4y ago
    [deleted]
- deleted 4y ago
  [deleted]
- 323 4y ago
  Because something still crashes on the user computer. Also the tragedy of the commons, if everybody does this we are back to square one. The actual solution is for the user to buy more memory, use a different app or change the workflow.
  - pas 4y ago
    Or Firefox to say to the user to buy more RAM, close other apps, etc. Also Windows should step in and say that this application is consuming almos all of the remaining memory, close others, or this or buy more RAM.
    - 323 4y ago
      > Also Windows should step in and say that this application is consuming almos all of the remaining memory This is much more complicated that it appears, because the user could be intentionally doing that - if you have little memory (8 GB), you are sort of always almost out of it, if you have a lot but you are using professional apps like Photoshop, video editing, or doing machine learning with large datasets. I'm sure that the reason Windows doesn't do this is because they couldn't figure out a good way to warn which is also not annoying. There are some Windows warnings, I've seen them, but they only appear when the situation is so dire the computer is about to blue screen.
      - pas 4y ago
        I disabled the swap in Windows, because I have 32G RAM (and why put unnecessary wear on the SSD). And now it runs out of memory while half the RAM is empty. A marvelous operating system. But yeah, at least I saw the warning fast, with the swap file it first came to a sudden halt, and after a lot of time it might have shown the alert.
        astrange 4y ago
        Turning off swap will lead to running out of memory earlier than you’d think because different tasks can allocate a lot instantaneously - you just need a few of them to line up and try allocating a lot at once. I don’t see how you can realistically operate a computer this way unless you control all the software on it, or you enjoy buying way more RAM than you need. (Also swap isn’t the only alternative to allocating memory - there’s also purgeable allocations and memory compression.)
- deleted 4y ago
  [deleted]
- adql 4y ago
  Well, you'd need to wrap every memory allocation with that wrapper.
jraph 4y ago
I think the click bait is assumed. The use of "this one weird trick" is humorous at this point (edit: and I found the article actually interesting/worth reading, the title not really deceptive, which is usually the main problem with click bait. and... yes, it is actually kind of a weird trick?). To expend on your summary, this trick is necessary on Windows (only), because it does not allow over-committing and also does not have an OOM killer which you can instruct to kill content processes instead of the main process.
- bityard 4y ago
  It was humorous five years ago, now it's just cliche
  - SilasX 4y ago
    Remember, this is Mozilla, the same folks who still think, after 20 years, it's the funniest thing in the world to give a cryptic "Zarro boogs found" message to users who are desperately searching the bug tracker to resolve a major software frustration, many of whom aren't native English speakers and won't get the ultra-hip reference. https://bugzilla.mozilla.org/show_bug.cgi?id=1374266 https://bugzilla.mozilla.org/show_bug.cgi?id=1374266
    - jraph 4y ago
      I understand that some people might have found this cryptic, but they added an explanation, so it's not that cryptic anymore. This problem is fixed. It was not even that cryptic. The first time I encountered this, I was a child and not a native English speaker and I still got it. I find it nice that people try to keep a bit of history/fun. Now, its not even the same people that are involved in this issue and this article. So no, its not "the same folks". Anyway, if it's the worst thing you have to complain about Mozilla, we are good I think. People really have weird battles to fight.
      - SilasX 4y ago
        >I understand that some people might have found this cryptic, but they added an explanation, so it's not that cryptic anymore. Right. After about 15 years, they finally relented. So, your position is, "it's not a problem, but they deserve credit for fixing that problem that shouldn't have been fixed"? >Anyway, if it's the worst thing you have to complain about Mozilla, we are good I think. Where are you getting that? When Mozilla did something stupid, I pointed to a related, similar stupid decision, originating from the same cultural practices. From that, you twist my words into meaning that's the only thing there is to criticize? That's the mind of an ideologue, not an honest evaluation for truth. But, if you want to stay in the mentality of automatically trivialzing every unforced error on the part of Mozilla, then, by all means, apply that same reflexive defense to these cases as well! - Looking Glass: Forcing a cryptic extension on users that accustoms them to ignoring changes that look like software compromise.[3] - The fact that, post-2016-update, we still don't have the same add-on functionality as before, including the ability to customize controls. - Not allowing side-loading of unsigned add-ons "because security" even though Chome has long allowed this with no issue. - Then neglecting to keep the signing key up-to-date. [1] - Then making previously-signed add-ons stop working as a result of that, compromising user privacy, possibly causing users in hostile countries to be killed. [1] - Then assuring users that, no, it's okay because we can remotely force updates via an opt-out feature buried in "studies".[2] All of those are worse, so no, the shitty, confusing in-joke at the expense of frustrated bug victims isn't actually the worst part about Mozilla, it's just the most relevant to the OP's comment. But sure, it provides a convenient way for you to imply that nothing else is wrong with Firefox. >People really have weird battles to fight. I'm sorry, what? Wanting a universal, everyday-utility software product to be accessible outside of a small clique ... is a weird battle to fight? That mentality is exactly Mozilla's problem! [1] https://news.ycombinator.com/item?id=19823701 https://news.ycombinator.com/item?id=19823701 [2] https://news.ycombinator.com/item?id=19826827 https://news.ycombinator.com/item?id=19826827 [3] https://news.ycombinator.com/item?id=15931730 https://news.ycombinator.com/item?id=15931730
kuroguro 4y ago
I don't really mind a clickbaity joke as long as the contents are good. Found the hack itself pretty cool.
deleted 4y ago
[deleted]
gsvelto 4y ago
But is it? I wanted it to be tongue-in-cheek but the title is actually a rather accurate description of the contents of the article. This is a ~20 lines weird hack that we threw at the wall to see if it would stick... and it massively improved Firefox stability. I can't stress it hard enough, it's the largest, most visible stability improvement in the past decade. More specifically - on the memory front - we spent lots of engineer/years working on improvements which sometimes would barely register, only for some weird trick make OOM crashes fall off the radar entirely. Also I'd like to point out that we have no way to tell if this is working because we give Windows time to resize the swap file, if it's because other processes in Firefox die, or it's because other unrelated processes die, or a mix of all the above. It's pure speculation on our part.

by avian 4y ago

Click bait title. On Windows, if they get an error on memory allocation, they wait a bit and try again. This gives the OS time to resize the swap file which means that the second try often succeeds.

Vinnl 4y ago

...which is a pretty weird trick, at least for someone like me who has no idea how memory allocation works.

pmoriarty 4y ago

Why don't more applications use this "weird trick" of just checking to see if memory allocation succeeded and waiting and retrying if it didn't, instead of unceremoniously crashing? It's not like this hasn't been possible to do for many decades already.

avian 4y ago
Even better question is, if it is this simple, why didn't Microsoft implement their malloc or whatever like this in the first case. If the OS needs to resize the page file, why don't they just delay the return of the function until that is done? I'm guessing the answer is something along the lines of "it makes applications freeze for <amount of time> and users don't like that".
- jraph 4y ago
  Sure, they prefer their applications to just crash :-) Bu yes, the problem is probably not totally simple. adding delay in malloc while application might expect an immediate return could cause bugs. But swapping is a thing anyway and introduces delays so I don't know why they would not do this.
- deleted 4y ago
  [deleted]
deleted 4y ago
[deleted]
323 4y ago
Because something still crashes on the user computer. Also the tragedy of the commons, if everybody does this we are back to square one. The actual solution is for the user to buy more memory, use a different app or change the workflow.
- pas 4y ago
  Or Firefox to say to the user to buy more RAM, close other apps, etc. Also Windows should step in and say that this application is consuming almos all of the remaining memory, close others, or this or buy more RAM.
  - 323 4y ago
    > Also Windows should step in and say that this application is consuming almos all of the remaining memory This is much more complicated that it appears, because the user could be intentionally doing that - if you have little memory (8 GB), you are sort of always almost out of it, if you have a lot but you are using professional apps like Photoshop, video editing, or doing machine learning with large datasets. I'm sure that the reason Windows doesn't do this is because they couldn't figure out a good way to warn which is also not annoying. There are some Windows warnings, I've seen them, but they only appear when the situation is so dire the computer is about to blue screen.
    - pas 4y ago
      I disabled the swap in Windows, because I have 32G RAM (and why put unnecessary wear on the SSD). And now it runs out of memory while half the RAM is empty. A marvelous operating system. But yeah, at least I saw the warning fast, with the swap file it first came to a sudden halt, and after a lot of time it might have shown the alert.
      - astrange 4y ago
        Turning off swap will lead to running out of memory earlier than you’d think because different tasks can allocate a lot instantaneously - you just need a few of them to line up and try allocating a lot at once. I don’t see how you can realistically operate a computer this way unless you control all the software on it, or you enjoy buying way more RAM than you need. (Also swap isn’t the only alternative to allocating memory - there’s also purgeable allocations and memory compression.)
deleted 4y ago
[deleted]
adql 4y ago
Well, you'd need to wrap every memory allocation with that wrapper.

jraph 4y ago

I think the click bait is assumed. The use of "this one weird trick" is humorous at this point (edit: and I found the article actually interesting/worth reading, the title not really deceptive, which is usually the main problem with click bait. and... yes, it is actually kind of a weird trick?). To expend on your summary, this trick is necessary on Windows (only), because it does not allow over-committing and also does not have an OOM killer which you can instruct to kill content processes instead of the main process.

bityard 4y ago
It was humorous five years ago, now it's just cliche
- SilasX 4y ago
  Remember, this is Mozilla, the same folks who still think, after 20 years, it's the funniest thing in the world to give a cryptic "Zarro boogs found" message to users who are desperately searching the bug tracker to resolve a major software frustration, many of whom aren't native English speakers and won't get the ultra-hip reference. https://bugzilla.mozilla.org/show_bug.cgi?id=1374266 https://bugzilla.mozilla.org/show_bug.cgi?id=1374266
  - jraph 4y ago
    I understand that some people might have found this cryptic, but they added an explanation, so it's not that cryptic anymore. This problem is fixed. It was not even that cryptic. The first time I encountered this, I was a child and not a native English speaker and I still got it. I find it nice that people try to keep a bit of history/fun. Now, its not even the same people that are involved in this issue and this article. So no, its not "the same folks". Anyway, if it's the worst thing you have to complain about Mozilla, we are good I think. People really have weird battles to fight.
    - SilasX 4y ago
      >I understand that some people might have found this cryptic, but they added an explanation, so it's not that cryptic anymore. Right. After about 15 years, they finally relented. So, your position is, "it's not a problem, but they deserve credit for fixing that problem that shouldn't have been fixed"? >Anyway, if it's the worst thing you have to complain about Mozilla, we are good I think. Where are you getting that? When Mozilla did something stupid, I pointed to a related, similar stupid decision, originating from the same cultural practices. From that, you twist my words into meaning that's the only thing there is to criticize? That's the mind of an ideologue, not an honest evaluation for truth. But, if you want to stay in the mentality of automatically trivialzing every unforced error on the part of Mozilla, then, by all means, apply that same reflexive defense to these cases as well! - Looking Glass: Forcing a cryptic extension on users that accustoms them to ignoring changes that look like software compromise.[3] - The fact that, post-2016-update, we still don't have the same add-on functionality as before, including the ability to customize controls. - Not allowing side-loading of unsigned add-ons "because security" even though Chome has long allowed this with no issue. - Then neglecting to keep the signing key up-to-date. [1] - Then making previously-signed add-ons stop working as a result of that, compromising user privacy, possibly causing users in hostile countries to be killed. [1] - Then assuring users that, no, it's okay because we can remotely force updates via an opt-out feature buried in "studies".[2] All of those are worse, so no, the shitty, confusing in-joke at the expense of frustrated bug victims isn't actually the worst part about Mozilla, it's just the most relevant to the OP's comment. But sure, it provides a convenient way for you to imply that nothing else is wrong with Firefox. >People really have weird battles to fight. I'm sorry, what? Wanting a universal, everyday-utility software product to be accessible outside of a small clique ... is a weird battle to fight? That mentality is exactly Mozilla's problem! [1] https://news.ycombinator.com/item?id=19823701 https://news.ycombinator.com/item?id=19823701 [2] https://news.ycombinator.com/item?id=19826827 https://news.ycombinator.com/item?id=19826827 [3] https://news.ycombinator.com/item?id=15931730 https://news.ycombinator.com/item?id=15931730

kuroguro 4y ago

I don't really mind a clickbaity joke as long as the contents are good. Found the hack itself pretty cool.

deleted 4y ago

[deleted]

gsvelto 4y ago

But is it? I wanted it to be tongue-in-cheek but the title is actually a rather accurate description of the contents of the article. This is a ~20 lines weird hack that we threw at the wall to see if it would stick... and it massively improved Firefox stability. I can't stress it hard enough, it's the largest, most visible stability improvement in the past decade. More specifically - on the memory front - we spent lots of engineer/years working on improvements which sometimes would barely register, only for some weird trick make OOM crashes fall off the radar entirely. Also I'd like to point out that we have no way to tell if this is working because we give Windows time to resize the swap file, if it's because other processes in Firefox die, or it's because other unrelated processes die, or a mix of all the above. It's pure speculation on our part.