5 ms·

Minimize Heap Allocations in Node.js

57 points by unripe_syntax 4y ago

chrisseaton 4y ago
> To allocate memory in the heap, a program must first scan the heap to find a contiguous memory block large enough to hold the data. Does V8 not have a linear allocation buffer? I’m sure it’s a bump allocation, not a scan for free space.
- cogman10 4y ago
  I skimmed this [1] but I believe you are correct. It's a generational moving GC which means bump allocation typically. [1] https://v8.dev/blog/trash-talk https://v8.dev/blog/trash-talk
  - rockwotj 4y ago
    Looking at the code it seems to use a linear allocation buffer: https://github.com/oilpan-gc/cppgc/blob/main/src/heap/cppgc/object-allocator.h https://github.com/oilpan-gc/cppgc/blob/main/src/heap/cppgc/...
    - Leszek 4y ago
      You're looking at Oilpan, which is the C++ GC for Blink, not the one for V8. But, the V8 one is very similar in many respects (the same team works on both), including linear allocation buffers.
      - rockwotj 4y ago
        They are one and the same I as I understand it now: https://v8.dev/blog/oilpan-library https://v8.dev/blog/oilpan-library https://github.com/v8/v8/blob/main/include/cppgc/README.md https://github.com/v8/v8/blob/main/include/cppgc/README.md Quote for the frist article: > We launched the library [Oilpan] in V8 v9.4 and enabled it in Blink starting in Chromium M94.
        Leszek 4y ago
        It's a bit confusing, I agree -- Oilpan is part of the V8 sources for shipping-the-org-chart reasons, but it isn't actually the GC used by V8 itself for JavaScript objects and the JS heap, it's the GC used by Blink for DOM objects etc. The biggest difference between the two is that JS objects are fully managed by the GC and V8 has full control over their layout and access (e.g. they are accessed via explicit pointer + field offset in the runtime), while Oilpan works on C++ objects; there's a billion consequences of this, in particular that JS objects are movable (because we can safely update pointers) and Oilpan objects aren't (because C++ can do pretty much whatever it wants).
        rockwotj 4y ago
        Thanks for clarifying!
        cogman10 4y ago
        At one point, it was supposed to be the universal GC for chrome. In fact, that was cited as the major holdup for getting the DartVM into chrome, they wanted a unified GC between dart, v8, and the DOM.
IshKebab 4y ago
> In JavaScript, primitives are assigned/passed by value and not by reference. If you assign a primitive type's value to another variable, the value is copied - not assigned a reference to the first variable (which is what happens for objects). Bit off topic but is there actually any observable difference between primitives being value based vs being references to constants? Here's the example they give: let num1 = 5; let num2 = num1; num1 = 6; console.log({ num1, num2 }); // { num1: 6, num2: 5 } But it's exactly the same for objects (e.g. arrays in this case): let num1 = [5]; let num2 = num1; num1 = [6]; console.log({ num1, num2 }); // { num1: [6], num2: [5] } Definitely easier to think about them being value based and of course they are implemented like that but I don't think it's the only possibly interpretation.
- LunaSea 4y ago
  You have to modify num1 to see the reference change. e.g: `num1[0] = 42` Your code is currently only reassigning a new value to the num1 variable.
  - IshKebab 4y ago
    No because I was talking about constants so you can't modify them. (Obviously you can literally modify them in the code I posted but it was just an illustration.)
- kevincox 4y ago
  There is no difference between pass-by-reference and pass-by-value for immutable objects (unless there is a way to check object identity). So if you call the JavaScript primitive types Number, String, undefined, boolean, reference then all of them are immutable so there is no difference. Of course most people don't think of JS as passing a "reference primitive by value", they think about "passing an object by reference". Since all of the primitives are immutable it is probably then easier to think of JS as a pass-by-reference language from the semantic point of view. In fact if you really want to simplify you can imagine that every variable is just a reference to a value since it is equivalent.
Leszek 4y ago
V8 dev here. Yes, please avoid leaking memory -- but the intro to this article is incorrect in almost all major points: > In JavaScript, primitives are assigned/passed by value and not by reference. If you assign a primitive type's value to another variable, the value is copied. No, primitives are immutable but still passed by reference (that reference is passed by value, which is why assigning to the reference won't change them -- the same is true for objects). Can you imagine if we'd copied strings every time they were passed into a function? > This also determines how they are stored in memory — primitive values are stored on the stack while objects are stored on the heap, with a reference to their location stored on the stack. Also no, all primitives except small integers ("Smis") are stored in the heap. > One important thing to note about variables stored in the stack is that their sizes must be known at compile time. This allows a program to allocate the right amount of memory in the stack. Because the JavaScript engine allocates a fixed amount of memory for primitive values, there is a limit to how large they can be. Again, no, with strings as the obvious counter example. There's a limit to how large values in general can be, but it's around indexable size and pointer width, not a fixed amount of memory for primitives. > Memory stored in the heap will live on unless it is explicitly deleted or deallocated. Is the author confusing the malloc heap with the (garbage collected) JS heap? Sure, V8 has to manage the pages it allocates, but JS authors certainly don't > To allocate memory in the heap, a program must first scan the heap to find a contiguous memory block large enough to hold the data. As a sibling comment points out, there are optimisations here with linear allocation buffers that permit bump allocation. Also, don't forget the semi space for young objects, since the GC is generational.
- kevincox 4y ago
  > No, primitives are immutable but still passed by reference For immutable values the pass-by-reference vs pass-by-value distinction isn't really important. Unless the language provides some way to get the identity of the object there isn't a way to tell a difference. > Can you imagine if we'd copied strings every time they were passed into a function? IIUC most Javascript interpreters optimize copying strings anyways, so this wouldn't be a big deal. For example even though `s.slice(1)` is a "copy" most engines (I believe V8 included) will use the same backing string for both. So the difference would be a pointer on the stack vs a pointer+length on the stack and copying an extra word is not a huge difference. But maybe this is just being nitpicky, since the article is focused on performance the lower level details do mater not the semantics. However for strings I don't think the difference between their model and reality is very significant.
aliceryhl 4y ago
> If you assign a primitive type's value to another variable, the value is copied - not assigned a reference to the first variable (which is what happens for objects). Assigning an object also just copies the value - i.e. the reference to the heap value. So the new variable does not end up pointing to the first variable. Rather, both variables end up pointing to the same heap value.
graderjs 4y ago
I like this, it looks like a good reference on memory management...but I see Leszek saying, "no". OK, maybe the authors don't know the theory, but they learned some useful tricks to reduce. Will still read. Related, when I need lots of memory in node, I use: `--max-old-space-size=` programmatically set to some % of total available memory on the system.
- Leszek 4y ago
  To be clear, maybe I was a bit negative, but I was just talking about the technical intro. The rest of the article on not accidentally leaking references through globals and timers, and what tools to use to catch them, is pretty solid.
kaladin_1 4y ago
The errors have already been pointed out by multiple comments. Opening the article, it is immediately obvious that the author sourced his material from random Js blogs. He understands very little about the v8 engine and just fills the gap with those material which are obviously wrong. On the hand, he can be happy that his article made it here where he would be corrected for free. Hope the author hangs out here...
deleted 4y ago
[deleted]
sod 4y ago
If you want to reduce memory consumption by your node.js server, minify it (e.g. via esbuild in node.js mode) and don't use inline source maps (if you want to keep source maps, use .map files). node.js keeps all source code as string in memory. So unminified code can bloat your runtime by a lot if you require a couple of external libraries. If you bundle the libraries as well, you don't even need to copy node_modules anymore, which can also save time in deploying a change. This small change can cut your memory requirement in half. Of course I wish node.js just had an option to free the memory of the source code garbage, but as of version 18.9.1 there is none.
losvedir 4y ago
Am I missing something, or does the article not go into its title of minimizing heap allocations at all? It talks about not having memory leaks, but no strategies for minimizing the use of the heap in the first place. As I understand it, this is an important technique to eke out the most performance. I know I've seen an article on minimizing allocations on Go, and doesn't Java have "value types" for that sort of thing? I would be interested in the JS equivalent, and am disappointed the article didn't actually get into it. It's imagine it's stuff like using byte arrays and such?
olliej 4y ago
This is not a good article, it confuses immutable with fixed size, claims primitives are stored on the stack but includes strings in that set, and then claims that V8 used to have a problem with cycles because it used refcounting???? Also I found the writing itself to be long winded and hard to read even without the many technical errors.

57 points by unripe_syntax 4y ago

> To allocate memory in the heap, a program must first scan the heap to find a contiguous memory block large enough to hold the data. Does V8 not have a linear allocation buffer? I’m sure it’s a bump allocation, not a scan for free space.

cogman10 4y ago
I skimmed this [1] but I believe you are correct. It's a generational moving GC which means bump allocation typically. [1] https://v8.dev/blog/trash-talk https://v8.dev/blog/trash-talk
- rockwotj 4y ago
  Looking at the code it seems to use a linear allocation buffer: https://github.com/oilpan-gc/cppgc/blob/main/src/heap/cppgc/object-allocator.h https://github.com/oilpan-gc/cppgc/blob/main/src/heap/cppgc/...
  - Leszek 4y ago
    You're looking at Oilpan, which is the C++ GC for Blink, not the one for V8. But, the V8 one is very similar in many respects (the same team works on both), including linear allocation buffers.
    - rockwotj 4y ago
      They are one and the same I as I understand it now: https://v8.dev/blog/oilpan-library https://v8.dev/blog/oilpan-library https://github.com/v8/v8/blob/main/include/cppgc/README.md https://github.com/v8/v8/blob/main/include/cppgc/README.md Quote for the frist article: > We launched the library [Oilpan] in V8 v9.4 and enabled it in Blink starting in Chromium M94.
      - Leszek 4y ago
        It's a bit confusing, I agree -- Oilpan is part of the V8 sources for shipping-the-org-chart reasons, but it isn't actually the GC used by V8 itself for JavaScript objects and the JS heap, it's the GC used by Blink for DOM objects etc. The biggest difference between the two is that JS objects are fully managed by the GC and V8 has full control over their layout and access (e.g. they are accessed via explicit pointer + field offset in the runtime), while Oilpan works on C++ objects; there's a billion consequences of this, in particular that JS objects are movable (because we can safely update pointers) and Oilpan objects aren't (because C++ can do pretty much whatever it wants).
        rockwotj 4y ago
        Thanks for clarifying!
        cogman10 4y ago
        At one point, it was supposed to be the universal GC for chrome. In fact, that was cited as the major holdup for getting the DartVM into chrome, they wanted a unified GC between dart, v8, and the DOM.

IshKebab 4y ago

> In JavaScript, primitives are assigned/passed by value and not by reference. If you assign a primitive type's value to another variable, the value is copied - not assigned a reference to the first variable (which is what happens for objects). Bit off topic but is there actually any observable difference between primitives being value based vs being references to constants? Here's the example they give: let num1 = 5; let num2 = num1; num1 = 6; console.log({ num1, num2 }); // { num1: 6, num2: 5 } But it's exactly the same for objects (e.g. arrays in this case): let num1 = [5]; let num2 = num1; num1 = [6]; console.log({ num1, num2 }); // { num1: [6], num2: [5] } Definitely easier to think about them being value based and of course they are implemented like that but I don't think it's the only possibly interpretation.

LunaSea 4y ago
You have to modify num1 to see the reference change. e.g: `num1[0] = 42` Your code is currently only reassigning a new value to the num1 variable.
- IshKebab 4y ago
  No because I was talking about constants so you can't modify them. (Obviously you can literally modify them in the code I posted but it was just an illustration.)
kevincox 4y ago
There is no difference between pass-by-reference and pass-by-value for immutable objects (unless there is a way to check object identity). So if you call the JavaScript primitive types Number, String, undefined, boolean, reference then all of them are immutable so there is no difference. Of course most people don't think of JS as passing a "reference primitive by value", they think about "passing an object by reference". Since all of the primitives are immutable it is probably then easier to think of JS as a pass-by-reference language from the semantic point of view. In fact if you really want to simplify you can imagine that every variable is just a reference to a value since it is equivalent.

Leszek 4y ago

V8 dev here. Yes, please avoid leaking memory -- but the intro to this article is incorrect in almost all major points: > In JavaScript, primitives are assigned/passed by value and not by reference. If you assign a primitive type's value to another variable, the value is copied. No, primitives are immutable but still passed by reference (that reference is passed by value, which is why assigning to the reference won't change them -- the same is true for objects). Can you imagine if we'd copied strings every time they were passed into a function? > This also determines how they are stored in memory — primitive values are stored on the stack while objects are stored on the heap, with a reference to their location stored on the stack. Also no, all primitives except small integers ("Smis") are stored in the heap. > One important thing to note about variables stored in the stack is that their sizes must be known at compile time. This allows a program to allocate the right amount of memory in the stack. Because the JavaScript engine allocates a fixed amount of memory for primitive values, there is a limit to how large they can be. Again, no, with strings as the obvious counter example. There's a limit to how large values in general can be, but it's around indexable size and pointer width, not a fixed amount of memory for primitives. > Memory stored in the heap will live on unless it is explicitly deleted or deallocated. Is the author confusing the malloc heap with the (garbage collected) JS heap? Sure, V8 has to manage the pages it allocates, but JS authors certainly don't > To allocate memory in the heap, a program must first scan the heap to find a contiguous memory block large enough to hold the data. As a sibling comment points out, there are optimisations here with linear allocation buffers that permit bump allocation. Also, don't forget the semi space for young objects, since the GC is generational.

kevincox 4y ago
> No, primitives are immutable but still passed by reference For immutable values the pass-by-reference vs pass-by-value distinction isn't really important. Unless the language provides some way to get the identity of the object there isn't a way to tell a difference. > Can you imagine if we'd copied strings every time they were passed into a function? IIUC most Javascript interpreters optimize copying strings anyways, so this wouldn't be a big deal. For example even though `s.slice(1)` is a "copy" most engines (I believe V8 included) will use the same backing string for both. So the difference would be a pointer on the stack vs a pointer+length on the stack and copying an extra word is not a huge difference. But maybe this is just being nitpicky, since the article is focused on performance the lower level details do mater not the semantics. However for strings I don't think the difference between their model and reality is very significant.

aliceryhl 4y ago

> If you assign a primitive type's value to another variable, the value is copied - not assigned a reference to the first variable (which is what happens for objects). Assigning an object also just copies the value - i.e. the reference to the heap value. So the new variable does not end up pointing to the first variable. Rather, both variables end up pointing to the same heap value.

graderjs 4y ago

I like this, it looks like a good reference on memory management...but I see Leszek saying, "no". OK, maybe the authors don't know the theory, but they learned some useful tricks to reduce. Will still read. Related, when I need lots of memory in node, I use: `--max-old-space-size=` programmatically set to some % of total available memory on the system.

Leszek 4y ago
To be clear, maybe I was a bit negative, but I was just talking about the technical intro. The rest of the article on not accidentally leaking references through globals and timers, and what tools to use to catch them, is pretty solid.

kaladin_1 4y ago

The errors have already been pointed out by multiple comments. Opening the article, it is immediately obvious that the author sourced his material from random Js blogs. He understands very little about the v8 engine and just fills the gap with those material which are obviously wrong. On the hand, he can be happy that his article made it here where he would be corrected for free. Hope the author hangs out here...

deleted 4y ago

[deleted]

sod 4y ago

If you want to reduce memory consumption by your node.js server, minify it (e.g. via esbuild in node.js mode) and don't use inline source maps (if you want to keep source maps, use .map files). node.js keeps all source code as string in memory. So unminified code can bloat your runtime by a lot if you require a couple of external libraries. If you bundle the libraries as well, you don't even need to copy node_modules anymore, which can also save time in deploying a change. This small change can cut your memory requirement in half. Of course I wish node.js just had an option to free the memory of the source code garbage, but as of version 18.9.1 there is none.

losvedir 4y ago

Am I missing something, or does the article not go into its title of minimizing heap allocations at all? It talks about not having memory leaks, but no strategies for minimizing the use of the heap in the first place. As I understand it, this is an important technique to eke out the most performance. I know I've seen an article on minimizing allocations on Go, and doesn't Java have "value types" for that sort of thing? I would be interested in the JS equivalent, and am disappointed the article didn't actually get into it. It's imagine it's stuff like using byte arrays and such?

olliej 4y ago

This is not a good article, it confuses immutable with fixed size, claims primitives are stored on the stack but includes strings in that set, and then claims that V8 used to have a problem with cycles because it used refcounting???? Also I found the writing itself to be long winded and hard to read even without the many technical errors.