4 ms·

> How can this be so much faster than even pg-native that uses the C library libpq? the c++ to javascript membrane in v8 is expensive to pass. there are many

> How can this be so much faster than even pg-native that uses the C library libpq? the c++ to javascript membrane in v8 is expensive to pass. there are many projects that are faster implementing protocols in javascript vs c/c++ because of this. redis is another.

eyelidlessness 5y ago
This can’t be overstated. Crossing the JS<->native barrier is inherently a perf hit. I’ve measured it trying a large variety of high performance messaging protocols and postMessage with structured clone nearly always wins. If that sounds like a bold claim, there are troves of Node and Deno issues where they’ve improved perf by staying in JS specifically because calling into native and back has been the bottleneck. There are countless similar issues in V8 and WebKit as well. You can get a perf boost with WASM or NAPI, but only for workloads which are CPU bound and where you expect to do that work in compiled code.
- w1nk 5y ago
  I responded to the OP, but I'll add here too for the conversation. Where are these projects that are implementing network protocols, where the nodejs versions are faster than the native ones? As I noted in my other comment, in -any- of those implementations, you're still going to be required to traverse from libc somewhere to nodejs (even if it's just to read the network data out of the socket and send it to v8). The performance gains would have to either come from a faster (JIT'd) protocol parser, or from eliminating additional FFI calls (that it isn't obvious why they'd exist). So, would you mind linking to one or some of those projects? I'd love to see what happened to the implementations.
  - porsager 5y ago
    I think you're right. Here's a project proving the exact opposite of native bindings being slow - https://github.com/uNetworking/uWebSockets.js https://github.com/uNetworking/uWebSockets.js
    - jerrysievert 5y ago
      I think you might find that a large part of their speedup is by not using any of the openssl trappings of node, and using boringssl instead.
  - jerrysievert 5y ago
    > The performance gains would have to either come from a faster (JIT'd) protocol parser, or from eliminating additional FFI calls (that it isn't obvious why they'd exist). or, as I noted above, and will try to simplify here: let's suppose you have a buffer which contains 5 key/value pairs, and you want to convert that into an array of 5 javascript objects in v8. 1. obtain a pointer to the isolate 2. create an array object, and check 3. create 5 objects, and check each one 4. create 5 key objects, and check each one 5. create 5 value objects, and check each one 6. convert each key into a v8 typed object, and check each one 7. convert each value into a v8 typed object, and check each one 8. attach each key to each object, and check each one 9. attach each value to each object, and check each one 10. attach each object to the array, and check each one 11. check that array one more time 12. done vs. 1. obtain a pointer to the isolate 2. create a buffer, and check 3. parse in javascript inside of v8, creating a javascript object inside of javascript is much faster than doing the same via c++, because of the additional checks that are needed each time you cross that barrier. Local<v8::Value>::New(isolate, String::NewFromUtf8(isolate, t).ToLocalChecked()); is an example of simply creating a string, not assigning it into an object.
    - w1nk 5y ago
      Hey Jerry! I'll respond here so we don't have split threads :). I'm also quite familiar with at least the older versions of the node internals (I too maintained a popular db binding for a number of years) and I'm very confused by the way you're positioning the operation of the v8 vm in these 2 scenarios. Sure, the c++ is going to require you to do some sanitizing as you force your data into v8, but as we noted that's inevitable no matter how you slice it. After that though, even once in javascript land, you're still crossing these barriers constantly to allocate data and objects and memory, etc. You don't just end at 'parse in javascript', the virtual machine is going nuts calling into this same c++ codebase. Now maybe in some cases the v8 internals offer some advantages the generic c++ api can't access, but this argument isn't convincing of their existence so far. My memories of the redis client is different than yours so I'd be quite interested to see those conversations / benchmarks. From what I recall those early advantages in the js redis client were similar to the ones we're seeing here, ie: better pipelining of commands. As a simple thought experiment, in the scenario you're describing we should see a javascript implementation of a JSON parser to beat the pants off the v8 engine implementation, but this doesn't seem to the case.
  - eyelidlessness 5y ago
    I can’t link to any projects, implementing network protocols per se isn’t what I focused on researching this. My focus was on optimizing postMessage between threads. They’re not that different but the important distinction is that the user-facing API is in JS, not a network boundary. My hypothesis was that: 1. Converting to binary data 2. Using facilities for shared memory Would yield better performance in a language well suited to de/serialize binary data, with a message encoding designed for performance. The latter does perform better! If you need to spend a lot of CPU time wrangling binary data, it’s a clear win. If you have a workload with many small messages, they invariably slow down compared to structuredClone. If your workload can avoid crossing the boundary that’s a clear win still. But the moment JS VM values need to get shuttled around, structuredClone is specifically optimized for structuredClone-able values, and optimizing the JS/native boundary for data which benefits from that is an extremely narrow edge case. The only way I’ve found to win is not to play. That said I don’t have the cleverness a lot of performance geeks have and maybe there’s some technique I’ve missed! But I honestly can’t imagine how I’d optimize JS/native interop better than the JIT without finding myself getting surprising new career opportunities on a VM team.
w1nk 5y ago
On the surface I'm not sure this explanation passes the smell test. Almost irrespective of how you get the data from your network card into v8 / nodejs, you're going to be crossing c++/v8 boundaries. It's possible that a native binding goes through this expense multiple times, but that's not likely now most of these bindings work (pg at least for sure). As the author noted elsewhere in thread, the performance gains are mostly coming from prepared statements and pipelining which seems a bit of an apples to oranges benchmark as most of these native drivers will support similar features, just not enable them transparently like this library does. Out of curiosity, do you have links to these other projects where they have similar benchmarking attempts/results?
- jerrysievert 5y ago
  > On the surface I'm not sure this explanation passes the smell test. Almost irrespective of how you get the data from your network card into v8 / nodejs, you're going to be crossing c++/v8 boundaries. yes, you are, but the differences are the object creation that occurs. a single buffer coming from c/c++ (a socket, let's say) can be parsed and turned into a large number of objects in javascript much more quickly. yes, you're passing through that barrier once, but creating all of those objects from c++ and passing through it 20-30 times is a lot more expensive. > Out of curiosity, do you have links to these other projects where they have similar benchmarking attempts/results? how about pg vs pg-native? https://github.com/porsager/postgres-benchmarks#results https://github.com/porsager/postgres-benchmarks#results and unfortunately, I cannot find the original discussions from when node-redis went from native to pure javascript, but it was about a 30-40% speed increase originally if memory serves (I was the one who did that original conversion after a lot of deep dives into v8 and performance crossing the barrier). as an aside, I'm also the maintainer of plv8, and am happy to discuss the same types of performance issues of dealing with jsonb vs json (which in Postgres is text): creating objects vs a simple JSON.parse() in c++ is a significant difference.

eyelidlessness 5y ago

This can’t be overstated. Crossing the JS<->native barrier is inherently a perf hit. I’ve measured it trying a large variety of high performance messaging protocols and postMessage with structured clone nearly always wins. If that sounds like a bold claim, there are troves of Node and Deno issues where they’ve improved perf by staying in JS specifically because calling into native and back has been the bottleneck. There are countless similar issues in V8 and WebKit as well. You can get a perf boost with WASM or NAPI, but only for workloads which are CPU bound and where you expect to do that work in compiled code.

w1nk 5y ago
I responded to the OP, but I'll add here too for the conversation. Where are these projects that are implementing network protocols, where the nodejs versions are faster than the native ones? As I noted in my other comment, in -any- of those implementations, you're still going to be required to traverse from libc somewhere to nodejs (even if it's just to read the network data out of the socket and send it to v8). The performance gains would have to either come from a faster (JIT'd) protocol parser, or from eliminating additional FFI calls (that it isn't obvious why they'd exist). So, would you mind linking to one or some of those projects? I'd love to see what happened to the implementations.
- porsager 5y ago
  I think you're right. Here's a project proving the exact opposite of native bindings being slow - https://github.com/uNetworking/uWebSockets.js https://github.com/uNetworking/uWebSockets.js
  - jerrysievert 5y ago
    I think you might find that a large part of their speedup is by not using any of the openssl trappings of node, and using boringssl instead.
- jerrysievert 5y ago
  > The performance gains would have to either come from a faster (JIT'd) protocol parser, or from eliminating additional FFI calls (that it isn't obvious why they'd exist). or, as I noted above, and will try to simplify here: let's suppose you have a buffer which contains 5 key/value pairs, and you want to convert that into an array of 5 javascript objects in v8. 1. obtain a pointer to the isolate 2. create an array object, and check 3. create 5 objects, and check each one 4. create 5 key objects, and check each one 5. create 5 value objects, and check each one 6. convert each key into a v8 typed object, and check each one 7. convert each value into a v8 typed object, and check each one 8. attach each key to each object, and check each one 9. attach each value to each object, and check each one 10. attach each object to the array, and check each one 11. check that array one more time 12. done vs. 1. obtain a pointer to the isolate 2. create a buffer, and check 3. parse in javascript inside of v8, creating a javascript object inside of javascript is much faster than doing the same via c++, because of the additional checks that are needed each time you cross that barrier. Local<v8::Value>::New(isolate, String::NewFromUtf8(isolate, t).ToLocalChecked()); is an example of simply creating a string, not assigning it into an object.
  - w1nk 5y ago
    Hey Jerry! I'll respond here so we don't have split threads :). I'm also quite familiar with at least the older versions of the node internals (I too maintained a popular db binding for a number of years) and I'm very confused by the way you're positioning the operation of the v8 vm in these 2 scenarios. Sure, the c++ is going to require you to do some sanitizing as you force your data into v8, but as we noted that's inevitable no matter how you slice it. After that though, even once in javascript land, you're still crossing these barriers constantly to allocate data and objects and memory, etc. You don't just end at 'parse in javascript', the virtual machine is going nuts calling into this same c++ codebase. Now maybe in some cases the v8 internals offer some advantages the generic c++ api can't access, but this argument isn't convincing of their existence so far. My memories of the redis client is different than yours so I'd be quite interested to see those conversations / benchmarks. From what I recall those early advantages in the js redis client were similar to the ones we're seeing here, ie: better pipelining of commands. As a simple thought experiment, in the scenario you're describing we should see a javascript implementation of a JSON parser to beat the pants off the v8 engine implementation, but this doesn't seem to the case.
- eyelidlessness 5y ago
  I can’t link to any projects, implementing network protocols per se isn’t what I focused on researching this. My focus was on optimizing postMessage between threads. They’re not that different but the important distinction is that the user-facing API is in JS, not a network boundary. My hypothesis was that: 1. Converting to binary data 2. Using facilities for shared memory Would yield better performance in a language well suited to de/serialize binary data, with a message encoding designed for performance. The latter does perform better! If you need to spend a lot of CPU time wrangling binary data, it’s a clear win. If you have a workload with many small messages, they invariably slow down compared to structuredClone. If your workload can avoid crossing the boundary that’s a clear win still. But the moment JS VM values need to get shuttled around, structuredClone is specifically optimized for structuredClone-able values, and optimizing the JS/native boundary for data which benefits from that is an extremely narrow edge case. The only way I’ve found to win is not to play. That said I don’t have the cleverness a lot of performance geeks have and maybe there’s some technique I’ve missed! But I honestly can’t imagine how I’d optimize JS/native interop better than the JIT without finding myself getting surprising new career opportunities on a VM team.

w1nk 5y ago

On the surface I'm not sure this explanation passes the smell test. Almost irrespective of how you get the data from your network card into v8 / nodejs, you're going to be crossing c++/v8 boundaries. It's possible that a native binding goes through this expense multiple times, but that's not likely now most of these bindings work (pg at least for sure). As the author noted elsewhere in thread, the performance gains are mostly coming from prepared statements and pipelining which seems a bit of an apples to oranges benchmark as most of these native drivers will support similar features, just not enable them transparently like this library does. Out of curiosity, do you have links to these other projects where they have similar benchmarking attempts/results?

jerrysievert 5y ago
> On the surface I'm not sure this explanation passes the smell test. Almost irrespective of how you get the data from your network card into v8 / nodejs, you're going to be crossing c++/v8 boundaries. yes, you are, but the differences are the object creation that occurs. a single buffer coming from c/c++ (a socket, let's say) can be parsed and turned into a large number of objects in javascript much more quickly. yes, you're passing through that barrier once, but creating all of those objects from c++ and passing through it 20-30 times is a lot more expensive. > Out of curiosity, do you have links to these other projects where they have similar benchmarking attempts/results? how about pg vs pg-native? https://github.com/porsager/postgres-benchmarks#results https://github.com/porsager/postgres-benchmarks#results and unfortunately, I cannot find the original discussions from when node-redis went from native to pure javascript, but it was about a 30-40% speed increase originally if memory serves (I was the one who did that original conversion after a lot of deep dives into v8 and performance crossing the barrier). as an aside, I'm also the maintainer of plv8, and am happy to discuss the same types of performance issues of dealing with jsonb vs json (which in Postgres is text): creating objects vs a simple JSON.parse() in c++ is a significant difference.