3 ms·

This seems like genuinely good stuff. I'm not a systems guy, so question: How long would it theoretically take for somebody to hack NodeJS to use this? What abo

This seems like genuinely good stuff. I'm not a systems guy, so question: How long would it theoretically take for somebody to hack NodeJS to use this? What about getting NodeJS to use zero-copy buffers as be mentioned, by somehow overriding JSON.stringify? Thanks.

cwp 12y ago
Impossible to know up front. I think one issue you'd run across when doing something like that it that Node is not an application in the sense that Redis is an application. It's mostly a JIT-compiler for applications that are written in Javascript. Porting a Node application to run on Arrakis would probably involve changes to both Node and the Javascript application code, particularly if you want to run close to the hardware performance limits. That's doable, but I doubt you could preserve the APIs that Node currently exposes to applications. (The FS module would be especially problematic!) What could be useful is to ditch Node's existing IO APIs and provide bindings to the Arrakis IO library directly.
binarymax 12y ago
I'm not a systems guy either, but from watching the video, I don't think it would be applicable. The example used was Redis and Memcached which, while both having rich APIs, are very purpose built for data tasks. The purpose of Arrakis is to remove the kernel layer from the data-plane for minimum overhead when doing that type of I/O. Node is not a data specific application and won't live entirely in the data-plane. It is general purpose and libuv especially has a lot of moving parts that might blur between the data-plane and control-plane. I would imagine you would have better luck building a unikernel dedicated for node than to try and shoehorn node into Arrakis.
Animats 12y ago
Don't get too excited about zero-copy. Copying is cheap if the data was recently referenced and is in cache. Conversely, if the data was put in memory by a peripheral device, it costs almost as much as a copy to get it into the CPU's caches. The bookkeeping associated with zero-copy often exceeds the copying cost. This was the curse of the original message-passing Mach implementation, which gave microkernels a bad name.
- marknadal 12y ago
  What are your thoughts on http://kentonv.github.io/capnproto/ http://kentonv.github.io/capnproto/ ?
  - Animats 12y ago
    That the sender can probably crash the receiver with a malformed offset in a message. I'd like to see marshalling as a language feature. It's compilable, done often, and has an effect on performance. Many marshalling systems, from OpenRPC to protocol buffers, use a precompiler. But that adds another level of language.
    - geofft 12y ago
      > That the sender can probably crash the receiver with a malformed offset in a message. Errr, that would be a major bug in capnproto. While it's definitely possible the software has bugs, it's certainly a design constraint that the sender absolutely cannot crash the receiver. http://kentonv.github.io/capnproto/faq.html#arent-messages-that-contain-pointers-a-huge-security-problem http://kentonv.github.io/capnproto/faq.html#arent-messages-t...

cwp 12y ago

Impossible to know up front. I think one issue you'd run across when doing something like that it that Node is not an application in the sense that Redis is an application. It's mostly a JIT-compiler for applications that are written in Javascript. Porting a Node application to run on Arrakis would probably involve changes to both Node and the Javascript application code, particularly if you want to run close to the hardware performance limits. That's doable, but I doubt you could preserve the APIs that Node currently exposes to applications. (The FS module would be especially problematic!) What could be useful is to ditch Node's existing IO APIs and provide bindings to the Arrakis IO library directly.

binarymax 12y ago

I'm not a systems guy either, but from watching the video, I don't think it would be applicable. The example used was Redis and Memcached which, while both having rich APIs, are very purpose built for data tasks. The purpose of Arrakis is to remove the kernel layer from the data-plane for minimum overhead when doing that type of I/O. Node is not a data specific application and won't live entirely in the data-plane. It is general purpose and libuv especially has a lot of moving parts that might blur between the data-plane and control-plane. I would imagine you would have better luck building a unikernel dedicated for node than to try and shoehorn node into Arrakis.

Animats 12y ago

Don't get too excited about zero-copy. Copying is cheap if the data was recently referenced and is in cache. Conversely, if the data was put in memory by a peripheral device, it costs almost as much as a copy to get it into the CPU's caches. The bookkeeping associated with zero-copy often exceeds the copying cost. This was the curse of the original message-passing Mach implementation, which gave microkernels a bad name.

marknadal 12y ago
What are your thoughts on http://kentonv.github.io/capnproto/ http://kentonv.github.io/capnproto/ ?
- Animats 12y ago
  That the sender can probably crash the receiver with a malformed offset in a message. I'd like to see marshalling as a language feature. It's compilable, done often, and has an effect on performance. Many marshalling systems, from OpenRPC to protocol buffers, use a precompiler. But that adds another level of language.
  - geofft 12y ago
    > That the sender can probably crash the receiver with a malformed offset in a message. Errr, that would be a major bug in capnproto. While it's definitely possible the software has bugs, it's certainly a design constraint that the sender absolutely cannot crash the receiver. http://kentonv.github.io/capnproto/faq.html#arent-messages-that-contain-pointers-a-huge-security-problem http://kentonv.github.io/capnproto/faq.html#arent-messages-t...