7 ms·

A guide to threads in Node.js

165 points by bibyte 8y ago

com2kid 8y ago
I wasn't a huge fan of the "single thread for everything" paradigm, then I helped design a single threaded cooperatively multitasked embedded OS in C/C++. No overhead from context switches. Everyone who is running knows how long they have to run, and knows that they are NOT going to get interrupted. No need to worry about locks or how to share data. Want to pass data to another module? Just pass it through well defined interfaces and you darn well know there will never be a read/write conflict, and the next time that module's code runs, it'll have access to that data. (No queue!) The only exception, and what made it all possible, was the interrupt routines from hardware[1]. Anyone who subscribed to hardware events (entire OS was subscription based, no polling reads ever) had to implement a "thunk pattern" to put data into a receive buffer. The design pattern to do this was the same everywhere in all modules, making code understandable across the entire project. It is an incredibly freeing paradigm to write in. It becomes so much easier to prove[2] the correctness of code when you can read all the code straight through and not have to ever worry about someone stomping on your data. It wasn't an RTOS, but even so, it becomes really easy to start providing performance guarantees. Internal builds had a watchdog timer[3] that would crash the device if it wasn't 'kicked' every so often. Set that to 3ms, start working with the code, and the stack traces tell you instantly who is over their CPU budget. Rewrite code and break it apart into multiple chunks that are scheduled for later execution, repeat until everyone is under their CPU allotment. For many tasks, single threaded code is nice. Getting rid of preemption is even nicer. Preemptive multithreading is a compromise. It means that no one thread/process can bring down the system by hogging 100% of resources, but it also creates a huge overhead where important work, work that makes for a better user experience, can (will!) get interrupted for work that honestly doesn't need to be done right now. The solution to this is just throw so much CPU at the problem that everything gets done in a reasonable amount of time. It has often been noted that "reasonable amount of time" means systems today are less responsive than a 486 running DOS from 1992. Of course it isn't reasonable to have a modern cooperatively multithreaded consumer OS, no way would the hundreds of processes ran at any one time all cooperate with each other. But if you ever get a chance to write code on a single threaded cooperative system, go do it. It is a lot of fun. Now all this meant that going from embedded C to NodeJS wasn't that large of a mental leap! Not having to directly read bytes off the wire was weird (seriously, took a bit of getting use to), but it turns out that "get data, do work, schedule what needs to be done, return early" ends up being the same paradigm at both the top and the bottom of the programming stacks! [1] All I/O was done using DMA engines, basically a fancy limited programmable piece of hardware that can read and write to all the different pieces of HW hanging off of the main chip, so for example as Bluetooth packets come in, the DMA engine shoves the packets into a buffer and when the buffer is full it raises an interrupt that lets the CPU know that data is waiting. It looks almost exactly like async I/O in any of the modern programming languages, except you have direct access to all that IO being hardware offloaded. Writing to the bare metal rocks. [2] For a reasonable enough degree of "prove" that software is reliable and doesn't crash from threading issues [3] A watchdog timer is a physical timer hooked across the power lines of your chip. If it isn't activated every so often (in the industry this is called "kicking the watchdog") it will, in debug builds it does a controlled crash of the CPU, and in retail builds it will reset the entire system. If you've ever had an embedded device reset itself before your very eyes, it is becomes the CPU got locked up and no one kicked the watchdog, so the entire system emergency reset itself.
- tonyarkles 8y ago
  Now and then I get to work in systems like this, and you're right. It's a joy! And thank you, I really like the idea of setting the watchdog to really small timeouts to put upper bounds on execution time in debug builds. That's a fantastically clever use of it! For the record, I much prefer the term "pet the watchdog" :).
  - com2kid 8y ago
    >For the record, I much prefer the term "pet the watchdog" :). I think the EEs who setup the watchdog were far too grizzled. :D
    - tonyarkles 8y ago
      Hah! I've only got a little bit of grey in my beard so far...
  - striking 8y ago
    Some others say "feed the watchdog", which to me makes the most sense.
- caprese 8y ago
  It took a little re-learning for me, since the aughts had so much furor and exuberance for multithreaded possibilities, as a holdover from using the newly abundant computational resources as efficiently as possible. The reality is, or has become, that the abundance of computational resources has resulted in computers being able to be created in the aether spontaneously, groups of computers even. Making them do one thing in a consistent time span is more important than filling up the theoretical limitation of their allocated resources. Computer in this context being the elements that allow for computation, as much as an abacus or a professional human has been called a computer. Since many times these compute instances reside solely in the memory of a host machine, somewhere up the chain are the machines made of metal. A cluster of node processes, of which have their own external memory store and their own database, communicating via REST, is great. The distinction between programming languages being moved into further into irrelevancy, with people that haven't re-learned this crying discrimination, now that their once-useful elitist gatekeeping only serves to remind everyone else how out of touch this person is. It is fascinating how many computers we actually use now. When factoring in the CDNs over top of our redundant clusters containing single threaded processes with an occasional worker.
dboreham 8y ago
Haven't read the article but I'm assuming it's an empty page? <ducks>
- hombre_fatal 8y ago
  No.
- benatkin 8y ago
  It is empty of the kind of threads where you don’t have to worry about promises (with or without async/await) or callbacks.
Trisell 8y ago
Honest question. Are we going to be able to pass functions with data to threads? Or are these only designed to use files? Which makes this feel like a better wrapper on the child process api that already exists.
- ryanpetrich 8y ago
  The code to run in the worker is passed as a string or a path, so no data can be captured. Data can be sent to the worker by posting messages, but the data is never shared—only copied. A limited set of types can be transferred to a worker, after which point they will become unusable in the parent.
  - 11235813213455 8y ago
    For sharing (byte) data, I think there is SharedArrayBuffer https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... But else, yes, the sent and received data is serialized (when using ipc, else you can also send raw data through streams, and handle the serialization yourself)
    - primq 8y ago
      memory sharing is explained in the article.
    - rndgermandude 8y ago
      SharedArrayBuffer -> shared ArrayBuffer -> ownership can be transferred around, but the data can only be owned by one thread at a time. The data itself isn't copied. Passing around ArrayBuffers back and forth is good enough for a lot of stuff. In a worker you can then put a node Buffer, TypedArray, or DataView around the ArrayBuffer again, if you want.
    - ilaksh 8y ago
      Supposedly SharedArrayBuffer and atomics have been in Node.js since version 9. I wonder if there are any articles where people use them in threads.
tannhaeuser 8y ago
> It was only in 2009 that Ryan Dahl, creator of Node.js, made it possible for developers to use the language to write back-end code Server-side JavaScript was a thing long before that. In fact, Netscape used it as early as 1994 [1], predating Java as a backend language. And CommonJS (on what nodejs' modules and many APIs are based) was a community effort towards a common API by 2000s SSJS implementations (helma, v8cgi/teajs, and others). Apart from that, a nice read for those who need a WebWorker-like API for CPU-bound nodejs tasks. [1]: https://en.wikipedia.org/wiki/Server-side_scripting https://en.wikipedia.org/wiki/Server-side_scripting [2]: http://wiki.commonjs.org/wiki/CommonJS http://wiki.commonjs.org/wiki/CommonJS
- denysonique 8y ago
  There was also Aptana Jaxer around 2008: https://en.wikipedia.org/wiki/Aptana#Aptana_Jaxer https://en.wikipedia.org/wiki/Aptana#Aptana_Jaxer http://www.jaxer.org/ http://www.jaxer.org/
  - bryanrasmussen 8y ago
    I wish Aptana had won the race, or at least if not beat Node, competed reasonably with Node.
- kbenson 8y ago
  And Rhino[1], which is a Javascript engine written by Netscape in Java, originally for a Java based browser but used quite a bit on the server when that project was killed). 1: https://en.wikipedia.org/wiki/Rhino_(JavaScript_engine) https://en.wikipedia.org/wiki/Rhino_(JavaScript_engine)
- egeozcan 8y ago
  I was playing with SilkJS at some point. Not sure if it was there earlier than node but the earliest archive is from 2012 https://web.archive.org/web/20120108050201/http://silkjs.org/ https://web.archive.org/web/20120108050201/http://silkjs.org... Funny now there's zero references to it on the internet. I had really liked it back then. It was fully synchronous and had support for threading. Cool stuff: https://web.archive.org/web/20130615124030/http://silkjs.net/ https://web.archive.org/web/20130615124030/http://silkjs.net... Edit: Repository seems to be alive! https://github.com/mschwartz/SilkJS/ https://github.com/mschwartz/SilkJS/
crabasa 8y ago
> type WorkerCallback = (err: any, result?: any) => any; export function runWorker(path: string, cb: WorkerCallback, workerData: object | null = null) { const worker = new Worker(path, { workerData }); worker.on('message', cb.bind(null, null)); worker.on('error', cb); worker.on('exit', (exitCode) => { if (exitCode === 0) { return null; } return cb(new Error(`Worker has stopped with code ${exitCode}`)); }); return worker; } For an article titled "A Guide to Threads in Node.js", I really wish the first example of writing a thread wasn't in TypeScript.
- danenania 8y ago
  Why? Just think of it as inline documentation. It in no way changes the semantics of the javascript and efficiently communicates how the API works.
  - kbenson 8y ago
    Probably because it adds extra cognitive overhead for anyone that doesn't know TypeScript as they interpret and translate TypeScript into JavaScript in their head? Or perhaps because it's slightly misleading?
    - GordonS 8y ago
      As a staunch supporter of static typing, I agree with you - the target audience is very obviously those from a JavaScript background, so it makes sense for the code snippets to be in JavaScript, not Typescript. If the author wanted to add additional info for context, IMO in this instance they should have used a JSDoc comment.
      - danenania 8y ago
        TypeScript is a more expressive, concise, intuitive, and popular version of jsdoc that can be statically enforced by a compiler, provide autocompletion, and will never get out of date. I just don’t see the downside of using TS in place of jsdoc—the syntax is even pretty comparable.
Scarbutt 8y ago
With the javascript community fixing most of it shortcomings, I suspect in a few years JS/TS is going to be way far ahead of any programming language in term of users.
- GordonS 8y ago
  Interesting that you say JS/TS - while Typescript of course transpile to JavaScript, the 2 languages take very different approaches to something that really divides devs - the question of static vs dynamic typing.
  - mynegation 8y ago
    Exactly. Together, this pair of languages will appeal to a broader range of developers - those preferring dynamic typing, those preferring static typic, and pragmatists who start the quick prototype in JS and move to TS for better maintainability.
  - Scarbutt 8y ago
    Added TS there because they share the runtime and the ecosystem in an almost seamless way, TS users use JS libs and vice versa. TS is being forced on JS users in indirect ways, so JS users will need to be able to at least read TS at some point. Surprisingly, npm inc states that 46% of npm users are already using TS.
sbhn 8y ago
A good video on parallel processing CPU bound taks with nodejs. https://youtu.be/ZYfSe9qKaZE https://youtu.be/ZYfSe9qKaZE

165 points by bibyte 8y ago

I wasn't a huge fan of the "single thread for everything" paradigm, then I helped design a single threaded cooperatively multitasked embedded OS in C/C++. No overhead from context switches. Everyone who is running knows how long they have to run, and knows that they are NOT going to get interrupted. No need to worry about locks or how to share data. Want to pass data to another module? Just pass it through well defined interfaces and you darn well know there will never be a read/write conflict, and the next time that module's code runs, it'll have access to that data. (No queue!) The only exception, and what made it all possible, was the interrupt routines from hardware[1]. Anyone who subscribed to hardware events (entire OS was subscription based, no polling reads ever) had to implement a "thunk pattern" to put data into a receive buffer. The design pattern to do this was the same everywhere in all modules, making code understandable across the entire project. It is an incredibly freeing paradigm to write in. It becomes so much easier to prove[2] the correctness of code when you can read all the code straight through and not have to ever worry about someone stomping on your data. It wasn't an RTOS, but even so, it becomes really easy to start providing performance guarantees. Internal builds had a watchdog timer[3] that would crash the device if it wasn't 'kicked' every so often. Set that to 3ms, start working with the code, and the stack traces tell you instantly who is over their CPU budget. Rewrite code and break it apart into multiple chunks that are scheduled for later execution, repeat until everyone is under their CPU allotment. For many tasks, single threaded code is nice. Getting rid of preemption is even nicer. Preemptive multithreading is a compromise. It means that no one thread/process can bring down the system by hogging 100% of resources, but it also creates a huge overhead where important work, work that makes for a better user experience, can (will!) get interrupted for work that honestly doesn't need to be done right now. The solution to this is just throw so much CPU at the problem that everything gets done in a reasonable amount of time. It has often been noted that "reasonable amount of time" means systems today are less responsive than a 486 running DOS from 1992. Of course it isn't reasonable to have a modern cooperatively multithreaded consumer OS, no way would the hundreds of processes ran at any one time all cooperate with each other. But if you ever get a chance to write code on a single threaded cooperative system, go do it. It is a lot of fun. Now all this meant that going from embedded C to NodeJS wasn't that large of a mental leap! Not having to directly read bytes off the wire was weird (seriously, took a bit of getting use to), but it turns out that "get data, do work, schedule what needs to be done, return early" ends up being the same paradigm at both the top and the bottom of the programming stacks! [1] All I/O was done using DMA engines, basically a fancy limited programmable piece of hardware that can read and write to all the different pieces of HW hanging off of the main chip, so for example as Bluetooth packets come in, the DMA engine shoves the packets into a buffer and when the buffer is full it raises an interrupt that lets the CPU know that data is waiting. It looks almost exactly like async I/O in any of the modern programming languages, except you have direct access to all that IO being hardware offloaded. Writing to the bare metal rocks. [2] For a reasonable enough degree of "prove" that software is reliable and doesn't crash from threading issues [3] A watchdog timer is a physical timer hooked across the power lines of your chip. If it isn't activated every so often (in the industry this is called "kicking the watchdog") it will, in debug builds it does a controlled crash of the CPU, and in retail builds it will reset the entire system. If you've ever had an embedded device reset itself before your very eyes, it is becomes the CPU got locked up and no one kicked the watchdog, so the entire system emergency reset itself.

tonyarkles 8y ago
Now and then I get to work in systems like this, and you're right. It's a joy! And thank you, I really like the idea of setting the watchdog to really small timeouts to put upper bounds on execution time in debug builds. That's a fantastically clever use of it! For the record, I much prefer the term "pet the watchdog" :).
- com2kid 8y ago
  >For the record, I much prefer the term "pet the watchdog" :). I think the EEs who setup the watchdog were far too grizzled. :D
  - tonyarkles 8y ago
    Hah! I've only got a little bit of grey in my beard so far...
- striking 8y ago
  Some others say "feed the watchdog", which to me makes the most sense.
caprese 8y ago
It took a little re-learning for me, since the aughts had so much furor and exuberance for multithreaded possibilities, as a holdover from using the newly abundant computational resources as efficiently as possible. The reality is, or has become, that the abundance of computational resources has resulted in computers being able to be created in the aether spontaneously, groups of computers even. Making them do one thing in a consistent time span is more important than filling up the theoretical limitation of their allocated resources. Computer in this context being the elements that allow for computation, as much as an abacus or a professional human has been called a computer. Since many times these compute instances reside solely in the memory of a host machine, somewhere up the chain are the machines made of metal. A cluster of node processes, of which have their own external memory store and their own database, communicating via REST, is great. The distinction between programming languages being moved into further into irrelevancy, with people that haven't re-learned this crying discrimination, now that their once-useful elitist gatekeeping only serves to remind everyone else how out of touch this person is. It is fascinating how many computers we actually use now. When factoring in the CDNs over top of our redundant clusters containing single threaded processes with an occasional worker.

dboreham 8y ago

Haven't read the article but I'm assuming it's an empty page? <ducks>

hombre_fatal 8y ago
No.
benatkin 8y ago
It is empty of the kind of threads where you don’t have to worry about promises (with or without async/await) or callbacks.

Trisell 8y ago

Honest question. Are we going to be able to pass functions with data to threads? Or are these only designed to use files? Which makes this feel like a better wrapper on the child process api that already exists.

ryanpetrich 8y ago
The code to run in the worker is passed as a string or a path, so no data can be captured. Data can be sent to the worker by posting messages, but the data is never shared—only copied. A limited set of types can be transferred to a worker, after which point they will become unusable in the parent.
- 11235813213455 8y ago
  For sharing (byte) data, I think there is SharedArrayBuffer https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... But else, yes, the sent and received data is serialized (when using ipc, else you can also send raw data through streams, and handle the serialization yourself)
  - primq 8y ago
    memory sharing is explained in the article.
  - rndgermandude 8y ago
    SharedArrayBuffer -> shared ArrayBuffer -> ownership can be transferred around, but the data can only be owned by one thread at a time. The data itself isn't copied. Passing around ArrayBuffers back and forth is good enough for a lot of stuff. In a worker you can then put a node Buffer, TypedArray, or DataView around the ArrayBuffer again, if you want.
  - ilaksh 8y ago
    Supposedly SharedArrayBuffer and atomics have been in Node.js since version 9. I wonder if there are any articles where people use them in threads.

tannhaeuser 8y ago

> It was only in 2009 that Ryan Dahl, creator of Node.js, made it possible for developers to use the language to write back-end code Server-side JavaScript was a thing long before that. In fact, Netscape used it as early as 1994 [1], predating Java as a backend language. And CommonJS (on what nodejs' modules and many APIs are based) was a community effort towards a common API by 2000s SSJS implementations (helma, v8cgi/teajs, and others). Apart from that, a nice read for those who need a WebWorker-like API for CPU-bound nodejs tasks. [1]: https://en.wikipedia.org/wiki/Server-side_scripting https://en.wikipedia.org/wiki/Server-side_scripting [2]: http://wiki.commonjs.org/wiki/CommonJS http://wiki.commonjs.org/wiki/CommonJS

denysonique 8y ago
There was also Aptana Jaxer around 2008: https://en.wikipedia.org/wiki/Aptana#Aptana_Jaxer https://en.wikipedia.org/wiki/Aptana#Aptana_Jaxer http://www.jaxer.org/ http://www.jaxer.org/
- bryanrasmussen 8y ago
  I wish Aptana had won the race, or at least if not beat Node, competed reasonably with Node.
kbenson 8y ago
And Rhino[1], which is a Javascript engine written by Netscape in Java, originally for a Java based browser but used quite a bit on the server when that project was killed). 1: https://en.wikipedia.org/wiki/Rhino_(JavaScript_engine) https://en.wikipedia.org/wiki/Rhino_(JavaScript_engine)
egeozcan 8y ago
I was playing with SilkJS at some point. Not sure if it was there earlier than node but the earliest archive is from 2012 https://web.archive.org/web/20120108050201/http://silkjs.org/ https://web.archive.org/web/20120108050201/http://silkjs.org... Funny now there's zero references to it on the internet. I had really liked it back then. It was fully synchronous and had support for threading. Cool stuff: https://web.archive.org/web/20130615124030/http://silkjs.net/ https://web.archive.org/web/20130615124030/http://silkjs.net... Edit: Repository seems to be alive! https://github.com/mschwartz/SilkJS/ https://github.com/mschwartz/SilkJS/

crabasa 8y ago

> type WorkerCallback = (err: any, result?: any) => any; export function runWorker(path: string, cb: WorkerCallback, workerData: object | null = null) { const worker = new Worker(path, { workerData }); worker.on('message', cb.bind(null, null)); worker.on('error', cb); worker.on('exit', (exitCode) => { if (exitCode === 0) { return null; } return cb(new Error(`Worker has stopped with code ${exitCode}`)); }); return worker; } For an article titled "A Guide to Threads in Node.js", I really wish the first example of writing a thread wasn't in TypeScript.

danenania 8y ago
Why? Just think of it as inline documentation. It in no way changes the semantics of the javascript and efficiently communicates how the API works.
- kbenson 8y ago
  Probably because it adds extra cognitive overhead for anyone that doesn't know TypeScript as they interpret and translate TypeScript into JavaScript in their head? Or perhaps because it's slightly misleading?
  - GordonS 8y ago
    As a staunch supporter of static typing, I agree with you - the target audience is very obviously those from a JavaScript background, so it makes sense for the code snippets to be in JavaScript, not Typescript. If the author wanted to add additional info for context, IMO in this instance they should have used a JSDoc comment.
    - danenania 8y ago
      TypeScript is a more expressive, concise, intuitive, and popular version of jsdoc that can be statically enforced by a compiler, provide autocompletion, and will never get out of date. I just don’t see the downside of using TS in place of jsdoc—the syntax is even pretty comparable.

Scarbutt 8y ago

With the javascript community fixing most of it shortcomings, I suspect in a few years JS/TS is going to be way far ahead of any programming language in term of users.

GordonS 8y ago
Interesting that you say JS/TS - while Typescript of course transpile to JavaScript, the 2 languages take very different approaches to something that really divides devs - the question of static vs dynamic typing.
- mynegation 8y ago
  Exactly. Together, this pair of languages will appeal to a broader range of developers - those preferring dynamic typing, those preferring static typic, and pragmatists who start the quick prototype in JS and move to TS for better maintainability.
- Scarbutt 8y ago
  Added TS there because they share the runtime and the ecosystem in an almost seamless way, TS users use JS libs and vice versa. TS is being forced on JS users in indirect ways, so JS users will need to be able to at least read TS at some point. Surprisingly, npm inc states that 46% of npm users are already using TS.

sbhn 8y ago

A good video on parallel processing CPU bound taks with nodejs. https://youtu.be/ZYfSe9qKaZE https://youtu.be/ZYfSe9qKaZE