2 ms·
I've done CPU laden tasks many times in Node. The key is how you break up the heavy task into pieces that can be distributed more evenly on the event loop inste
by NathanKP 11y ago
I've done CPU laden tasks many times in Node. The key is how you break up the heavy task into pieces that can be distributed more evenly on the event loop instead of blocking it. For example processing a huge data stream is typically done by breaking it into chunks where each chunk can be processed in a few ms, rather than doing one giant blocking operation on the entire stream.
Done properly this also leads to a low memory process that will grab data from an incoming socket stream, transform it, then push that data back out through an outgoing socket stream, only ever keeping a few MB of the stream in working memory at a time. Slap a cluster of these streaming processes together running on a single box to utilize all CPU cores and you can easily push many Mbps of IO through your server quickly and efficiently.
Additionally even if you do have a heavy blocking system IO operation that you can't break up very well you can increase `UV_THREADPOOL_SIZE` when needed to reduce the impact of the blocking.
- SignMeTheHELLUp 11y agoBut now you've just implemented thread context switching by pulling it's implementation details into your algorithm... Just use a language that supports threading if you have real work to do...
- NathanKP 11y agoSometimes it is more efficient to implement the context switching yourself. For extremely high performance server use cases such as realtime analytics, streaming video processing, high frequency trading, etc it isn't uncommon to have server code written in C/C++. It really depends on what your most important metric is. If the most important metric is speed then having more control over context switching is often better.