3 ms·
This was a strange article for me to read. It starts out with a long discussion of Node's "single-threaded compute / multi-threaded I/O" model as if it were a p
by jdlshore 8y ago
This was a strange article for me to read. It starts out with a long discussion of Node's "single-threaded compute / multi-threaded I/O" model as if it were a problem for compute-bound applications. But multithreading doesn't magically make compute-bound tasks go faster. Multiprocessing does, but you can fork a Node server so it runs across multiple processors quite easily.
(Keeps reading...) Ah, about halfway through the article they talk about introducing multiprocessing using the `cluster` module. But then they say the process is tied up until the request is fully received, bottle-necking the process based on client speed, which is not how I understand Node to work at all. Sure, the slow connection will tie up a socket and memory, so a buffering proxy makes sense, but it shouldn't affect compute. Maybe it's a limitation of `cluster`? My experience is with `worker-farm`.
Which brings me to the second oddity: CPUs are fast. What is Airbnb doing that takes so much compute? Perhaps server-side React is slow? On my own, admittedly low-traffic Node site, I use jade/pug for templating, and the initial template compile step can be expensive. But then I cache the compiled template and everything is too fast to care about thereafter.
I don't doubt that they had a problem, or how they fixed it, and I enjoyed this deep dive. Nonetheless, their reasoning about the underlying causes don't ring true for me.
- cynicalreason 8y agoserver-side react is absolutely slower than jade/pug for templating. there's a lot of work being done in the area in terms of caching but it is a bit more complex (not sure I grasp enough of it to explain it) than how jade/pug work. I'll read the article and come back on your other points :)
- cynicalreason 8y agoso, after reading the article, yeah, think he explained it a little bad. basically what he's saying in a single node thread node's parallel model doesn't magically fix compute requirements. that graphic of fn1 and fn2 and compute is confusing. personally never used clustering and the cluster module (we did that at haproxy lvl with containerised individual node processes), but I think that's exactly what he's saying, that the cluster module is the limitation (https://nodejs.org/api/cluster.html https://nodejs.org/api/cluster.html). not sure how they use it but it seems to imply requests being fully read in the cluster PID than passed to the workers