6 ms·

> two full fat Node.js processes Do you know about copy on write?

zaarn 7y ago
Starting up a second Node.js process gives you no CoW advantages, that would occur if you fork off the original node process. But Multiprocess support in Node is piss-poor at best.
- spion 7y ago
  https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads.html https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads...
  - zaarn 7y ago
    Have you considered that workers in JS absolutely suck?
    - spion 7y ago
      Have considered you that suck conditions race more with threads and shared memory parallelism? If your counter-suggestion is Rust, have you considered.unwrap() that.unwrap() thinking.unwrap() about.unwrap() managing.unwrap() memory.unwrap() all the time might not be the wisest choice and that a GC is a useful abstraction to have?
      - zaarn 7y ago
        unwrap() can very very often be avoided in favor of a simple ? operation, the amount of nesting you suggest does occur... but not in a single function so it's spread over many lines and functions. There are GC libraries for Rust if you need them. Or Python/Lua bindings that have it too. Have you considered that avoiding race conditions isn't hard and you can't avoid having to avoid them even in JS?
        spion 7y ago
        "Race conditions of threads with shared memory parallelism" - yes you can avoid those in JS, the "yield" points are obvious in all code (await). Although its there if you need it (SharedArrayBuffer) Regarding whether its hard or not, yes, I have some first-hand (non-Rust) experience. On second thought I agree with your original point. Node isn't great for CPU-efficient + multithreaded. But * most platforms aren't (either not great because unsafe, non-cpu-efficient or both) * most of the time its not what you really need. So a good question would be, what would you use shared memory parallelism for in typical back-end programming?
        zaarn 7y ago
        I've developed plenty of shared memory software. In 99% of cases, a well placed mutex lock will solve the problem entirely. Of the remaining percent, maybe 5% are the type where you can just not lock and it'll work out sufficiently often. Another 45% might benefit from designing a channel/queue. The reamining 50% need a specialized lock to run well. But all shared memory problems can, without much difficulty, be solved by using a mutex. It might not be best performance but you'll likely not need that much performance in the first place (after all, you're coming from the JS ecosystem). Memory barriers aren't particularly hard to understand either, I've written some systems using them that ran very stable. Same for lockless or atomic or reentrant algorithms. It's all very fun and doesn't take that much skill if you're willing to read into it.
        spion 7y ago
        And plenty of people have said otherwise, and have developed entire languages and formal method frameworks to better reason about these problems. (e.g. TLA+) So anyway... what would you use shared memory parallelism for in typical back-end programming?
        zaarn 7y ago
        You use the advanced methods to squeeze out more performance, it is not necessary to actually build solutions. There is plenty of reasons for shared memory in back-ends, workers queues with zero-copy messaging would be one example that a lot of applications can benefit from.
- justinmeiners 7y ago
  > that would occur if you fork off the original node process. Right, so why wouldn't you? Node has a "fork" where you can give it a path to a script to run. Yeah it has to JIT it, but the V8 and Node part isn't duplicated.
  - zaarn 7y ago
    Because running fork() in Rust,C&Friends is more efficient? I don't have to reload the code image at all and I instantly share all data, including any configuration. Plus you can more easily do shmem in those languages than JS which gives you another performance advantage over pipes.

Starting up a second Node.js process gives you no CoW advantages, that would occur if you fork off the original node process. But Multiprocess support in Node is piss-poor at best.

spion 7y ago
https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads.html https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads...
- zaarn 7y ago
  Have you considered that workers in JS absolutely suck?
  - spion 7y ago
    Have considered you that suck conditions race more with threads and shared memory parallelism? If your counter-suggestion is Rust, have you considered.unwrap() that.unwrap() thinking.unwrap() about.unwrap() managing.unwrap() memory.unwrap() all the time might not be the wisest choice and that a GC is a useful abstraction to have?
    - zaarn 7y ago
      unwrap() can very very often be avoided in favor of a simple ? operation, the amount of nesting you suggest does occur... but not in a single function so it's spread over many lines and functions. There are GC libraries for Rust if you need them. Or Python/Lua bindings that have it too. Have you considered that avoiding race conditions isn't hard and you can't avoid having to avoid them even in JS?
      - spion 7y ago
        "Race conditions of threads with shared memory parallelism" - yes you can avoid those in JS, the "yield" points are obvious in all code (await). Although its there if you need it (SharedArrayBuffer) Regarding whether its hard or not, yes, I have some first-hand (non-Rust) experience. On second thought I agree with your original point. Node isn't great for CPU-efficient + multithreaded. But * most platforms aren't (either not great because unsafe, non-cpu-efficient or both) * most of the time its not what you really need. So a good question would be, what would you use shared memory parallelism for in typical back-end programming?
        zaarn 7y ago
        I've developed plenty of shared memory software. In 99% of cases, a well placed mutex lock will solve the problem entirely. Of the remaining percent, maybe 5% are the type where you can just not lock and it'll work out sufficiently often. Another 45% might benefit from designing a channel/queue. The reamining 50% need a specialized lock to run well. But all shared memory problems can, without much difficulty, be solved by using a mutex. It might not be best performance but you'll likely not need that much performance in the first place (after all, you're coming from the JS ecosystem). Memory barriers aren't particularly hard to understand either, I've written some systems using them that ran very stable. Same for lockless or atomic or reentrant algorithms. It's all very fun and doesn't take that much skill if you're willing to read into it.
        spion 7y ago
        And plenty of people have said otherwise, and have developed entire languages and formal method frameworks to better reason about these problems. (e.g. TLA+) So anyway... what would you use shared memory parallelism for in typical back-end programming?
        zaarn 7y ago
        You use the advanced methods to squeeze out more performance, it is not necessary to actually build solutions. There is plenty of reasons for shared memory in back-ends, workers queues with zero-copy messaging would be one example that a lot of applications can benefit from.
justinmeiners 7y ago
> that would occur if you fork off the original node process. Right, so why wouldn't you? Node has a "fork" where you can give it a path to a script to run. Yeah it has to JIT it, but the V8 and Node part isn't duplicated.
- zaarn 7y ago
  Because running fork() in Rust,C&Friends is more efficient? I don't have to reload the code image at all and I instantly share all data, including any configuration. Plus you can more easily do shmem in those languages than JS which gives you another performance advantage over pipes.