5 ms·

You can get rid of the need to multi threading by deploying more containers in the same machine or via orchestration. I don't understand why people keep insist

by bertolo1988 9y ago

You can get rid of the need to multi threading by deploying more containers in the same machine or via orchestration. I don't understand why people keep insisting that the lack of multi threading support is a Javascript problem when there are better and more scalable ways of using your machine resources.

CaveTech 9y ago
A container per thread sounds like the least efficient solution possible.
- stingraycharles 9y ago
  Assuming everything you do is non-blocking, this effectively means a container per core. Not ideal, but not too bad either.
  - stupidcar 9y ago
    And if it isn't non-blocking? What if I'm writing a game and want to run parts of the rendering, AI, physics, etc. in parallel? Do I create a bunch of containers for each subsystem of my game and have them all communicate via IPC? This doesn't sound like a recipe for good memory/performance characteristics.
    - stingraycharles 9y ago
      But we’re talking about javascript here, a language that is async about almost everything. I completely agree with what you’re saying for those other problems you describe. But for the typical node.js webapp, doing the one-container-per-core is perfectly fine.
- bertolo1988 9y ago
  Its not up to me to know how to manage processes or threads. That's the SO task, not mine. My app should be stateless and scale by replication. This is the most efficient solution in every possible way.
  - aspett 9y ago
    Sounds like your app should be.... functional.
- bcoates 9y ago
  A thread is just a process that shares memory; a container is just a process that has a different network/filesystem/etc than the rest of the processes. Containers don't cost meaningfully more than threads unless you create expensive unique resources for each one.
  - CaveTech 9y ago
    Network/Filesystem/Memory are not expensive resources? It's a lot of overhead. If you're going to claim that you can share memory between containers than arguably you don't have multiple containers but rather a single one. This is way more overhead than threads which can share all of those resources.
    - bcoates 9y ago
      Network namespaces, virtual ethernet interfaces, iptables rules, & union filesystems are all very cheap and have little to no overhead for normal use cases. N processes in 1 container isn't a perf win over N processes in N containers. Shared process memory isn't the easy memory-consumption win it sounds like, locking is hard to get right, potentially very destructive to the parallel performance that was the point of the whole exercise, and marries you to a single physical box. Even if you want to take advantage of shared-address-space shared memory you probably want to do it in a more principled way than fork() One-copy-per-thread and share-by-communicating both give you braindead simple scaling without dealing with that.
code-is-code 9y ago
I tested this with docker and could not observe a big performance-penalty on the cpu usage. Of corse you need more memory and the biggest thing is that you have to build your application in a way that you can do this later.
stephenr 9y ago
Why would you need containers to run multiple instances?
- bertolo1988 9y ago
  http://lmgtfy.com/?q=why+should+i+containerize+my+app http://lmgtfy.com/?q=why+should+i+containerize+my+app
chrisseaton 9y ago
> You can get rid of the need to multi threading by deploying more containers in the same machine or via orchestration. What if you have a large shared in-memory data structure that you want to update with lots of irregular translations in parallel? Like many graph problems? How are you going to do that with multiple containers? As in industry we just don't understand how to distribute that kind of problem effectively.
- maxpert 9y ago
  Shared data structure among multiple threads... this sounds utterly fimilar and evil! Redis is single-threaded, probably one of fastest,has data different structures, can handle high loads, code is easy to reason, something that just works. One of the reasons Node is successful is the simplicity of single threaded code. Way easier to reason, I would question the usage of Node if you are doing something CPU bound with it. You can use golang or C# with tasks for that.
  - Gonzih 9y ago
    So just because shared memory is hard you are ok with sacrificing performance and replacing memory access with io hops? That sounds like an overkill and not suitable for every task.
    - egeozcan 9y ago
      I like node.js and use it very often (had my first package reach over 100 stars, woohoo :) ) but I don't understand why it needs to be suitable for every task. If you really want to do something creative with the shared memory, I guess you could do that in a "native module" written in c++ or even Rust[1]. I'm not saying that it's not doable with JS, it's just that it's already been done (as in, has a solution that works). [1]: https://github.com/neon-bindings/neon https://github.com/neon-bindings/neon
      - PunchTornado 9y ago
        Why should i learn a new language for that? It's good to have as many options as possible in js and you take the one that fits you best.
Gonzih 9y ago
This is actually very similar to your idea. Each worker is executed in a separate v8 instance and napa provides a way to communicate between workers. A bit more efficient since you dont carry container runtime around.
- kbenson 9y ago
  So, Perl ithreads then? It works for specific types of work, but not necessarily as a low cost abstraction unless you pre-thread. In other words, it works well for some cases where threads are used, and horribly for others.
cdoxsey 9y ago
I think this has two major downsides: 1. Setting up containers and the communication between them is really complex - it also only makes sense for a deployed, long-running, server process. 2. Communication between containers is very expensive. You're never going to beat an in-process pointer to shared memory. Web workers would make a lot more sense for most javascript programs.
- bertolo1988 9y ago
  1 - How is REST complex? 2 - That is only true if the your bottleneck is on communication somehow. Usually it is not. And a huge advantage: You will write stateless and easy to scale apps that do not care how the machine resources are being handled.
  - cdoxsey 9y ago
    You need to run multiple applications side-by-side and they need to be able to talk to each other. With containers that means figuring out the network layer and service discovery. It also means accounting for each system failing in your own app, retries with exponential backoff, timeouts, logging errors, circuit breakers, plus all the exotic ways a network layer can fail - if your RPC protocol has arbitrary limits (message size, timeouts, etc) You will probably want to use kubernetes with istio, not raw docker. All very do-able, but definitely not simple. I agree that services make sense, but there's a level between single-threaded and micro-services where having concurrency within your application is useful.

CaveTech 9y ago

A container per thread sounds like the least efficient solution possible.

stingraycharles 9y ago
Assuming everything you do is non-blocking, this effectively means a container per core. Not ideal, but not too bad either.
- stupidcar 9y ago
  And if it isn't non-blocking? What if I'm writing a game and want to run parts of the rendering, AI, physics, etc. in parallel? Do I create a bunch of containers for each subsystem of my game and have them all communicate via IPC? This doesn't sound like a recipe for good memory/performance characteristics.
  - stingraycharles 9y ago
    But we’re talking about javascript here, a language that is async about almost everything. I completely agree with what you’re saying for those other problems you describe. But for the typical node.js webapp, doing the one-container-per-core is perfectly fine.
bertolo1988 9y ago
Its not up to me to know how to manage processes or threads. That's the SO task, not mine. My app should be stateless and scale by replication. This is the most efficient solution in every possible way.
- aspett 9y ago
  Sounds like your app should be.... functional.
bcoates 9y ago
A thread is just a process that shares memory; a container is just a process that has a different network/filesystem/etc than the rest of the processes. Containers don't cost meaningfully more than threads unless you create expensive unique resources for each one.
- CaveTech 9y ago
  Network/Filesystem/Memory are not expensive resources? It's a lot of overhead. If you're going to claim that you can share memory between containers than arguably you don't have multiple containers but rather a single one. This is way more overhead than threads which can share all of those resources.
  - bcoates 9y ago
    Network namespaces, virtual ethernet interfaces, iptables rules, & union filesystems are all very cheap and have little to no overhead for normal use cases. N processes in 1 container isn't a perf win over N processes in N containers. Shared process memory isn't the easy memory-consumption win it sounds like, locking is hard to get right, potentially very destructive to the parallel performance that was the point of the whole exercise, and marries you to a single physical box. Even if you want to take advantage of shared-address-space shared memory you probably want to do it in a more principled way than fork() One-copy-per-thread and share-by-communicating both give you braindead simple scaling without dealing with that.

code-is-code 9y ago

I tested this with docker and could not observe a big performance-penalty on the cpu usage. Of corse you need more memory and the biggest thing is that you have to build your application in a way that you can do this later.

stephenr 9y ago

Why would you need containers to run multiple instances?

bertolo1988 9y ago
http://lmgtfy.com/?q=why+should+i+containerize+my+app http://lmgtfy.com/?q=why+should+i+containerize+my+app

chrisseaton 9y ago

> You can get rid of the need to multi threading by deploying more containers in the same machine or via orchestration. What if you have a large shared in-memory data structure that you want to update with lots of irregular translations in parallel? Like many graph problems? How are you going to do that with multiple containers? As in industry we just don't understand how to distribute that kind of problem effectively.

maxpert 9y ago
Shared data structure among multiple threads... this sounds utterly fimilar and evil! Redis is single-threaded, probably one of fastest,has data different structures, can handle high loads, code is easy to reason, something that just works. One of the reasons Node is successful is the simplicity of single threaded code. Way easier to reason, I would question the usage of Node if you are doing something CPU bound with it. You can use golang or C# with tasks for that.
- Gonzih 9y ago
  So just because shared memory is hard you are ok with sacrificing performance and replacing memory access with io hops? That sounds like an overkill and not suitable for every task.
  - egeozcan 9y ago
    I like node.js and use it very often (had my first package reach over 100 stars, woohoo :) ) but I don't understand why it needs to be suitable for every task. If you really want to do something creative with the shared memory, I guess you could do that in a "native module" written in c++ or even Rust[1]. I'm not saying that it's not doable with JS, it's just that it's already been done (as in, has a solution that works). [1]: https://github.com/neon-bindings/neon https://github.com/neon-bindings/neon
    - PunchTornado 9y ago
      Why should i learn a new language for that? It's good to have as many options as possible in js and you take the one that fits you best.

Gonzih 9y ago

This is actually very similar to your idea. Each worker is executed in a separate v8 instance and napa provides a way to communicate between workers. A bit more efficient since you dont carry container runtime around.

kbenson 9y ago
So, Perl ithreads then? It works for specific types of work, but not necessarily as a low cost abstraction unless you pre-thread. In other words, it works well for some cases where threads are used, and horribly for others.

cdoxsey 9y ago

I think this has two major downsides: 1. Setting up containers and the communication between them is really complex - it also only makes sense for a deployed, long-running, server process. 2. Communication between containers is very expensive. You're never going to beat an in-process pointer to shared memory. Web workers would make a lot more sense for most javascript programs.

bertolo1988 9y ago
1 - How is REST complex? 2 - That is only true if the your bottleneck is on communication somehow. Usually it is not. And a huge advantage: You will write stateless and easy to scale apps that do not care how the machine resources are being handled.
- cdoxsey 9y ago
  You need to run multiple applications side-by-side and they need to be able to talk to each other. With containers that means figuring out the network layer and service discovery. It also means accounting for each system failing in your own app, retries with exponential backoff, timeouts, logging errors, circuit breakers, plus all the exotic ways a network layer can fail - if your RPC protocol has arbitrary limits (message size, timeouts, etc) You will probably want to use kubernetes with istio, not raw docker. All very do-able, but definitely not simple. I agree that services make sense, but there's a level between single-threaded and micro-services where having concurrency within your application is useful.