4 ms·

Let’s say you have a run of the mill nodejs express app that connects to Postgres (let’s assume it’s managed in this scenario). How do you make use of all the

Let’s say you have a run of the mill nodejs express app that connects to Postgres (let’s assume it’s managed in this scenario). How do you make use of all the 48 cores and the RAM available to you? (I’m genuinely interested so please ignore any snarky tone if any)

louis-lau 5y ago
You'd either run multiple instances of your app with an http load balancer in front, or make use of nodejs clustering: https://nodejs.org/api/cluster.html https://nodejs.org/api/cluster.html
deleted 5y ago
[deleted]
juanse 5y ago
I will start by saying that I'm super average and still learning, but it is my understanding that you use RAM to keep things on memory (faster access), and cores for parallelization (different instances reads). In my case, I use Ruby on Rails, so I configure Puma based on the number of cores that I have to set the workers (threats). Not yet, but I will use Redis for caching. It is also an "in-memory" database.
- j45 5y ago
  Using more resources than average also comes into play when using less performance (but rapid to develop with) technologies. Best practices are a funny thing, understanding where they come from and who they were intended for an go a long way in having a better context.
lostcolony 5y ago
There's two parts to this. The first is the broad question of "how will any app (regardless of the technology) saturate that much hardware", and to that I simply say it won't; it will saturate -something- first (with enough load), be it RAM, CPU, network I/O, disk I/O, etc, which will limit utilization of the rest. If this was your own datacenter you might try and "right size" to that, to try and have as little unused hardware capacity as possible, but in the cloud you don't (can't) do that. The second is maybe more what you're trying to ask, which is given that node is single threaded, you would only be using one core, and that would implicitly also limit how much of the other resources you can use. To that, you'd use something like PM2, a node native utility which essentially spins up multiple instances of your app locally, and load balances across them, or the cluster lib, a less feature rich solution to the same problem, but supplied as part of the standard library.
mirker 5y ago
I don’t know about nodejs, but serving a twitter-like feed with an in-memory database of everyone’s tweets is one use-case. It’s a database query workload and would benefit from caching raw data or views of the twitter feed and has a latency requirement of 100 milliseconds or so.
CodeWriter23 5y ago
> How do you make use of all the 48 cores and the RAM available to you? The point I take from gp’s comment is, there’s a lot of room to scale within a single instance when the nano-sized VM you started with is overwhelmed. Focus on the things that cause growth first, then add complexities of distributed computing when it is actually needed.
HelloNurse 5y ago
Most likely answer: the application is small and there's no need, and no useful way, to extract value many cores or a lot of memory. So don't waste effort on something that isn't needed. Boring answer: use a pool of threads (or subprocesses, like Apache) to deal with web requests. Without stupid locking, many concurrent requests can be handled for each core. If you don't know your QoS targets, how many threads you need to meet them, and how much memory each thread requires it's a serious red flag. Pragmatic answer: do you want to invest in fault tolerance and being able to upgrade your application? Then you need a load balancer to route traffic to multiple instances and switch them on and off properly, and a good load balancer is going to be outside your VM and probably a specific piece of hardware, irrespective of clever techniques in your application.
dehrmann 5y ago
If you know you're going to want this type of parallelism and simple deployments, use languages that have better threading stories. That said, one place I worked had a legacy Python service on 16+ core machines. We happened to deploy with dpkg, apt, and docker (I wouldn't necessarily recommend this), and the start script spun up cpu count - 1 instances with CPU affinity set. There was something like nginx for routing out in front.
dzonga 5y ago
depending on your node.js version. nodejs has a cluster mode. before that tools like pm2 had cluster | loadbalancing modes. But yeah you can easily run as many instances of the node.js app on your 60 virtual cores. Let node use half of those. Then the rest for db, monitoring. Then have a load balance ie nginx. --Edit also node.js now has Workers e.g for instances where you need to do CPU bound work. Remember node.js is really fast. most apps out there won't be able to use it to the max.

louis-lau 5y ago

You'd either run multiple instances of your app with an http load balancer in front, or make use of nodejs clustering: https://nodejs.org/api/cluster.html https://nodejs.org/api/cluster.html

deleted 5y ago

[deleted]

juanse 5y ago

I will start by saying that I'm super average and still learning, but it is my understanding that you use RAM to keep things on memory (faster access), and cores for parallelization (different instances reads). In my case, I use Ruby on Rails, so I configure Puma based on the number of cores that I have to set the workers (threats). Not yet, but I will use Redis for caching. It is also an "in-memory" database.

j45 5y ago
Using more resources than average also comes into play when using less performance (but rapid to develop with) technologies. Best practices are a funny thing, understanding where they come from and who they were intended for an go a long way in having a better context.

lostcolony 5y ago

There's two parts to this. The first is the broad question of "how will any app (regardless of the technology) saturate that much hardware", and to that I simply say it won't; it will saturate -something- first (with enough load), be it RAM, CPU, network I/O, disk I/O, etc, which will limit utilization of the rest. If this was your own datacenter you might try and "right size" to that, to try and have as little unused hardware capacity as possible, but in the cloud you don't (can't) do that. The second is maybe more what you're trying to ask, which is given that node is single threaded, you would only be using one core, and that would implicitly also limit how much of the other resources you can use. To that, you'd use something like PM2, a node native utility which essentially spins up multiple instances of your app locally, and load balances across them, or the cluster lib, a less feature rich solution to the same problem, but supplied as part of the standard library.

mirker 5y ago

I don’t know about nodejs, but serving a twitter-like feed with an in-memory database of everyone’s tweets is one use-case. It’s a database query workload and would benefit from caching raw data or views of the twitter feed and has a latency requirement of 100 milliseconds or so.

CodeWriter23 5y ago

> How do you make use of all the 48 cores and the RAM available to you? The point I take from gp’s comment is, there’s a lot of room to scale within a single instance when the nano-sized VM you started with is overwhelmed. Focus on the things that cause growth first, then add complexities of distributed computing when it is actually needed.

HelloNurse 5y ago

Most likely answer: the application is small and there's no need, and no useful way, to extract value many cores or a lot of memory. So don't waste effort on something that isn't needed. Boring answer: use a pool of threads (or subprocesses, like Apache) to deal with web requests. Without stupid locking, many concurrent requests can be handled for each core. If you don't know your QoS targets, how many threads you need to meet them, and how much memory each thread requires it's a serious red flag. Pragmatic answer: do you want to invest in fault tolerance and being able to upgrade your application? Then you need a load balancer to route traffic to multiple instances and switch them on and off properly, and a good load balancer is going to be outside your VM and probably a specific piece of hardware, irrespective of clever techniques in your application.

dehrmann 5y ago

If you know you're going to want this type of parallelism and simple deployments, use languages that have better threading stories. That said, one place I worked had a legacy Python service on 16+ core machines. We happened to deploy with dpkg, apt, and docker (I wouldn't necessarily recommend this), and the start script spun up cpu count - 1 instances with CPU affinity set. There was something like nginx for routing out in front.

dzonga 5y ago

depending on your node.js version. nodejs has a cluster mode. before that tools like pm2 had cluster | loadbalancing modes. But yeah you can easily run as many instances of the node.js app on your 60 virtual cores. Let node use half of those. Then the rest for db, monitoring. Then have a load balance ie nginx. --Edit also node.js now has Workers e.g for instances where you need to do CPU bound work. Remember node.js is really fast. most apps out there won't be able to use it to the max.