13 ms·

This looks great, but it's important to bear in mind the architecture you intend to run on. I recently made an application blazingly fast by - among other thing

This looks great, but it's important to bear in mind the architecture you intend to run on. I recently made an application blazingly fast by - among other things - parallelization using the node cluster module. On my 4-core laptop it flies. Imagine my surprise when I deployed to the cloud environment and found the typical virtual server in our cluster has only a single CPU core. The worker threads just sit around waiting for their turn to run, one at a time. On the other hand, the platform for serverless functions has 8 cores. At a minumum, before you jump into multi-threading, know what `require('os').cpus()` tells you on your target platform.

santoriv 9y ago
Which platform for serverless functions has 8 cores? I have a CPU intensive data deployment script (Node.js) that takes 12 hours on a single thread but can be chopped up to take advantage of more cores. Our build server on ec2 has 2 cores so it's about 6 hours. It would be great to know if we could push the job into serverless and get it done a lot faster.
- drderidder 9y ago
  Time limits on serverless functions are 9 minutes on GCE and even less on AWS, so for long-running stateful tasks they're probably not suitable, unless you can divide the work into serial as well as parallel subsets.
  - santoriv 9y ago
    Thanks for the reply. The task is not stateful. It's just pulling data from a db, performing some transforms, and then pushing it into elasticsearch. I suppose that I could slice the tasks up to be arbitrarily small though having a longer window would be helpful. I guess I will take a look at GCE. Thank you!
  - WaxProlix 9y ago
    FWIW I think Azure's Functions are 10 minutes and I hear they're talking about upping it again.
- mirko22 9y ago
  How can a "serverless function" have cores if it doesn't have servers and "it is just code" :/
  - XaspR8d 9y ago
    Because "server" started taking on the meaning of "configurable box" to people who were frustrated with configuration, so "serverless" means "unprovisioned/unconfigurable" machines. Now if we started talking about "computeless" architecture I'll be confused. (Though maybe that'll be the trendy name for serverless data sources/sinks in a few years...)
    - mirko22 9y ago
      > so "serverless" means "unprovisioned/unconfigurable" machines I am pretty sure in English serverless means no server and "unprovisioned/unconfigurable" machines means you didn't provision them and you cannot configure them. Even in analogical sense this makes no sense. Something i could relate to is something like "Pay as you use" or "configurationless servers". But that is just me, and if you think it is ok to randomly change the meaning of words that means me personally and randomly don't need to accept your new meaning (not giving out, just trying to explain my rationale) Downvote all you want, but please do point out where i am wrong.
      - abritinthebay 9y ago
        I get it, I do. But that ship sailed loooong ago
      - nathankunicki 9y ago
        The "serverless" in this context means that they may as well be invisible to you, since you don't need to care about them. In the case of Lambda, you upload your code somehow Amazon runs it on a spare server somewhere. Language is all about context. The meaning of words changes depending on it, even in plain English settings.
      - swsieber 9y ago
- minxomat 9y ago
  Not serverless, but we host our build servers on Scaleway. 8 cores and more importantly 32GB for 25$/m. Can't complain. Clean up is done by the ci agents from VSTS. Project is a React client and express backend. It's built and tested dockerized. We use testcafe and Chrome headless, so more memory is always useful for parallel builds.
- vineet 9y ago
  Different ec2 server types have different numbers of cores. You might be able to just change the server type to get some easy performance boost.
- icahnvalyou 9y ago
  Just use EC2's r4.16xlarge instance.
thejosh 9y ago
This is also a painpoint for me, I have an app that doesn't use much memory, but can use multiple cores/threads.. but most providers sell you on RAM, not CPU.
- drderidder 9y ago
  Also many providers charge by CPU usage, so it behooves customers to spec machines as lightly as possible and fall back on auto-scaling to spin up additional, equally lightly-spec'd VMs, on demand. Rarely are there many idle cores just sitting around.
paulddraper 9y ago
The distinguishing factor between multi-processing on one machine and multi-"machining" is latency of communication. Likewise, the distinguising factor between multi-threading and multi-processing is shared memory, i.e. again, the speed of communication. Multi-threading does well for some problems, but often, multi-machining or multi-processing is sufficient, which is why so many runtimes don't really do multi-threading: Node.JS, Python, Ruby.
- drderidder 9y ago
  Very true, and we've seen better performance from multi-process than multi-threading in some high-traffic (10Gbps) systems (those were C++ however, not node). It was puzzling, but I put it down to the OS scheduler; I imagine that a single multi-threaded process would be pre-empted in the typical fashion, but that multiple single-threaded processes would, in aggregate, spend less time in a pre-empted state.
  - datavirtue 9y ago
    Context switching is expensive and very tempting to forget about in multi-threaded development. You eliminate that overhead constraining the execution code to a single thread context.
  - immutable_ai 9y ago
    Yes, multi-threading has high overhead to provide the illusion of parallel operation, but when all the cores are saturated you are in the same boat, whether you have 1 or 100. The benefit to programs that don't use threading and use event loop and shared nothing multi-process is that they don't have the overhead when things are maxed out. This is why virtually every high performance server (nginx, redis, memcached, etc) is written this way and things like varnish (thread per request) are multiples or orders of magnitude slower. Funny people criticizing nodejs for using the same architecture that all the best-in-class products use.
- cjbillington 9y ago
  Note that Python does do multithreading - just the actual Python interpreter will not interpret Python code from two threads at once. If threads are busy running code that doesn't require the interpreter, whether it is CPU bound work or not, then as many threads can run as you have cores. In most CPU bound applications, the performance critical parts are not pure Python - they're calling out to some extension code such as numpy or cython or something. Such code runs in parallel in CPython if you start multiple threads. This might have been what you meant when you said the runtime isn't really multi-threaded - but since the CPython ecosystem rests so heavily on C code, in practice multithreading is a good solution a lot of the time.
chrisweekly 9y ago
good point. I’ve used pm2 to manage node apps and like how clear it makes what’s running on what.
WhitneyLand 9y ago
Great reminder of just how expensive of cloud compute cycles are. Imagine buying a new desktop computer, not the most expensive but a good performing one, and setting it up at home to serve some kind of cloud services. With its cpu fully utilized 24x7, I bet buying equivalent compute at AWS would be crazy expensive per month. Of course there are many reasons most people don’t use desktop systems as home servers, but I bet there are a few scenarios where it could payoff. One example might be a bootstrapped startup tight on cash flow, with cpu biased workload, so ISP bandwidth and local disk throughput didn’t bottleneck before the compute did. And it couldn’t be a mission critical service, something where some maintenance windows wouldn’t kill your reputation. Finally you’d need a way to really minimize admin/devops costs. That kind of work doesn’t take many hours to kill your savings not to mention opportunity costs.
- brianwawok 9y ago
  I use a laptop for my CI server. 8 cores and 16gb ram, hides in my basement. Would easily be $400 a month for those specs.
  - Sohcahtoa82 9y ago
    An m4.2xlarge on AWS EC2 provides 8 cores and 32 gig of RAM for 40 cents/hour. That'd be $288/month. You could also get an m4.xlarge with 4 cores and 16 gig for $144/month. Of course, that doesn't include storage and bandwidth. I mean, sure. I've got a 5 year old laptop that will outperform the t2.micro I'm pay $8.35/month for. But I don't trust my home internet to be stable or fast enough. Not to mention that my primary usage is an IRC bouncer, so I need it to not be on my home internet connection so some script kiddie doesn't DDoS me after I ban them from a channel because they were spamming racial slurs. Yes, that has actually happened.
    - speedplane 9y ago
      In the US you can't really get a reliable network connection to your residence. The entire shift towards the cloud is in no small part due to crappy internet. The large ISPs really missed the boat on this.

santoriv 9y ago

Which platform for serverless functions has 8 cores? I have a CPU intensive data deployment script (Node.js) that takes 12 hours on a single thread but can be chopped up to take advantage of more cores. Our build server on ec2 has 2 cores so it's about 6 hours. It would be great to know if we could push the job into serverless and get it done a lot faster.

drderidder 9y ago
Time limits on serverless functions are 9 minutes on GCE and even less on AWS, so for long-running stateful tasks they're probably not suitable, unless you can divide the work into serial as well as parallel subsets.
- santoriv 9y ago
  Thanks for the reply. The task is not stateful. It's just pulling data from a db, performing some transforms, and then pushing it into elasticsearch. I suppose that I could slice the tasks up to be arbitrarily small though having a longer window would be helpful. I guess I will take a look at GCE. Thank you!
- WaxProlix 9y ago
  FWIW I think Azure's Functions are 10 minutes and I hear they're talking about upping it again.
mirko22 9y ago
How can a "serverless function" have cores if it doesn't have servers and "it is just code" :/
- XaspR8d 9y ago
  Because "server" started taking on the meaning of "configurable box" to people who were frustrated with configuration, so "serverless" means "unprovisioned/unconfigurable" machines. Now if we started talking about "computeless" architecture I'll be confused. (Though maybe that'll be the trendy name for serverless data sources/sinks in a few years...)
  - mirko22 9y ago
    > so "serverless" means "unprovisioned/unconfigurable" machines I am pretty sure in English serverless means no server and "unprovisioned/unconfigurable" machines means you didn't provision them and you cannot configure them. Even in analogical sense this makes no sense. Something i could relate to is something like "Pay as you use" or "configurationless servers". But that is just me, and if you think it is ok to randomly change the meaning of words that means me personally and randomly don't need to accept your new meaning (not giving out, just trying to explain my rationale) Downvote all you want, but please do point out where i am wrong.
    - abritinthebay 9y ago
      I get it, I do. But that ship sailed loooong ago
    - nathankunicki 9y ago
      The "serverless" in this context means that they may as well be invisible to you, since you don't need to care about them. In the case of Lambda, you upload your code somehow Amazon runs it on a spare server somewhere. Language is all about context. The meaning of words changes depending on it, even in plain English settings.
    - swsieber 9y ago
minxomat 9y ago
Not serverless, but we host our build servers on Scaleway. 8 cores and more importantly 32GB for 25$/m. Can't complain. Clean up is done by the ci agents from VSTS. Project is a React client and express backend. It's built and tested dockerized. We use testcafe and Chrome headless, so more memory is always useful for parallel builds.
vineet 9y ago
Different ec2 server types have different numbers of cores. You might be able to just change the server type to get some easy performance boost.
icahnvalyou 9y ago
Just use EC2's r4.16xlarge instance.

thejosh 9y ago

This is also a painpoint for me, I have an app that doesn't use much memory, but can use multiple cores/threads.. but most providers sell you on RAM, not CPU.

drderidder 9y ago
Also many providers charge by CPU usage, so it behooves customers to spec machines as lightly as possible and fall back on auto-scaling to spin up additional, equally lightly-spec'd VMs, on demand. Rarely are there many idle cores just sitting around.

paulddraper 9y ago

The distinguishing factor between multi-processing on one machine and multi-"machining" is latency of communication. Likewise, the distinguising factor between multi-threading and multi-processing is shared memory, i.e. again, the speed of communication. Multi-threading does well for some problems, but often, multi-machining or multi-processing is sufficient, which is why so many runtimes don't really do multi-threading: Node.JS, Python, Ruby.

drderidder 9y ago
Very true, and we've seen better performance from multi-process than multi-threading in some high-traffic (10Gbps) systems (those were C++ however, not node). It was puzzling, but I put it down to the OS scheduler; I imagine that a single multi-threaded process would be pre-empted in the typical fashion, but that multiple single-threaded processes would, in aggregate, spend less time in a pre-empted state.
- datavirtue 9y ago
  Context switching is expensive and very tempting to forget about in multi-threaded development. You eliminate that overhead constraining the execution code to a single thread context.
- immutable_ai 9y ago
  Yes, multi-threading has high overhead to provide the illusion of parallel operation, but when all the cores are saturated you are in the same boat, whether you have 1 or 100. The benefit to programs that don't use threading and use event loop and shared nothing multi-process is that they don't have the overhead when things are maxed out. This is why virtually every high performance server (nginx, redis, memcached, etc) is written this way and things like varnish (thread per request) are multiples or orders of magnitude slower. Funny people criticizing nodejs for using the same architecture that all the best-in-class products use.
cjbillington 9y ago
Note that Python does do multithreading - just the actual Python interpreter will not interpret Python code from two threads at once. If threads are busy running code that doesn't require the interpreter, whether it is CPU bound work or not, then as many threads can run as you have cores. In most CPU bound applications, the performance critical parts are not pure Python - they're calling out to some extension code such as numpy or cython or something. Such code runs in parallel in CPython if you start multiple threads. This might have been what you meant when you said the runtime isn't really multi-threaded - but since the CPython ecosystem rests so heavily on C code, in practice multithreading is a good solution a lot of the time.

chrisweekly 9y ago

good point. I’ve used pm2 to manage node apps and like how clear it makes what’s running on what.

WhitneyLand 9y ago

Great reminder of just how expensive of cloud compute cycles are. Imagine buying a new desktop computer, not the most expensive but a good performing one, and setting it up at home to serve some kind of cloud services. With its cpu fully utilized 24x7, I bet buying equivalent compute at AWS would be crazy expensive per month. Of course there are many reasons most people don’t use desktop systems as home servers, but I bet there are a few scenarios where it could payoff. One example might be a bootstrapped startup tight on cash flow, with cpu biased workload, so ISP bandwidth and local disk throughput didn’t bottleneck before the compute did. And it couldn’t be a mission critical service, something where some maintenance windows wouldn’t kill your reputation. Finally you’d need a way to really minimize admin/devops costs. That kind of work doesn’t take many hours to kill your savings not to mention opportunity costs.

brianwawok 9y ago
I use a laptop for my CI server. 8 cores and 16gb ram, hides in my basement. Would easily be $400 a month for those specs.
- Sohcahtoa82 9y ago
  An m4.2xlarge on AWS EC2 provides 8 cores and 32 gig of RAM for 40 cents/hour. That'd be $288/month. You could also get an m4.xlarge with 4 cores and 16 gig for $144/month. Of course, that doesn't include storage and bandwidth. I mean, sure. I've got a 5 year old laptop that will outperform the t2.micro I'm pay $8.35/month for. But I don't trust my home internet to be stable or fast enough. Not to mention that my primary usage is an IRC bouncer, so I need it to not be on my home internet connection so some script kiddie doesn't DDoS me after I ban them from a channel because they were spamming racial slurs. Yes, that has actually happened.
  - speedplane 9y ago
    In the US you can't really get a reliable network connection to your residence. The entire shift towards the cloud is in no small part due to crappy internet. The large ISPs really missed the boat on this.