5 ms·

Which platform for serverless functions has 8 cores? I have a CPU intensive data deployment script (Node.js) that takes 12 hours on a single thread but can be c

Which platform for serverless functions has 8 cores? I have a CPU intensive data deployment script (Node.js) that takes 12 hours on a single thread but can be chopped up to take advantage of more cores. Our build server on ec2 has 2 cores so it's about 6 hours. It would be great to know if we could push the job into serverless and get it done a lot faster.

drderidder 9y ago
Time limits on serverless functions are 9 minutes on GCE and even less on AWS, so for long-running stateful tasks they're probably not suitable, unless you can divide the work into serial as well as parallel subsets.
- santoriv 9y ago
  Thanks for the reply. The task is not stateful. It's just pulling data from a db, performing some transforms, and then pushing it into elasticsearch. I suppose that I could slice the tasks up to be arbitrarily small though having a longer window would be helpful. I guess I will take a look at GCE. Thank you!
- WaxProlix 9y ago
  FWIW I think Azure's Functions are 10 minutes and I hear they're talking about upping it again.
mirko22 9y ago
How can a "serverless function" have cores if it doesn't have servers and "it is just code" :/
- XaspR8d 9y ago
  Because "server" started taking on the meaning of "configurable box" to people who were frustrated with configuration, so "serverless" means "unprovisioned/unconfigurable" machines. Now if we started talking about "computeless" architecture I'll be confused. (Though maybe that'll be the trendy name for serverless data sources/sinks in a few years...)
  - mirko22 9y ago
    > so "serverless" means "unprovisioned/unconfigurable" machines I am pretty sure in English serverless means no server and "unprovisioned/unconfigurable" machines means you didn't provision them and you cannot configure them. Even in analogical sense this makes no sense. Something i could relate to is something like "Pay as you use" or "configurationless servers". But that is just me, and if you think it is ok to randomly change the meaning of words that means me personally and randomly don't need to accept your new meaning (not giving out, just trying to explain my rationale) Downvote all you want, but please do point out where i am wrong.
    - abritinthebay 9y ago
      I get it, I do. But that ship sailed loooong ago
    - nathankunicki 9y ago
      The "serverless" in this context means that they may as well be invisible to you, since you don't need to care about them. In the case of Lambda, you upload your code somehow Amazon runs it on a spare server somewhere. Language is all about context. The meaning of words changes depending on it, even in plain English settings.
    - swsieber 9y ago
      You're not wrong. "Pay as you use" doesn't quite capture it, and "configurationless servers" isn't quite buzz-wordy (eg. short and memorable) to make marketable. Wide spread terminology suffers from the evolutionary pressures of marketing. Only the catchiest, most marketable terms propagate.
    - cwyers 9y ago
      > please do point out where i am wrong Because speech (and writing) can be figurative, not just literal. And because the term has reached wide adoption (at least, in the subset interested in discussing such things) and so not using the term makes conversation difficult and litigating the issue every time it's discussed adds no value to the discussion whatsoever.
- drderidder 9y ago
  Though the functions are serverless in the sense that you needn't allocate virtual infrastructure, the platform obviously utilizes CPU resources to execute your function. That platform underlying the serverless function is, as it happens, multi-core in GCE.
  - mirko22 9y ago
    > Though the functions are serverless in the sense that you needn't allocate virtual infrastructure, the platform obviously utilizes CPU resources to execute your function. So it does allocate virtual infrastructure?
    - ralmeida 9y ago
      Someone does, but not you. Obviously, there's a server somewhere which runs your code. But since you don't deal with it directly, it's mostly as if there was no server - since it was abstracted away form you so you could focus on what you really want to do. If you were to go down deeper, a server is just an electrical machine which shuffles eletrons around. So, you could say "there's no point in talking about 'servers', if we're just using transistors when you really think about it". But it would convey no useful information if someone asked you "what are you using to run your service?" and you replied "well, I just move electrons around", would it?
      - mirko22 9y ago
        No, i am not advocating reductionism, server has a analogical meaning as it serves something. Processing has analogical meaning as we do processes something, serverless implies there are no servers involved. As far as managing goes, most of the server are not managed by you anyway. But yeah i get it, everyone likes it so...
        ralmeida 9y ago
        On the management side, I'm curious what you mean by "most of the server are not managed by you anyway". In my experience, unless you're in a very big company, you are either managing or at least minding the server, or someone within earshot range is. In that scenario, not having to even consider if the error you're getting is because your server is restarting/out-of-memory/needing-update/broken-by-a-coworker like 99% of the time is as close to "serverless" as it gets, in terms of day-to-day work activities and worries.
- deleted 9y ago
  [deleted]
minxomat 9y ago
Not serverless, but we host our build servers on Scaleway. 8 cores and more importantly 32GB for 25$/m. Can't complain. Clean up is done by the ci agents from VSTS. Project is a React client and express backend. It's built and tested dockerized. We use testcafe and Chrome headless, so more memory is always useful for parallel builds.
vineet 9y ago
Different ec2 server types have different numbers of cores. You might be able to just change the server type to get some easy performance boost.
icahnvalyou 9y ago
Just use EC2's r4.16xlarge instance.

drderidder 9y ago

Time limits on serverless functions are 9 minutes on GCE and even less on AWS, so for long-running stateful tasks they're probably not suitable, unless you can divide the work into serial as well as parallel subsets.

santoriv 9y ago
Thanks for the reply. The task is not stateful. It's just pulling data from a db, performing some transforms, and then pushing it into elasticsearch. I suppose that I could slice the tasks up to be arbitrarily small though having a longer window would be helpful. I guess I will take a look at GCE. Thank you!
WaxProlix 9y ago
FWIW I think Azure's Functions are 10 minutes and I hear they're talking about upping it again.

mirko22 9y ago

How can a "serverless function" have cores if it doesn't have servers and "it is just code" :/

XaspR8d 9y ago
Because "server" started taking on the meaning of "configurable box" to people who were frustrated with configuration, so "serverless" means "unprovisioned/unconfigurable" machines. Now if we started talking about "computeless" architecture I'll be confused. (Though maybe that'll be the trendy name for serverless data sources/sinks in a few years...)
- mirko22 9y ago
  > so "serverless" means "unprovisioned/unconfigurable" machines I am pretty sure in English serverless means no server and "unprovisioned/unconfigurable" machines means you didn't provision them and you cannot configure them. Even in analogical sense this makes no sense. Something i could relate to is something like "Pay as you use" or "configurationless servers". But that is just me, and if you think it is ok to randomly change the meaning of words that means me personally and randomly don't need to accept your new meaning (not giving out, just trying to explain my rationale) Downvote all you want, but please do point out where i am wrong.
  - abritinthebay 9y ago
    I get it, I do. But that ship sailed loooong ago
  - nathankunicki 9y ago
    The "serverless" in this context means that they may as well be invisible to you, since you don't need to care about them. In the case of Lambda, you upload your code somehow Amazon runs it on a spare server somewhere. Language is all about context. The meaning of words changes depending on it, even in plain English settings.
  - swsieber 9y ago
    You're not wrong. "Pay as you use" doesn't quite capture it, and "configurationless servers" isn't quite buzz-wordy (eg. short and memorable) to make marketable. Wide spread terminology suffers from the evolutionary pressures of marketing. Only the catchiest, most marketable terms propagate.
  - cwyers 9y ago
    > please do point out where i am wrong Because speech (and writing) can be figurative, not just literal. And because the term has reached wide adoption (at least, in the subset interested in discussing such things) and so not using the term makes conversation difficult and litigating the issue every time it's discussed adds no value to the discussion whatsoever.
drderidder 9y ago
Though the functions are serverless in the sense that you needn't allocate virtual infrastructure, the platform obviously utilizes CPU resources to execute your function. That platform underlying the serverless function is, as it happens, multi-core in GCE.
- mirko22 9y ago
  > Though the functions are serverless in the sense that you needn't allocate virtual infrastructure, the platform obviously utilizes CPU resources to execute your function. So it does allocate virtual infrastructure?
  - ralmeida 9y ago
    Someone does, but not you. Obviously, there's a server somewhere which runs your code. But since you don't deal with it directly, it's mostly as if there was no server - since it was abstracted away form you so you could focus on what you really want to do. If you were to go down deeper, a server is just an electrical machine which shuffles eletrons around. So, you could say "there's no point in talking about 'servers', if we're just using transistors when you really think about it". But it would convey no useful information if someone asked you "what are you using to run your service?" and you replied "well, I just move electrons around", would it?
    - mirko22 9y ago
      No, i am not advocating reductionism, server has a analogical meaning as it serves something. Processing has analogical meaning as we do processes something, serverless implies there are no servers involved. As far as managing goes, most of the server are not managed by you anyway. But yeah i get it, everyone likes it so...
      - ralmeida 9y ago
        On the management side, I'm curious what you mean by "most of the server are not managed by you anyway". In my experience, unless you're in a very big company, you are either managing or at least minding the server, or someone within earshot range is. In that scenario, not having to even consider if the error you're getting is because your server is restarting/out-of-memory/needing-update/broken-by-a-coworker like 99% of the time is as close to "serverless" as it gets, in terms of day-to-day work activities and worries.
deleted 9y ago
[deleted]

minxomat 9y ago

Not serverless, but we host our build servers on Scaleway. 8 cores and more importantly 32GB for 25$/m. Can't complain. Clean up is done by the ci agents from VSTS. Project is a React client and express backend. It's built and tested dockerized. We use testcafe and Chrome headless, so more memory is always useful for parallel builds.

vineet 9y ago

Different ec2 server types have different numbers of cores. You might be able to just change the server type to get some easy performance boost.

icahnvalyou 9y ago

Just use EC2's r4.16xlarge instance.