3 ms·

The lede is kind of buried. The authors were attempting to use FaaS for a long-running, number crunching, distributed computing algorithm. Case of selecting th

The lede is kind of buried. The authors were attempting to use FaaS for a long-running, number crunching, distributed computing algorithm. Case of selecting the wrong tool for the wrong job. Not what serverless was designed to do.

NotAnEconomist 8y ago
It's really just that they selected the wrong serverless architecture. Code repo -> docker build -> AWS Fargate. Easy to set up CI/CD pipeline that doesn't require you manage servers and can support long-running, CPU intensive, and even GPU applications. (You may have to manage the fleet a bit for GPU support, ie create your own ECS cluster with autoscaling -- but not actively manage the deployment of instances.)
munns 8y ago
+1 on this. The paper doesn't really address the most typical usecases of Serverless which are typically API backends, streaming data processing (feeding off Kinesis/Kafka/similar), and reactions to one off-ish events like object uploads to storage or events from other systems. While folks like those building Pywren have found success with highly parallel workloads, FaaS still isn't the typical way people go after distributed computing workloads, and no one from the vendor side is pushing this. - munns - Lead Dev Advocate - AWS Serverless
twtw 8y ago
But the question is could it be made to be good for that domain? See https://arxiv.org/abs/1702.04024 https://arxiv.org/abs/1702.04024 [cited by this paper] for promising results doing linear algebra using aws lambda.
deleted 8y ago
[deleted]
djhworld 8y ago
> Not what serverless was designed to do. I think the point of the piece was more "what serverless could be" rather than what we have right now. I don't think there's any concrete definition of what serverless is, let alone what its designed to do! The thing the article brushes over is the maintenance/operational burden of running stuff on VMs. It says they achieved much better performance training a machine learning model on a single EC2 instanec, but failed to mention setting up monitoring, logging, deploying code, making that reliable etc. That's the trade off you make

NotAnEconomist 8y ago

It's really just that they selected the wrong serverless architecture. Code repo -> docker build -> AWS Fargate. Easy to set up CI/CD pipeline that doesn't require you manage servers and can support long-running, CPU intensive, and even GPU applications. (You may have to manage the fleet a bit for GPU support, ie create your own ECS cluster with autoscaling -- but not actively manage the deployment of instances.)

munns 8y ago

+1 on this. The paper doesn't really address the most typical usecases of Serverless which are typically API backends, streaming data processing (feeding off Kinesis/Kafka/similar), and reactions to one off-ish events like object uploads to storage or events from other systems. While folks like those building Pywren have found success with highly parallel workloads, FaaS still isn't the typical way people go after distributed computing workloads, and no one from the vendor side is pushing this. - munns - Lead Dev Advocate - AWS Serverless

twtw 8y ago

But the question is could it be made to be good for that domain? See https://arxiv.org/abs/1702.04024 https://arxiv.org/abs/1702.04024 [cited by this paper] for promising results doing linear algebra using aws lambda.

deleted 8y ago

[deleted]

djhworld 8y ago

> Not what serverless was designed to do. I think the point of the piece was more "what serverless could be" rather than what we have right now. I don't think there's any concrete definition of what serverless is, let alone what its designed to do! The thing the article brushes over is the maintenance/operational burden of running stuff on VMs. It says they achieved much better performance training a machine learning model on a single EC2 instanec, but failed to mention setting up monitoring, logging, deploying code, making that reliable etc. That's the trade off you make