3 ms·
If your target workflows are ad hoc jobs, wouldn’t running on lambda make more sense than dealing with the headache of provisioning ec2?
by by_the_bay 4y ago
If your target workflows are ad hoc jobs, wouldn’t running on lambda make more sense than dealing with the headache of provisioning ec2?
- hrichardlee1 4y agoYes! (in some cases) The tradeoffs are that lambdas are way faster to start up, but they max out at 10GB of memory, 15 minutes of runtime, you can't use GPUs, and you get 250MB to store your code (you get more if you build a container) and 512MB of temp space. We're in the middle of working on adding this as an option--we want to make it seamless to switch between running your code on Lambda (for short, quick jobs) vs EC2 (for longer, more resource-intensive jobs) depending on what kind of workload you have.
- by_the_bay 4y agoCool, look forward to trying it out
- r00tanon 4y agoGood news. We've found lambdas orchestrated with step functions and properly split out as to function can do a lot of data processing even with the limits you stated before needing to switch over to AWS batch or ECS. Nice to have that tool in the belt. I'll check it out.
- yunohn 4y agoTheir landing page claims that Lambda support is planned. But I think there’s scope for both, data jobs needs ec2.