4 ms·
Lambda has a long way to go, it's a promising product but if you want to develop anything for it, it's a bit painful and archaic. I've had issues with the temp
by doc_holliday 9y ago
Lambda has a long way to go, it's a promising product but if you want to develop anything for it, it's a bit painful and archaic.
I've had issues with the temp memory. It is completely unreliable, you have to store files when processing in /tmp/ and for whatever reason, reading from this results is patchy. I had to resort to storing every file I process in the RAM.
The tooling is only just taking off now, 6 months ago simulation was painful, at least there is AWS SAM now and docker support.
However despite the pain, it's giving me a solution for scalable video encoding that is very cost competitive and I don't have to worry about submitting jobs and monitoring resources.
- davidgf 9y agoAWS SAM is ok, but I encourage you to take a look at Serverless framework https://github.com/serverless/serverless https://github.com/serverless/serverless, which eases a lot managing the whole infrastructure. There are already some applications running completely on Lambda + Serverless, such as this one https://github.com/microapps/MoonMail https://github.com/microapps/MoonMail
- doc_holliday 9y agoThanks, serverless looks interesting.
- jackpeterfletch 9y agoIts not really for stateful services though right?
- doc_holliday 9y agoVideo encoding would be stateless. There is no reliance on previous runs / state. Lambda is designed to be used for tasks like that, such as resizing images, parsing file etc.
- jackpeterfletch 9y agoIn a semantic sense it is stateless. But what I mean is that its clearly not designed for you to be dropping files onto the local disk. Video encoding is kind of a corner case in that the size of the data really stops you from doing the whole thing in memory, though the persistance isn't needed in between runs. Probably what you want to look into is attaching an EBS volume to the job, and doing the work from there.