3 ms·
Sure: you likely don't want to be dealing with large files on Lambda. Why? * The maximum timeout window for a function invocation is 300 seconds * The maximum
by djm_ 11y ago
Sure: you likely don't want to be dealing with large files on Lambda. Why?
* The maximum timeout window for a function invocation is 300 seconds
* The maximum available temp disk space per instance is 500MB
* Memory is maxed at 1.5GB
In the function invocation time window you need to:
* retrieve the file (to memory or disk)
* transcode the file (outputting to memory or disk)
* upload the file (as the disk is not persistent)
This along with the following facts make it infeasible:
* transfers from S3 are fast, but non-s3 sources are likely to be much slower.
* assuming you mean large as in GB - you have nowhere to put the files (disk too small, memory too small).
* transfers to S3 are fast, but uploading the transcoded video to a non-s3 source will likely me much slower.
Hope that helps.
- cosmie 11y agoI don't know much about video transcoding, but if FFMPEG can utilize streams it's easy to work around the lambda size constraints. You can process several GBs in the 5 minute window by piping your S3 download stream through your transformation steps then directly into an S3 upload stream. Nothing ever persists to disk, so your only worry if anything is managing your stream buffers so you don't run out of memory. As long as any single step of your pipeline doesn't exceed the time limit, you can make really nifty pipelines for large file processing by using the S3 upload as "temp space" then an S3 event to automatically trigger the next step of your pipeline.
- jdp 11y agoffmpeg can utilize streams, in both input and output. The trouble comes from different codecs and containers, especially on output. Some formats aren't append-only—the prime example being MP4 + h.264—and so ffmpeg needs to be able to write to a seekable output device, ruling out streaming output in those cases.