3 ms·
> Splitting thousands of hours of video into 1-2s chunks, uploading them to s3, then processing them in isolation? Now that you place it like that, it does sou
by rewmie 3y ago
> Splitting thousands of hours of video into 1-2s chunks, uploading them to s3, then processing them in isolation?
Now that you place it like that, it does sound very dubious. The overhead required by uploading MBs go S3 go afterwards have to download it back to the lamdda's execution context should have the same order of magnitude as processing the payload. I'm sure they were aware of this but we're hoping that things would scale well due to the embarrassingly parallel nature of the problem.