I worked on something like this for a small video rendering startup a few years ago. Our backend needed to ingest short (<= 12s) videos and strip the parts we

I worked on something like this for a small video rendering startup a few years ago. Our backend needed to ingest short (<= 12s) videos and strip the parts we needed in our rendering stack. We polled S3 (SNS wasn't consistently fast enough at the time) for new files in our Upload directory. The uploaded video was sent to multiple microservices in a group of tens or hundreds of instances. One would "normalize" the video so we had the same frame-rate, size, and bitrates to work with. Another would rip the audio into a couple formats. Another would generate HLS streams with a few resolutions. Another would generate the HLS playlist. Another would split the video up into images. Each service uploaded its results to S3, named as a SHA1 hash of itself, and reported to a single API which kept track of the "tree" of files generated from each upload. The system was based upon an early version of NodeJS, which worked fantastically well as a generic daemonized wrapper around the command line tools we were using for all the ripping and modifying. About a year before I left the team we changed to GCS for the upload server because they had reliable immediate notifications to webhooks for uploads, but kept the rest of the system on S3, since our processing instances were all EC2. Would have loved to have Lambda back then! We had a similar set of microservices for rendering the eventual videos in near-real-time - but that's a whole different conversation.