4 ms·
Fuck K8. You literally don't need it. Maybe he needs it because he's building on google cloud. AWS is easier, but you can do it with anything. The basic steps
by mannyv 3y ago
Fuck K8. You literally don't need it. Maybe he needs it because he's building on google cloud.
AWS is easier, but you can do it with anything. The basic steps are:
1. Upload the file somewhere
2. Transcode it
3. Put the parts somewhere
4. Serve the parts
You should really transcode everything into HLS. It's 2023, and everything that matters supports it. If you want 4k you can use HLS or the other thing (which I keep forgetting the acronym for).
If you want to get fancy you can do rendition audio, which not everything supports. Rendition audio means sharing one audio stream amongst N number of video streams.
You can use FFMPEG to transcode, but I'd suggest using AWS MediaConvert. It's cheap, fast, and probably does everything you want. Using FFmpeg directly works, but why bother. You will get an option wrong and screw everything up. You don't want your video to not work on some random device that 50k people are using in some country you didn't think about.
He's using RabbitMQ but you should use SQS, because SQS can trigger lambdas...which means no polling required. But use whatever queue you want.
You can kick the process off by attaching a Lambda to S3, which will start the process when the file is uploaded.
You can kick your "availability activation" off by attaching a Lambda to the S3 output bucket.
Background: I help run a streaming service and built the backend pipeline.
This omits the entire "metadata management and analytics" side as well. That's left as an exercise for the user.
- mannyv 3y agoOne thing people pointed out is minimizing egress fees. You do that by using CloudFlare, fastly, or another CDN. You can get bandwidth costs down to < .001/gb by committing to $1500/mo in bandwidth. The CDNs will pull from S3 once, then cache it forever (assuming you do it right).
- mannyv 3y agoBut even if you'd don't use AWS use something else. Any video encoder service will be cheaper and more reliable than using k8 + your own hacked together docker instance. It's literally not worth it. Do you really want to spend your time messing around with ffmpeg and setting the correct GOP values? Or trying to create your own b-frame track? At some point you actually need to do what all these settings are for. Until that time you should use other services.
- jonnycoder 3y agoWhat would you recommend using as an alternative to being locked into AWS?
- afvid 3y agoFull disclosure I currently work for Bitmovin, but we work well on AWS, Google and Azure. We have some migration tools and are one of Microsoft's recommended alternatives since they are sunsetting Azure Media Services. Our Streams product is a simple, one API call solution that just works, similar to some of the things mentioned already, but we also have a pretty advanced API that can do whatever you need. We're also about to wrap up a beta for GPU acceleration with NVIDIA and make that publicly available.
- jiggawatts 3y agoThis post is somewhat unfairly voted down. Cloud services like S3 and Azure Storage were invented specifically for hosting images and video. That’s their origin story, their foundation, their very reason for being. Similarly, cloud functions / lambda were invented for background processing of blobs. The first demos were always of resizing images! Building out this infrastructure yourself is a little insane. Unless you’re Netflix, don’t bother. Just dump your videos into blobs. It’s like driving to your cousin’s place, but step one is building your own highway because you couldn’t be bothered to check the map to see if one already existed. PS: Netflix serves video from BSD running directly on bare metal because at that scale efficiency matters! If efficiency doesn’t matter that much, use blobs. Kubernetes is going to be even worse.
- dilyevsky 3y agoI actually worked on system just like OP is describing and we ran everything ourselves including DBs (on k8s) so I can offer some perspective. For egress heavy services such as video minimizing outrageous cloud egress fees is key. Having an ability to tell your cloud account manager you can easily run multi-cloud or on-prem makes it a lot easier to negotiate sane rates. Ultimately you’re competing with the likes of Twitch (IVS), youtube and cloudflare on price and they ALL run their own compute so at certain size you will have to run your own hardware to stay competitive especially now that zirp is in rear view mirror. See Dropbox as another example of this but in documents/storage space
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- _joel 3y agoNot one mention of MPEG-DASH
- mikeryan 3y agoI believe thats what "or the other thing (which I keep forgetting the acronym for)." was referring to.
- _joel 3y agoWell it's an international standard, no biggie.
- bmurphy1976 3y agoYou need to use MPEG DASH if you are contractually obligated to use DRM. You can create an M3U8 and an MPEG DASH manifest that share the same encrypted segment files.
- mikeryan 3y agoI would like to know what the costs come out to per minute of video encoded and how many outputs they're getting in order to compare this to something like Media Convert (AWS), Google Transcode or something dedicated like Mux. For reference Google Transcoder is about $0.13 per minute of encoding (at four resolutions). Mux is at $0.032 / min, AWS Media Convert $0.0188. I should note I know Mux's pricing well, use them a lot, happily. It gets a bit confusing with Google and Media Convert because I'm not sure how these costs map to the resulting bitrate renditions that get created and I've not got the time for a deeper dive to get a more straight apples to apples comparison (ignore scale discounts) [1] https://cloud.google.com/transcoder/pricing https://cloud.google.com/transcoder/pricing [2] https://www.mux.com/pricing/video https://www.mux.com/pricing/video [3] https://aws.amazon.com/mediaconvert/pricing/ https://aws.amazon.com/mediaconvert/pricing/
- hckr1292 3y agoFull disclosure, I currently work at Mux on the video product. Previously though, I worked at an education startup with user generated video content. Like many others commenting on this thread, I built a simple queuing system using RabbitMQ and celery, transcoding on EC2 with ffmpeg. While we might have saved some money by doing this in house, we almost certainly discouraged users from uploading content because the entire video needed to transcoded before it could be viewed. For use cases like the breaking news or high traffic user generated content, you really want to minimize wait time, and that requires some kind of special sauce. At Mux, we encode content just in time for very fast publish times. It’s very challenging to do this on your own.
- c7DJTLrn 3y agoIf I'm understanding correctly you're suggesting using Lambda for processing. Is that a good idea? Lambda is actually expensive for heavy workloads.
- mannyv 3y agoHeck no, lambda kicks mediaconvert off.
- pyrophane 3y agoI believe the Lambda would just trigger a MediaConvert process, so it would actually be doing very little.
- c7DJTLrn 3y agoAh, got it, thanks.
- goeiedaggoeie 3y agoIf your video is short enough to encode in the lambda limit it is worth considering that https://aws.amazon.com/blogs/media/processing-user-generated-content-using-aws-lambda-and-ffmpeg/ https://aws.amazon.com/blogs/media/processing-user-generated... mediaconvert is expensive, generally AWS video stack is expensive (MediaConvert, Elemental, Media Connect)
- whalesalad 3y agoserverless on aws is underrated if your workload can fit on it. i have an app that hasn't received a single ounce of maintenance in like 3 years that still "just works", collects revenue from stripe, does all the business logic on lambdas, generates downloadable print-friendly pdf's on lambdas, etc. the supporting tech is dynamo + triggers for lambda, s3 and related triggers for lambda. but it would be hard for a non-expert aws user to fathom this sort of architecture, so i don't fault others for falling down the nih rabbit hole.
- irksomehails 3y agoHey, just curios, can you link your app.
- deleted 3y ago[deleted]
- TheFragenTaken 3y agoRecently got burnt by Microsoft deprecating Azure Media Services, so I can understand if the author doesn't want to use a cloud offering for encoding.
- mrd3v0 3y agoDepending on proprietary software such as cloud offerings for something as essential to its requirements such as encoding is not sustainable, and will create technical debt as your software/company will rely on the profitable success of the cloud service.