5 ms·

Live has a _huge_ advantage in the storage side. In a purely "live" sense all of the content is temporally synchronised; every viewer is requesting approximatel

Live has a _huge_ advantage in the storage side. In a purely "live" sense all of the content is temporally synchronised; every viewer is requesting approximately the same segments at the same time. Store the current chunks, and the last few minutes of seek time, in memory and put out on the wire to all of the viewers. Twitch talked about this a bit just before/after the AMZN acquisition. In a prerecorded video CDN managing that catalog is a PITA and does drive meaningful infrastructure cost. You need the "right" content to be in the correct location for low cost peering/transit/distribution, on the correct media for the total throughput:size, in the optimal number of encodings for efficient/quality playback, etc. This job is a lot easier when the provider controls the catalog, and has a limited catalog size. See some of the OpenConnect talks where they're "preloading" content offpeak to optimize IO allocation on the appliances. It was an absolute nightmare to try and manage with a many PB catalog with 3P content that service didnt control the release/popularity of. Edit: source, principal at AWS and was responsible for a lot of the prime video delivery once upon a time.

andsoitis 2y ago
> Live has a _huge_ advantage in the storage side. In a purely "live" sense all of the content is temporally synchronised; every viewer is requesting approximately the same segments at the same time. Store the current chunks, and the last few minutes of seek time, in memory and put out on the wire to all of the viewers. Twitch talked about this a bit just before/after the AMZN acquisition. With Netflix's Live events, you can seek anywhere up to time zero, not just the last few minutes.
- adamomada 2y ago
  Apple does the same with the MLS soccer matches. I think you’re conflating “live” with “live and available on-demand anytime thereafter” Interestingly enough the Apple and I assume Netflix live streams come from the colo equipment in your ISP. So each box has their own recording as it happens.
- gbalduzzi 2y ago
  Netflix already solved the on demand streaming at scale though, for them it is harder to do live events given the fact they are new to it
flutas 2y ago
> Live has a _huge_ advantage in the storage side. In a purely "live" sense all of the content is temporally synchronised; every viewer is requesting approximately the same segments at the same time. Used to work at a live-streaming company on our stream infra. I mostly disagree, unless it's pure live no replay at all and no closely timed events required. Usually live platforms will offer some sort of a VOD (VODs, Replays, Rebroadcasts), all of which will require a storage solution. Couple that in with the fact that anything requiring more complex timing than "show video live~ish" can get messy fast with sync and latency issues.
- donavanm 2y ago
  Yes, i was referring to “live only” and not VOD/“low latency hls” cases. This is a decade ago but my examples off hand are things like video game, game shows, and contests. Was definitely a category, infrastructure looked a lot closer to multicastish RTMP than todays dynamic manifest mpeg segment CDNs. Edit: the above notwithstanding live sports etc is _still_ better on the storage side as viewers are so heavily synchronized. Lots of nice cache efficiencies when everyone is watching the same content at the same time.
amadeuspagel 2y ago
Does the recommendation algorithm account for this? If I'm in a specific place, am I more likely to see content that's already in the right data center?
- donavanm 2y ago
  I cant speak to content recommendations; I worked on the “backend” infrastructure storing bytes and delivering bits. But in that realm yes absolutely. Any CDNs #1 job is to route an end user request to a nearby (or otherwise optimal) datacenter, usually via DNS response. For streaming content I believe “everyone” is doing (part of) this at the streaming client/API layer these days. When you request to start playing the returned url will include/encode hints that help the CDN to send your request to the correct part of the CDN that holds your requested catalog title. ie CDNs arent homogenous and not all content will be stored in every edge location. The service API servers may/will even allocate different requests to different CDNs. eg the streaming service might use any combination of 1P (OpenConnect, CloudFront) and 3P (limelight, akamai, level 3) CDNs.

andsoitis 2y ago

> Live has a _huge_ advantage in the storage side. In a purely "live" sense all of the content is temporally synchronised; every viewer is requesting approximately the same segments at the same time. Store the current chunks, and the last few minutes of seek time, in memory and put out on the wire to all of the viewers. Twitch talked about this a bit just before/after the AMZN acquisition. With Netflix's Live events, you can seek anywhere up to time zero, not just the last few minutes.

adamomada 2y ago
Apple does the same with the MLS soccer matches. I think you’re conflating “live” with “live and available on-demand anytime thereafter” Interestingly enough the Apple and I assume Netflix live streams come from the colo equipment in your ISP. So each box has their own recording as it happens.
gbalduzzi 2y ago
Netflix already solved the on demand streaming at scale though, for them it is harder to do live events given the fact they are new to it

flutas 2y ago

> Live has a _huge_ advantage in the storage side. In a purely "live" sense all of the content is temporally synchronised; every viewer is requesting approximately the same segments at the same time. Used to work at a live-streaming company on our stream infra. I mostly disagree, unless it's pure live no replay at all and no closely timed events required. Usually live platforms will offer some sort of a VOD (VODs, Replays, Rebroadcasts), all of which will require a storage solution. Couple that in with the fact that anything requiring more complex timing than "show video live~ish" can get messy fast with sync and latency issues.

donavanm 2y ago
Yes, i was referring to “live only” and not VOD/“low latency hls” cases. This is a decade ago but my examples off hand are things like video game, game shows, and contests. Was definitely a category, infrastructure looked a lot closer to multicastish RTMP than todays dynamic manifest mpeg segment CDNs. Edit: the above notwithstanding live sports etc is _still_ better on the storage side as viewers are so heavily synchronized. Lots of nice cache efficiencies when everyone is watching the same content at the same time.

amadeuspagel 2y ago

Does the recommendation algorithm account for this? If I'm in a specific place, am I more likely to see content that's already in the right data center?

donavanm 2y ago
I cant speak to content recommendations; I worked on the “backend” infrastructure storing bytes and delivering bits. But in that realm yes absolutely. Any CDNs #1 job is to route an end user request to a nearby (or otherwise optimal) datacenter, usually via DNS response. For streaming content I believe “everyone” is doing (part of) this at the streaming client/API layer these days. When you request to start playing the returned url will include/encode hints that help the CDN to send your request to the correct part of the CDN that holds your requested catalog title. ie CDNs arent homogenous and not all content will be stored in every edge location. The service API servers may/will even allocate different requests to different CDNs. eg the streaming service might use any combination of 1P (OpenConnect, CloudFront) and 3P (limelight, akamai, level 3) CDNs.