3 ms·

Very interesting! Can you tell me about that Cloudflare Workers setup? Would you have some sort of worker process the ingress video? Very new to the concept of

by GOATS- 5y ago

Very interesting! Can you tell me about that Cloudflare Workers setup? Would you have some sort of worker process the ingress video? Very new to the concept of serverless computing, so I would love to learn more.

bob1029 5y ago
The workers would handle both ingress and egress. The ingress feed should already be encoded for distribution by whatever system is pushing it. The durable objects in the middle would hold a moderate buffer of content which would be referred to on the egress routes. Archival would be a separate system that listens to all egress sessions at the same time. A 3rd websocket CF worker route could be used to exclusively manage video player/streamer/editor events. One low-latency system I had in mind would go: individual streamer frames (jpeg) -> PUT CF route -> route adds durable object per frame -> GET CF route w/ current frame counter returns newer frames -> Client web UI displays the jpeg frames with appropriate timing via a 2-3 frame buffer. This has much higher bandwidth requirements due to the loss of interframe compression effects, but it is compatible with 100% of platforms and very fast. It also allows for very simple seeking. If you are willing to leverage cloudflare and your target audience can handle a ~25 mbps download requirement, then this model could fit. You can get more mileage per bps if you reduce quality/resolution too. You can also put some intelligence in the stream path. Distributed compositing pipelines and other fun ideas. It's infinitely easier when everything is natively "just an image". Maybe some server's only job is to draw the hud on top of a frame received from a datacenter across the street.
- VWWHFSfQ 5y ago
  so you're going to playback mjpeg to people? I don't think this is going to work very well. at the very least it needs to be a shipping video in the form of a remotely modern codec/container to people.
  - bob1029 5y ago
    > I don't think this is going to work very well. I've built prototypes using libjpegturbo for encoding to web clients and it works out fantastically in my experience. I don't see a requirement to ship anything remotely modern if the desired product experience is achieved regardless. No one said it always has to work on every class of device everywhere for every type of user. Maybe someone just wants something that functions in any reasonable web browser and isnt interested in directly competing with twitch.tv.
    - VWWHFSfQ 5y ago
      I mean, I get what you're going for but there's a reason why nobody does this. mjpeg is just about the most inefficient thing you could possibly do. Maybe it would be fine if the desired product experience is "make it look like an animated gif and take 40 gigabytes to store an hour of video" edit: not trying to discourage! Just saying this is a pretty well-trodden path and mjpeg is mostly abandoned despite ubiquitous support for it
- NavinF 5y ago
  > the jpeg frames with appropriate timing via a 2-3 frame buffer Oh people have tried this many times. Don’t bother. Both x264 and NVENC can encode with no buffering (1 frame at a time) and get the same latency benefits without killing quality and bandwidth. JPEGs just look terrible unless they’re lossless. That aside, you brought up the real issue with open source DASH client implementations: They have terrible buffering algorithms that keep several seconds of video in memory instead of 2-3 frames even though the user might have a decent connection. Oh and by default they use the HTTP Date header for time synchronization between client/server so you only get 1 second resolution lol Part of the issue is that the majority of nontechnical users are on a shitty connection (WiFi or mobile data). They’ll always have to buffer >1 second (>60 frames) due to packet loss so they won’t see any benefits from a better implementation.

by GOATS- 5y ago

bob1029 5y ago

The workers would handle both ingress and egress. The ingress feed should already be encoded for distribution by whatever system is pushing it. The durable objects in the middle would hold a moderate buffer of content which would be referred to on the egress routes. Archival would be a separate system that listens to all egress sessions at the same time. A 3rd websocket CF worker route could be used to exclusively manage video player/streamer/editor events. One low-latency system I had in mind would go: individual streamer frames (jpeg) -> PUT CF route -> route adds durable object per frame -> GET CF route w/ current frame counter returns newer frames -> Client web UI displays the jpeg frames with appropriate timing via a 2-3 frame buffer. This has much higher bandwidth requirements due to the loss of interframe compression effects, but it is compatible with 100% of platforms and very fast. It also allows for very simple seeking. If you are willing to leverage cloudflare and your target audience can handle a ~25 mbps download requirement, then this model could fit. You can get more mileage per bps if you reduce quality/resolution too. You can also put some intelligence in the stream path. Distributed compositing pipelines and other fun ideas. It's infinitely easier when everything is natively "just an image". Maybe some server's only job is to draw the hud on top of a frame received from a datacenter across the street.

VWWHFSfQ 5y ago
so you're going to playback mjpeg to people? I don't think this is going to work very well. at the very least it needs to be a shipping video in the form of a remotely modern codec/container to people.
- bob1029 5y ago
  > I don't think this is going to work very well. I've built prototypes using libjpegturbo for encoding to web clients and it works out fantastically in my experience. I don't see a requirement to ship anything remotely modern if the desired product experience is achieved regardless. No one said it always has to work on every class of device everywhere for every type of user. Maybe someone just wants something that functions in any reasonable web browser and isnt interested in directly competing with twitch.tv.
  - VWWHFSfQ 5y ago
    I mean, I get what you're going for but there's a reason why nobody does this. mjpeg is just about the most inefficient thing you could possibly do. Maybe it would be fine if the desired product experience is "make it look like an animated gif and take 40 gigabytes to store an hour of video" edit: not trying to discourage! Just saying this is a pretty well-trodden path and mjpeg is mostly abandoned despite ubiquitous support for it
NavinF 5y ago
> the jpeg frames with appropriate timing via a 2-3 frame buffer Oh people have tried this many times. Don’t bother. Both x264 and NVENC can encode with no buffering (1 frame at a time) and get the same latency benefits without killing quality and bandwidth. JPEGs just look terrible unless they’re lossless. That aside, you brought up the real issue with open source DASH client implementations: They have terrible buffering algorithms that keep several seconds of video in memory instead of 2-3 frames even though the user might have a decent connection. Oh and by default they use the HTTP Date header for time synchronization between client/server so you only get 1 second resolution lol Part of the issue is that the majority of nontechnical users are on a shitty connection (WiFi or mobile data). They’ll always have to buffer >1 second (>60 frames) due to packet loss so they won’t see any benefits from a better implementation.