3 ms·

I’ve been trying to wrap my mind around whether (and how much) QUIC would be better than TCP for video streams that rely on order-sensitive delivery of frames,

I’ve been trying to wrap my mind around whether (and how much) QUIC would be better than TCP for video streams that rely on order-sensitive delivery of frames, especially for frames that have to be split into multiple packets (and losing one packet or receiving it out of order would lose the entire frame and any dependent frames). We used to use UDP for mpegts packets but found TCP with a reset when buffers are backlogged after switching to h264 to be a much better option over lossy WAN uplinks, (scenario is even worse when you have b or p frames present). The problem would be a lot easier if there were a feedback loop to the compressor where you can dynamically reduce quality/bandwidth as the connection quality deteriorates but currently using the stock raspivid (or v4l2) interface makes that a bit difficult unless you’re willing to explicitly stop and start the encoding all over again, which breaks the stream anyway.

londons_explore 3y ago
> stop and start the encoding all over again, which breaks the stream anyway. It's a common thing I wish encoders could do - if I try to compress a frame and the result is too large to fit in my packet/window, I wish I could 'undo' and retry compression of the same frame with different options. Sadly all hardware video compressors mutate internal state when something is compressed, so there is no way to undo.
mleo 3y ago
QUIC handles the packet ordering and retransmission of lost packets before it is handed to the upper layer of the application. However, if you are just sending one channel of video packets through the pipe, it probably won’t buy you much more. Where QUIC additionally excels is being able to send and/or receive multiple streams of data across the “single” connection where each is effectively independent and does not suffer head of line blocking across all streams.

londons_explore 3y ago

> stop and start the encoding all over again, which breaks the stream anyway. It's a common thing I wish encoders could do - if I try to compress a frame and the result is too large to fit in my packet/window, I wish I could 'undo' and retry compression of the same frame with different options. Sadly all hardware video compressors mutate internal state when something is compressed, so there is no way to undo.

mleo 3y ago

QUIC handles the packet ordering and retransmission of lost packets before it is handed to the upper layer of the application. However, if you are just sending one channel of video packets through the pipe, it probably won’t buy you much more. Where QUIC additionally excels is being able to send and/or receive multiple streams of data across the “single” connection where each is effectively independent and does not suffer head of line blocking across all streams.