4 ms·
Interesting project! Curious what exactly you tried - H.264 is a codec, and there's a bunch of ways of delivering it to the client (HLS, WebRTC, some have buil
by detaro 8y ago
Interesting project!
Curious what exactly you tried - H.264 is a codec, and there's a bunch of ways of delivering it to the client (HLS, WebRTC, some have build WebSocket-based streaming, ...), and I'd expect that the main latency is hidden there, not in the decoding?
- ryanmjacobs 8y agoOf course. FFMpeg was used consistently on our backend; we cycled through various codecs to find the best one for our purpose and changed the frontend library accordingly. And I think you're right on the encoding latency. I believe that H.264 buffers a little bit before it makes a decision on how to compress the frames, where as MPEG1 doesn't? I could be completely wrong but my gut is telling me that MPEG1 is basically independent, slightly-modified JPEG frames.
- josteink 8y ago> I could be completely wrong but my gut is telling me that MPEG1 is basically independent, slightly-modified JPEG frames. Not that simple, but not that far off. Basically MPEG1 has key-frames (“JPEG” frames) and (forward) delta-frames (diffs). From my understanding H264 has many improvements, including reverse delta-frames. My guess is that disabling those in the encoder will improve real-time capabilities.
- Orphis 8y agoBingo, that's one of the tuning you can use for real-time communication. There's a lot of ways to tune an encoder, and getting optimal setting for each use case is always tricky.
- ryanmjacobs 8y agoAh gotcha, I think I was mixing up MPEG1 and MJPEG in my head. I didn't realize that you could disable those features in the encoder. I'll have to look into x264 tunables that lower latency. Might be interesting.
- wolf550e 8y agolibx264 (which is the H.264 software encoder everyone uses) has a mode optimized for zero latency (--tune zerolatency).