In my experience, the problem with this is you're still going to run into issues with A/V sync if the input is muxed weird. You can end up having to buffer a to

In my experience, the problem with this is you're still going to run into issues with A/V sync if the input is muxed weird. You can end up having to buffer a ton of A or V before you get the packet for the timestamp in the present. I have an old project using fixed size packet buffers between the demuxer and decoders, and for some inputs it just deadlocks as one side gets full. My workaround to avoid having to buffer a potentially infinite amount of data was to instantiate the demuxer twice, once for A and once for V. Then they can run out of sync with each other in terms of the muxing, and you can keep them in sync in terms of presentation timestamps.