3 ms·

This article is incorrect about WebRTC. I don’t know about other protocols and what they offer. * Clock Recovery I have had no problems with measuring NTP dri

This article is incorrect about WebRTC. I don’t know about other protocols and what they offer. * Clock Recovery I have had no problems with measuring NTP drift. As the clocks change I would measure. * Common Clock for Audio and Video Sender Reports contain a mapping of Sequence Number to RTP Sequence Numbers. This is respected by every player I have used. My guess is author put their media in different MediaStreams. If you want all your tracks to be synced you need to mark them as one MediaStream. * Defined latency WebRTC provides playoutDelay. https://webrtc.googlesource.com/src/+/main/docs/native-code/rtp-hdrext/playout-delay/README.md https://webrtc.googlesource.com/src/+/main/docs/native-code/.... This allows the sender to add delay to give a better experience. * Legacy transport You can transport anything you want via RTP or DataChannels. Maybe I am missing something with this one?

derf_ 3y ago
> I have had no problems with measuring NTP drift. Yeah, their claim is just weird. RTP does not impose an accuracy requirement on its timestamps (despite the name "NTP timestamp" in the Sender Reports, they are not actually expected to be synchronized with an NTP source), but I am skeptical such requirements would be met in practice if they did exist. The author only talks about video, but audio is a much bigger problem: if you do not send the right number of samples to the sound card at the right rate, you are going to get annoying clicks and pops, and "dropping or duplicating" audio frames will only cause more problems. You do not need to send media for a day to have these issues. Oscillators are bad enough they show up in minutes, and WebRTC obviously has strategies for dealing with them. > If you want all your tracks to be synced you need to mark them as one MediaStream. More specifically, the underlying mechanism of giving tracks that need to be synchronized the same CNAME has existed in RTP since the original RFC 1889 from the year 1996. The timestamp wrapping does require some care to get right, but is basically a non-issue. That said, a lot of WebRTC applications will not ask for synchronization because it necessarily introduces latency, and for interactive use cases you are often better served by sacrificing exact sync for lower latency (as long as latency is low enough, sync is never going to get too bad, anyway). But that is very different from saying the standards do not support it if you want it.
- izacus 3y ago
  Don't get myopic here - the timing constraints are critical for systems like DVB-T/C/S (and whatever the US equivalent is), not as much web. The things you're talking about might be dismissable when you're sending things from your blog app, but TS is primarily used in broadcasting. The DVB machines I've worked with were very sensitive to any kind of jitter and clock skew.
- kierank 3y ago
  The RFC does not mandate that the RTCP timestamp (which you need to handle wraparound if you join a stream halfway through) needs to be the same as the video/audio clock. In practice this clock is generated via the PC clock so it isn't the same clock at all: https://chromium.googlesource.com/external/webrtc/+/lkgr/modules/rtp_rtcp/source/rtcp_sender.cc#457 https://chromium.googlesource.com/external/webrtc/+/lkgr/mod... RTCP SRs are sent quite rarely (defaulting to 1s for video, 5s for audio) so quite poor for precise clock recovery required in professional applications. Probably practical implementations just use buffer fullness to drive their resampler.
  - derf_ 3y ago
    > In practice this clock is generated via the PC clock so it isn't the same clock at all... Yes, that is the point. WebRTC has to work with cheap / commodity hardware running a general purpose OS with clocks that are not synchronized. You get a bunch of clocks in different units running at different rates, and periodically are told the mappings between them. It is your job not to "run out of memory, or advance too quickly and run out of data to process," and indeed, WebRTC implementations have methods of solving those problems. > Probably practical implementations just use buffer fullness to drive their resampler. You are correct that in practice this gets driven by the jitter buffer in libwebrtc, but there is no resampler at all. Small changes in sampling rate are hard / computationally expensive to do well (as you certainly know). That is an implementation detail, though. You can use a different strategy if you have different requirements. > ...precise clock recovery required in professional applications. What are your actual precision requirements? I also do not understand your concerns with wrap-around. Even SRs once every 5 seconds are more than enough to resolve ambiguity. You would have to go over half a day without seeing one before you could actually get confused, by which point you would have bigger issues. Keep in mind that even at the start of the stream, the media timestamps are not absolute: each chooses an initial offset randomly.
kierank 3y ago
Author here: >I have had no problems with measuring NTP drift. As the clocks change I would measure. Did you read the article? NTP is not the same as the video/audio clock which is what you need to care about. I have to now take a drink even though it's 5am here in Singapore. > Common Clock for Audio and Video No idea what sequence numbers have to do with clocks here. Maybe you mean a mapping of absolute time to relative time in RTCP? If the RTCP SR value for absolute time is using NTP (or any other wrong clock as it's not mandated to match audio/video clock), then it's by definition impossible to know how to sync audio and video after several wraparounds of each RTP timestamp. > WebRTC provides playoutDelay. This is not the same as a defined, video frame or audio sample accurate delay (ten milliseconds as a unit...) to allow for variations in frame size to maximise quality. It also appears to mix up network delays vs VBV delays. They are separate delays and are handled at different layers of the stack. > You can transport anything you want via RTP or DataChannels None of this is standardised and therefore requires control of both ends. Also high end applications need Uncompressed Audio and for the above RTP timestamp reasons this can't be precisely synced with video.
- asabil 3y ago
  Each RTP packet has a 32bit timestamp, and a 32 bit SSRC. Each "sender" in an RTP session must use the same SSRC, this is how synchronisation between audio and video streams from the same sender (lip-sync) is achieved. The timestamps have a resolution defined by the clock rate communicated externally through a signalling channel.
  - akira2501 3y ago
    MPEG-TS allows for packaging multiple streams with independent time bases.
    - ta1243 3y ago
      For example audio at 48k and video at 90k. The problem with that being that as they wrap around at different times. So video wraps first, then audio wraps later, and this can happen several times over
kuschku 3y ago
Regarding clock, ideally you'd want to be able to genlock them. The goal is to ensure that all input sourced send their video frames at the exact same point in time, and that each audio device also samples at the exact same points in time. In the past that was even more important, as you'd want to make sure the scanline of CRTs in the studio and of the camera were perfectly synced.

derf_ 3y ago

> I have had no problems with measuring NTP drift. Yeah, their claim is just weird. RTP does not impose an accuracy requirement on its timestamps (despite the name "NTP timestamp" in the Sender Reports, they are not actually expected to be synchronized with an NTP source), but I am skeptical such requirements would be met in practice if they did exist. The author only talks about video, but audio is a much bigger problem: if you do not send the right number of samples to the sound card at the right rate, you are going to get annoying clicks and pops, and "dropping or duplicating" audio frames will only cause more problems. You do not need to send media for a day to have these issues. Oscillators are bad enough they show up in minutes, and WebRTC obviously has strategies for dealing with them. > If you want all your tracks to be synced you need to mark them as one MediaStream. More specifically, the underlying mechanism of giving tracks that need to be synchronized the same CNAME has existed in RTP since the original RFC 1889 from the year 1996. The timestamp wrapping does require some care to get right, but is basically a non-issue. That said, a lot of WebRTC applications will not ask for synchronization because it necessarily introduces latency, and for interactive use cases you are often better served by sacrificing exact sync for lower latency (as long as latency is low enough, sync is never going to get too bad, anyway). But that is very different from saying the standards do not support it if you want it.

izacus 3y ago
Don't get myopic here - the timing constraints are critical for systems like DVB-T/C/S (and whatever the US equivalent is), not as much web. The things you're talking about might be dismissable when you're sending things from your blog app, but TS is primarily used in broadcasting. The DVB machines I've worked with were very sensitive to any kind of jitter and clock skew.
kierank 3y ago
The RFC does not mandate that the RTCP timestamp (which you need to handle wraparound if you join a stream halfway through) needs to be the same as the video/audio clock. In practice this clock is generated via the PC clock so it isn't the same clock at all: https://chromium.googlesource.com/external/webrtc/+/lkgr/modules/rtp_rtcp/source/rtcp_sender.cc#457 https://chromium.googlesource.com/external/webrtc/+/lkgr/mod... RTCP SRs are sent quite rarely (defaulting to 1s for video, 5s for audio) so quite poor for precise clock recovery required in professional applications. Probably practical implementations just use buffer fullness to drive their resampler.
- derf_ 3y ago
  > In practice this clock is generated via the PC clock so it isn't the same clock at all... Yes, that is the point. WebRTC has to work with cheap / commodity hardware running a general purpose OS with clocks that are not synchronized. You get a bunch of clocks in different units running at different rates, and periodically are told the mappings between them. It is your job not to "run out of memory, or advance too quickly and run out of data to process," and indeed, WebRTC implementations have methods of solving those problems. > Probably practical implementations just use buffer fullness to drive their resampler. You are correct that in practice this gets driven by the jitter buffer in libwebrtc, but there is no resampler at all. Small changes in sampling rate are hard / computationally expensive to do well (as you certainly know). That is an implementation detail, though. You can use a different strategy if you have different requirements. > ...precise clock recovery required in professional applications. What are your actual precision requirements? I also do not understand your concerns with wrap-around. Even SRs once every 5 seconds are more than enough to resolve ambiguity. You would have to go over half a day without seeing one before you could actually get confused, by which point you would have bigger issues. Keep in mind that even at the start of the stream, the media timestamps are not absolute: each chooses an initial offset randomly.

kierank 3y ago

Author here: >I have had no problems with measuring NTP drift. As the clocks change I would measure. Did you read the article? NTP is not the same as the video/audio clock which is what you need to care about. I have to now take a drink even though it's 5am here in Singapore. > Common Clock for Audio and Video No idea what sequence numbers have to do with clocks here. Maybe you mean a mapping of absolute time to relative time in RTCP? If the RTCP SR value for absolute time is using NTP (or any other wrong clock as it's not mandated to match audio/video clock), then it's by definition impossible to know how to sync audio and video after several wraparounds of each RTP timestamp. > WebRTC provides playoutDelay. This is not the same as a defined, video frame or audio sample accurate delay (ten milliseconds as a unit...) to allow for variations in frame size to maximise quality. It also appears to mix up network delays vs VBV delays. They are separate delays and are handled at different layers of the stack. > You can transport anything you want via RTP or DataChannels None of this is standardised and therefore requires control of both ends. Also high end applications need Uncompressed Audio and for the above RTP timestamp reasons this can't be precisely synced with video.

asabil 3y ago
Each RTP packet has a 32bit timestamp, and a 32 bit SSRC. Each "sender" in an RTP session must use the same SSRC, this is how synchronisation between audio and video streams from the same sender (lip-sync) is achieved. The timestamps have a resolution defined by the clock rate communicated externally through a signalling channel.
- akira2501 3y ago
  MPEG-TS allows for packaging multiple streams with independent time bases.
  - ta1243 3y ago
    For example audio at 48k and video at 90k. The problem with that being that as they wrap around at different times. So video wraps first, then audio wraps later, and this can happen several times over

kuschku 3y ago

Regarding clock, ideally you'd want to be able to genlock them. The goal is to ensure that all input sourced send their video frames at the exact same point in time, and that each audio device also samples at the exact same points in time. In the past that was even more important, as you'd want to make sure the scanline of CRTs in the studio and of the camera were perfectly synced.