6 ms·

This approach has a rather severe disadvantage: packet loss. Especially in crowded WiFi networks the performance is going to be dismal. There are many codecs t

This approach has a rather severe disadvantage: packet loss. Especially in crowded WiFi networks the performance is going to be dismal. There are many codecs that address this such as G.711 codec. PCM is not the right approach.

userbinator 8y ago
To be precise, G.711 is PCM but with a much lower bitrate (and correspondingly lower quality.)
- edejong 8y ago
  You are absolutely right. I have used an incorrect source. The search was for forward error-correction in modern protocols... G.711 has neither properties. SMPTE2022-5/7 have such provisions, but I am not an expert in these protocols. Would an expert like to chime in? What are good (open) protocols to do home-streaming with low latency given occasional packet-loss (< 10%)?
  - gokhan 8y ago
    You sounded quite expert in your first post.
    - edejong 8y ago
      Well, I know for sure that PCM is not the right format for these kind of applications. However, I couldn't remember the abbreviation for the protocols anymore.
dfox 8y ago
While there are many codecs than can somehow cope with noisy and lossy channels, G.711 (ie. what is in telco world known as "The PCM") is not one of them.
gsich 8y ago
Why G711? G722 is much better with the same 64 kbit/s.
kkielhofner 8y ago
Let me start by saying this is a cool tool and I'm glad the author created it and released it. I wouldn't know the first thing about making such a beast. That said there is quite a bit they could learn from decades of progress in this field. I assume PCM was used because it is simple and could be done with existing Windows frameworks. That said there are many approaches that could be taken to improve this implementation over the wire (or air): - Applications like this are what CELT was designed for. You could certainly use straight CELT but these days you're better off grabbing OPUS and configuring it for 48 kHz, essentially putting it in CELT mode. I don't think standalone CELT libraries are still maintained by anyone anyway... - While the 44.1 kHz sample rate is famous for use in CD audio it's kind of ridiculous for other applications. 48 kHz is a far better starting choice for many, many reasons. Even better adjusting the rate on the fly depending on the source could be cool but I bet at the end of the day everything ends up getting resampled to some fixed sample rate somewhere in the audio stack. Audio in operating systems is actually quite interesting, what with the different sample rates of sources, abilities of physical hardware, mixing, etc, etc. - The packet format sounds goofy. Multicast RTP and other formats and implementations are a thing and they handle most of the issues the author is (rightfully) concerned about - packetization, MTU, etc. NIH DIY approaches such as this are one of my pet peeves... - Regarding Wifi and frame format, some compression should certainly be used. The 1/180 second of audio per frame comes to a packetization interval of about 5ms, which would cause a borderline insane flood (> 200 packets per second) of still relatively large packets. The author says 980 bytes at UDP, that still doesn't include the IP, Ethernet, etc frame overhead. For comparison purposes most realtime applications start with a packetization interval of about 20ms, resulting in about 50 packets per second. Of course the encoded frame size combined with the packetization interval determines how much bandwidth is actually used (fun fact, with some codecs you can end up with far more bandwidth used by headers than actual payload). Long story short, CELT with a standard RTP implementation running at 20ms will utilize substantially less bandwidth (too lazy to calculate it out). - OPUS incorporates SILK, which has a very robust FEC (forward error correction) and PLC (packet loss concealment) implementation. However, this only really applies at more-or-less speech sample rates when OPUS is running in SILK or hybrid modes (AFAIK). Even then very, very few implementations fully realize the dynamic packet loss adaptation that SILK is capable of. It requires an exquisitely complex implementation where the decoder exports loss statistics in real time to the encoder, which then determines the exact amount of FEC parity (essentially) to include in encoded packets to send. Then on the decoder side, the OPUS implementation needs to be able to access the jitter buffer to reconstruct lost frames from the parity information in adjacent packets. In the end most implementations just hardcode packet loss to some fixed value (10%), wasting some bandwidth (and potentially exacerbating packet loss). Fixed values are usually "good enough" but you still need to peek the jitter buffer on decode, and that's tough enough for a variety of reasons. Of course all of this also ignores asymmetric routing scenarios where send and receive packet loss potentially isn't equal, but that's a completely different rabbit hole and doesn't apply to "simple" scenarios like this where usage is on a simple, flat LAN. On paper latency will be higher with this approach but the quality will be substantially better and you won't be thrashing the hell out of your network infrastructure/spectrum/etc. Actually I'd be interested to see a real-world comparison, it's quite likely that this 200 pps approach using a ton of bandwidth actually has deterious effects on latency and the CELT+RTP+etc approach may actually end up with better latency too in real world scenarios.

userbinator 8y ago

To be precise, G.711 is PCM but with a much lower bitrate (and correspondingly lower quality.)

edejong 8y ago
You are absolutely right. I have used an incorrect source. The search was for forward error-correction in modern protocols... G.711 has neither properties. SMPTE2022-5/7 have such provisions, but I am not an expert in these protocols. Would an expert like to chime in? What are good (open) protocols to do home-streaming with low latency given occasional packet-loss (< 10%)?
- gokhan 8y ago
  You sounded quite expert in your first post.
  - edejong 8y ago
    Well, I know for sure that PCM is not the right format for these kind of applications. However, I couldn't remember the abbreviation for the protocols anymore.

dfox 8y ago

While there are many codecs than can somehow cope with noisy and lossy channels, G.711 (ie. what is in telco world known as "The PCM") is not one of them.

gsich 8y ago

Why G711? G722 is much better with the same 64 kbit/s.

kkielhofner 8y ago

Let me start by saying this is a cool tool and I'm glad the author created it and released it. I wouldn't know the first thing about making such a beast. That said there is quite a bit they could learn from decades of progress in this field. I assume PCM was used because it is simple and could be done with existing Windows frameworks. That said there are many approaches that could be taken to improve this implementation over the wire (or air): - Applications like this are what CELT was designed for. You could certainly use straight CELT but these days you're better off grabbing OPUS and configuring it for 48 kHz, essentially putting it in CELT mode. I don't think standalone CELT libraries are still maintained by anyone anyway... - While the 44.1 kHz sample rate is famous for use in CD audio it's kind of ridiculous for other applications. 48 kHz is a far better starting choice for many, many reasons. Even better adjusting the rate on the fly depending on the source could be cool but I bet at the end of the day everything ends up getting resampled to some fixed sample rate somewhere in the audio stack. Audio in operating systems is actually quite interesting, what with the different sample rates of sources, abilities of physical hardware, mixing, etc, etc. - The packet format sounds goofy. Multicast RTP and other formats and implementations are a thing and they handle most of the issues the author is (rightfully) concerned about - packetization, MTU, etc. NIH DIY approaches such as this are one of my pet peeves... - Regarding Wifi and frame format, some compression should certainly be used. The 1/180 second of audio per frame comes to a packetization interval of about 5ms, which would cause a borderline insane flood (> 200 packets per second) of still relatively large packets. The author says 980 bytes at UDP, that still doesn't include the IP, Ethernet, etc frame overhead. For comparison purposes most realtime applications start with a packetization interval of about 20ms, resulting in about 50 packets per second. Of course the encoded frame size combined with the packetization interval determines how much bandwidth is actually used (fun fact, with some codecs you can end up with far more bandwidth used by headers than actual payload). Long story short, CELT with a standard RTP implementation running at 20ms will utilize substantially less bandwidth (too lazy to calculate it out). - OPUS incorporates SILK, which has a very robust FEC (forward error correction) and PLC (packet loss concealment) implementation. However, this only really applies at more-or-less speech sample rates when OPUS is running in SILK or hybrid modes (AFAIK). Even then very, very few implementations fully realize the dynamic packet loss adaptation that SILK is capable of. It requires an exquisitely complex implementation where the decoder exports loss statistics in real time to the encoder, which then determines the exact amount of FEC parity (essentially) to include in encoded packets to send. Then on the decoder side, the OPUS implementation needs to be able to access the jitter buffer to reconstruct lost frames from the parity information in adjacent packets. In the end most implementations just hardcode packet loss to some fixed value (10%), wasting some bandwidth (and potentially exacerbating packet loss). Fixed values are usually "good enough" but you still need to peek the jitter buffer on decode, and that's tough enough for a variety of reasons. Of course all of this also ignores asymmetric routing scenarios where send and receive packet loss potentially isn't equal, but that's a completely different rabbit hole and doesn't apply to "simple" scenarios like this where usage is on a simple, flat LAN. On paper latency will be higher with this approach but the quality will be substantially better and you won't be thrashing the hell out of your network infrastructure/spectrum/etc. Actually I'd be interested to see a real-world comparison, it's quite likely that this 200 pps approach using a ton of bandwidth actually has deterious effects on latency and the CELT+RTP+etc approach may actually end up with better latency too in real world scenarios.