5 ms·

If all participants have good internet and are geographically close it should theoretically be possible to have delay not much greater than rtt/2 for everybody.

If all participants have good internet and are geographically close it should theoretically be possible to have delay not much greater than rtt/2 for everybody. With rtt < 20ms that should make musical performances possible. After all, sound only travels less than four meters in 10ms. So this is just like singing in a choir (with more visual delay - but that can be solved by having a conductor). Unfortunately I'm not aware of any software making that a practical reality, even with ftth.

lachenmayer 6y ago
You're assuming that network latency is the only latency that's involved here, but a huge latency source is the audio codec. Opus adds ~20ms latency, and that's the most low latency codec that's widely supported at the moment. You can see a comparison here: https://www.opus-codec.org/comparison/ https://www.opus-codec.org/comparison/ There are all sorts of other latency that need to be taken into consideration too, and unfortunately in practice those do add up to live music being unplayable on pretty much any network. There's a really interesting project called NINJAM https://www.cockos.com/ninjam/ https://www.cockos.com/ninjam/ which is designed for live music jam sessions. It flips this fundamental constraint on its head - instead of being real-time, it streams everyone else's output delayed by one bar (theoretically any interval >RTT I guess?). I haven't tried it, but it's a really cool idea.
- tpolzer 6y ago
  Just because 20ms is the default doesn't mean that it has to be that way. The chart you linked shows a big "bubble" for opus for a reason. Opus minimum frame size is actually 2.5ms: https://tools.ietf.org/html/rfc6716#section-2.1.4 https://tools.ietf.org/html/rfc6716#section-2.1.4 Of course there's a ton of other potential sources of delay that make my fantasy hard to achieve, probably already starting at the typical USB microphones (in headsets/cameras).
  - namibj 6y ago
    20ms rtt through e.g. opus on a loopback network interface is already decidedly non-trivial to archive with "normal" hardware. When you do have low-latency devices, it becomes easy, but not everyone has those. USB should not really be an issue here, however.
hunter2_ 6y ago
I haven't tried it yet, but sofasession.com seems optimized for this. Using wired Ethernet instead of WiFi can go a long way, from what I've heard. Has anyone here tried it?
atoav 6y ago
Depends on the type of music, something slow and choral can easily deal with high latencies, while something quick, rhythmic and precise can't be harder to deal with.
kragen 6y ago
Musicians building digital audio workstations commonly have to replace the whole software stack to get audio latency down to an acceptable (<10ms) level: JACK instead of PulseAudio, a Linux kernel recompiled with custom options for low latency, other software reconfigured to use the JACK APIs, and so on. Sometimes they can't use whatever standard audio hardware. (And remember that USB polling frequency is normally only 100 Hz: 10 ms worst-case by itself.) Minimizing latency is certainly technically feasible, it's just hard for stupid reasons.

lachenmayer 6y ago

You're assuming that network latency is the only latency that's involved here, but a huge latency source is the audio codec. Opus adds ~20ms latency, and that's the most low latency codec that's widely supported at the moment. You can see a comparison here: https://www.opus-codec.org/comparison/ https://www.opus-codec.org/comparison/ There are all sorts of other latency that need to be taken into consideration too, and unfortunately in practice those do add up to live music being unplayable on pretty much any network. There's a really interesting project called NINJAM https://www.cockos.com/ninjam/ https://www.cockos.com/ninjam/ which is designed for live music jam sessions. It flips this fundamental constraint on its head - instead of being real-time, it streams everyone else's output delayed by one bar (theoretically any interval >RTT I guess?). I haven't tried it, but it's a really cool idea.

tpolzer 6y ago
Just because 20ms is the default doesn't mean that it has to be that way. The chart you linked shows a big "bubble" for opus for a reason. Opus minimum frame size is actually 2.5ms: https://tools.ietf.org/html/rfc6716#section-2.1.4 https://tools.ietf.org/html/rfc6716#section-2.1.4 Of course there's a ton of other potential sources of delay that make my fantasy hard to achieve, probably already starting at the typical USB microphones (in headsets/cameras).
- namibj 6y ago
  20ms rtt through e.g. opus on a loopback network interface is already decidedly non-trivial to archive with "normal" hardware. When you do have low-latency devices, it becomes easy, but not everyone has those. USB should not really be an issue here, however.

hunter2_ 6y ago

I haven't tried it yet, but sofasession.com seems optimized for this. Using wired Ethernet instead of WiFi can go a long way, from what I've heard. Has anyone here tried it?

atoav 6y ago

Depends on the type of music, something slow and choral can easily deal with high latencies, while something quick, rhythmic and precise can't be harder to deal with.

kragen 6y ago

Musicians building digital audio workstations commonly have to replace the whole software stack to get audio latency down to an acceptable (<10ms) level: JACK instead of PulseAudio, a Linux kernel recompiled with custom options for low latency, other software reconfigured to use the JACK APIs, and so on. Sometimes they can't use whatever standard audio hardware. (And remember that USB polling frequency is normally only 100 Hz: 10 ms worst-case by itself.) Minimizing latency is certainly technically feasible, it's just hard for stupid reasons.