4 ms·

We hold regular flute meetings and play together. In this quarantine time we wanted to meet online, but if we play all at once, it seems I cannot hear everyone

by vitro 7y ago

We hold regular flute meetings and play together. In this quarantine time we wanted to meet online, but if we play all at once, it seems I cannot hear everyone else at the same time. I guess it is as if everyone was shouting over everyone else, which is not the case when you have a meeting where usually only one person speaks at a time Will this also fix this issue? So everyone will be able to hear everyone?

gnud 7y ago
You won't be able to play together because of latency. You will think you are in time with someone, but you will react when you hear/see them on your screen, which is maybe .15 seconds after they actually made the sound/movement. And then they will hear/see your reaction .15 seconds later again.
- lmm 7y ago
  Musicians already deal with that kind of issue when doing particular kinds of performance (e.g. famously at Wagner's festival opera house, where the orchestra is in a deep pit below the singers).
  - gnud 7y ago
    Yes - then you all follow a conductor. And you _don't_ listen for cues, so there's no reason to worry about not hearing everyone at the same time...
- tpolzer 7y ago
  If all participants have good internet and are geographically close it should theoretically be possible to have delay not much greater than rtt/2 for everybody. With rtt < 20ms that should make musical performances possible. After all, sound only travels less than four meters in 10ms. So this is just like singing in a choir (with more visual delay - but that can be solved by having a conductor). Unfortunately I'm not aware of any software making that a practical reality, even with ftth.
  - lachenmayer 7y ago
    You're assuming that network latency is the only latency that's involved here, but a huge latency source is the audio codec. Opus adds ~20ms latency, and that's the most low latency codec that's widely supported at the moment. You can see a comparison here: https://www.opus-codec.org/comparison/ https://www.opus-codec.org/comparison/ There are all sorts of other latency that need to be taken into consideration too, and unfortunately in practice those do add up to live music being unplayable on pretty much any network. There's a really interesting project called NINJAM https://www.cockos.com/ninjam/ https://www.cockos.com/ninjam/ which is designed for live music jam sessions. It flips this fundamental constraint on its head - instead of being real-time, it streams everyone else's output delayed by one bar (theoretically any interval >RTT I guess?). I haven't tried it, but it's a really cool idea.
    - tpolzer 7y ago
      Just because 20ms is the default doesn't mean that it has to be that way. The chart you linked shows a big "bubble" for opus for a reason. Opus minimum frame size is actually 2.5ms: https://tools.ietf.org/html/rfc6716#section-2.1.4 https://tools.ietf.org/html/rfc6716#section-2.1.4 Of course there's a ton of other potential sources of delay that make my fantasy hard to achieve, probably already starting at the typical USB microphones (in headsets/cameras).
      - namibj 7y ago
        20ms rtt through e.g. opus on a loopback network interface is already decidedly non-trivial to archive with "normal" hardware. When you do have low-latency devices, it becomes easy, but not everyone has those. USB should not really be an issue here, however.
  - hunter2_ 7y ago
    I haven't tried it yet, but sofasession.com seems optimized for this. Using wired Ethernet instead of WiFi can go a long way, from what I've heard. Has anyone here tried it?
  - 7y ago
- vitro 7y ago
  Latency is not that big problem I'd say. We play a music where it does not matter that much, sometimes just playing one long tone for the length of everyone's breath. I just would like to hear everybody at the same time, but what I hear is always one person's sound getting preference over others. Or sounds just alternate randomly based on the volume, I'd guess.
- Moru 7y ago
  Has anyone tried Mumble for this? It's very low latency but I can't find exactly how low the latency is. It ofcourse depends also on your internet connection and other settings but the base latency that comes from buffering the sound before sending. Mumble also has lots of settings for sound quality and different sound formats so might work for music if you try all the settings.
  - Nextgrid 7y ago
    Mumble has a setting for the audio buffer size and in fact they make you set it during initial configuration. It works great, has low latency and doesn't use much bandwidth (I hosted a server on a 1Mbps DSL connection for several people back in the days).
    - Moru 7y ago
      I'm hosting one on my Pi-Hole but no one want's to use it any more.
nh2 7y ago
My understanding is that WebRTC echo cancellation seems to work work by just attenuating down the everybody who is not the loudest speaker. You could try an app like Mumble, where you can turn that off (and also have other detail controls, at the cost of a bit more initial setup).
- kabes 7y ago
  That's not how it works. In fact there are multiple algorithms depending on the browser, it's not defined in the spec. The most used one currently would be AEC3 from Google, which is quite a bit more advanced than what you describe.

by vitro 7y ago

gnud 7y ago

You won't be able to play together because of latency. You will think you are in time with someone, but you will react when you hear/see them on your screen, which is maybe .15 seconds after they actually made the sound/movement. And then they will hear/see your reaction .15 seconds later again.

lmm 7y ago
Musicians already deal with that kind of issue when doing particular kinds of performance (e.g. famously at Wagner's festival opera house, where the orchestra is in a deep pit below the singers).
- gnud 7y ago
  Yes - then you all follow a conductor. And you _don't_ listen for cues, so there's no reason to worry about not hearing everyone at the same time...
tpolzer 7y ago
If all participants have good internet and are geographically close it should theoretically be possible to have delay not much greater than rtt/2 for everybody. With rtt < 20ms that should make musical performances possible. After all, sound only travels less than four meters in 10ms. So this is just like singing in a choir (with more visual delay - but that can be solved by having a conductor). Unfortunately I'm not aware of any software making that a practical reality, even with ftth.
- lachenmayer 7y ago
  You're assuming that network latency is the only latency that's involved here, but a huge latency source is the audio codec. Opus adds ~20ms latency, and that's the most low latency codec that's widely supported at the moment. You can see a comparison here: https://www.opus-codec.org/comparison/ https://www.opus-codec.org/comparison/ There are all sorts of other latency that need to be taken into consideration too, and unfortunately in practice those do add up to live music being unplayable on pretty much any network. There's a really interesting project called NINJAM https://www.cockos.com/ninjam/ https://www.cockos.com/ninjam/ which is designed for live music jam sessions. It flips this fundamental constraint on its head - instead of being real-time, it streams everyone else's output delayed by one bar (theoretically any interval >RTT I guess?). I haven't tried it, but it's a really cool idea.
  - tpolzer 7y ago
    Just because 20ms is the default doesn't mean that it has to be that way. The chart you linked shows a big "bubble" for opus for a reason. Opus minimum frame size is actually 2.5ms: https://tools.ietf.org/html/rfc6716#section-2.1.4 https://tools.ietf.org/html/rfc6716#section-2.1.4 Of course there's a ton of other potential sources of delay that make my fantasy hard to achieve, probably already starting at the typical USB microphones (in headsets/cameras).
    - namibj 7y ago
      20ms rtt through e.g. opus on a loopback network interface is already decidedly non-trivial to archive with "normal" hardware. When you do have low-latency devices, it becomes easy, but not everyone has those. USB should not really be an issue here, however.
- hunter2_ 7y ago
  I haven't tried it yet, but sofasession.com seems optimized for this. Using wired Ethernet instead of WiFi can go a long way, from what I've heard. Has anyone here tried it?
- 7y ago
vitro 7y ago
Latency is not that big problem I'd say. We play a music where it does not matter that much, sometimes just playing one long tone for the length of everyone's breath. I just would like to hear everybody at the same time, but what I hear is always one person's sound getting preference over others. Or sounds just alternate randomly based on the volume, I'd guess.
Moru 7y ago
Has anyone tried Mumble for this? It's very low latency but I can't find exactly how low the latency is. It ofcourse depends also on your internet connection and other settings but the base latency that comes from buffering the sound before sending. Mumble also has lots of settings for sound quality and different sound formats so might work for music if you try all the settings.
- Nextgrid 7y ago
  Mumble has a setting for the audio buffer size and in fact they make you set it during initial configuration. It works great, has low latency and doesn't use much bandwidth (I hosted a server on a 1Mbps DSL connection for several people back in the days).
  - Moru 7y ago
    I'm hosting one on my Pi-Hole but no one want's to use it any more.

nh2 7y ago

My understanding is that WebRTC echo cancellation seems to work work by just attenuating down the everybody who is not the loudest speaker. You could try an app like Mumble, where you can turn that off (and also have other detail controls, at the cost of a bit more initial setup).

kabes 7y ago
That's not how it works. In fact there are multiple algorithms depending on the browser, it's not defined in the spec. The most used one currently would be AEC3 from Google, which is quite a bit more advanced than what you describe.