8 ms·

Latency isn't physically solvable though. You'll need another network not the internet.

klodolph 6y ago
It’s more or less workable for local (metropolitan) networks.
AmericanChopper 6y ago
Over surprisingly short distances you run into issues with the speed of light. I did live sound engineering when I was younger, and worked in some studio recordings, and the conventional wisdom of the field is that musicians will begin to get distracted by latency greater than 10ms (which you have to care about when sending foldback to a musician, even when all of your equipment is in the same room). 10ms only gets light 2/3rds of the straight line distance between LA and NY. After you start accounting latency in infrastructure components, round trips, and the fact that cables aren't laid in straight lines between two users, the idea of live musical collaboration over even short distances starts to seem implausible.
- bhj 6y ago
  The threshold is more like 3ms in my experience. When tracking vocals with headphones (no acoustic latency) anything higher will start feeling weird. This is also why Thunderbolt is preferred over USB for recording; you can get in and out of the CPU with effects much faster.
  - saurik 6y ago
    3ms by sound is only 3.36ft away.
    - AmericanChopper 6y ago
      Distance from the source of sound is actually very important in live performances. There have been some empirical studies on it, but it’s still something people often get rather opinionated about. There is a reason though that a small ensemble can play to the beat of something like a drum, but a large ensemble like an orchestra can’t. There’s plenty of writing about how orchestras lag behind conductors and how different sections of orchestras have to take their timing queues differently depending on where they’re situated. 10ms is what I was always told was the generally accepted threshold after which a musician start to experience increased difficulty as a result of the latency. Which largely aligns with my own experience with audio latency. From there the difficulty just keeps increasing, and the performance quality keeps degrading until at some point the musicians and the audience gives up.
      - pmiller2 6y ago
        Thanks for writing this. I made the exact same point in another subthread, that 10ms is enough to mess with your brain. My only experience of it is when the audio and the visual out of sync by only about 10ms while watching something (I forget what). I can't watch anything like that for any real length of time, and I'm not even trying to play along.
        saurik 6y ago
        10ms by light is 1862.824 miles away (which is approximately the distance from Los Angeles to Chicago). I honestly just don't think you are truly appreciating that light is almost a million times faster than sound.
        AmericanChopper 6y ago
        The refractive index of single model fiber is ~1.5. So 10ms of fiber transit is closer to 1250 miles. The fact that no two real world internet users are connected by a straight line of fiber optic cabling is going to bring that in even further. If you were relying on round trip latency for some reason, that would cut the distance in half. And that’s just for an idealized communication network. A real world use case would have a lot more latency introduced by routing/switching, and all of the typical quality issues you’d expect from ISPs. I used to live within line of sight of the data centre that hosted an online video game that I played, and the lower bound of my ping was about 15ms, it was usually in the 20s. Those factors are why something like this is unlikely to ever provide a high quality experience over even relatively short distances. I’ve just always found it interesting how quickly you start to run into issues with the speed of light when you’re trying to optimize for network latency.
        saurik 6y ago
        I mean, the stated use case is for a music school. I actually live in a University town. We have tons of students all "working remotely" within a single mile radius of campus, and many of them are connected together by our campus intranet -- the University does wireless point-to-multipoint links to connect all of these random housing options (which include large buildings they simply bought in the community) -- which is something I think we could easily push to include more buildings. I bet we could easily pull off 2ms networking overhead. (And no: you are wrong about round trip. If you care about round trip time then 3ms of round trip by sound is so close that the violin player is going to be elbowing the singer. Sound is fundamentally slow, and you need to just accept that.)
- Bud 6y ago
  It's actually not implausible; it's already happening. No, you can't do it between LA and NY, but you definitely can do it across distances of, say, several hundred miles quite well. https://www.ianhowellcountertenor.com/soundjack-comprehensive-user-guide https://www.ianhowellcountertenor.com/soundjack-comprehensiv...
- pmiller2 6y ago
  Well, I don't know about how fast electrons travel in copper, but I imagine it's well below c. In optical fiber, the speed of light is about 2/3 c. So, you should probably at least multiply your numbers by 2/3. In that case, you can only get 4/9 of the way between LA and NYC.
  - visarga 6y ago
    > how fast electrons travel in copper Signals are fast, electrons are slow - on the order of a quarter of a millimeter per second. > the drift speed through a copper wire of cross-sectional area 3.00 x 10-6 m^2, with a current of 10 A will be approximately 2.5 x 10-4 m/s https://www.physlink.com/education/askexperts/ae69.cfm https://www.physlink.com/education/askexperts/ae69.cfm
  - daniel-cussen 6y ago
    Signal travels at roughly 2/3 c in copper as well.
smolder 6y ago
The established technique and one that many people have independently invented (myself included) is to chain the performers so one person hears only a metronome and each other person hears the previous members on the chain, forwarding along the audio in sync. As I've said elsewhere in the thread zoom is the wrong tree to bark up since these tools exist.
- hunter2_ 6y ago
  As a musician, audio engineer, and software engineer, this just blew my mind. I can't believe I've never heard of or thought of this until now. But my primary focus has been non-audio software through the pandemic, so I haven't been actively trying to find solutions to this. Thank you. What applications would you recommend? Can it also do video, if not each link in the chain, at least one camera for the conductor (the node providing the metronome audio)? I realize the conductor would hear nobody at all, until playing back a recording from the end of the chain. Playback of said recording could begin after the song, or one measure into the song if there are no tempo/time changes (NINJAM style).
  - smolder 6y ago
    Someone else in this thread linked to software like I described: https://github.com/gwillen/solstice-audio-test https://github.com/gwillen/solstice-audio-test I will try to search around and get you some more examples. I can't remember off the top of my head what else worked that way.
  - pas 6y ago
    There's a sea shanty called the Wellerman that recently blow up on TikTok, and listening to it I was amazed how in sync the whole thing was despite all the layers. (It uses the same chain technique, but not "live".)
    - hunter2_ 6y ago
      Not-live overdubbing with perfect sync has been going on since the dawn of multi-track recording, although the popularity of including video from each layer (and DIY without a tracking engineer running the session) surged in popularity during the pandemic. With modern editing in a DAW, quantizing every phrase, word, or even syllable to the grid (or other layers) isn't difficult at all, just a bit tedious.
MrStonedOne 6y ago
I'm not sure on the percentage of assuming latency is only coming from the network/internet layer.
Bud 6y ago
Latency can be radically improved with SoundJack and tweaks to router settings, combined with audio equipment that introduces as little latency as possible. (i.e., don't use cheap USB mics.) Do all of this, and you can collaborate really well with people in the same geographic area as you. Of course you can't get around the speed of light, but you can do a lot better than the default setup or Zoom will give you.
- saurik 6y ago
  Yeah: it is maybe worth noting explicitly for people that the speed of sound is sufficiently slower than the speed of light that two people laying instruments on opposite sides of a small stage can actually, if they have pickup mics and headphones using software carefully designed to minimize latency, be reasonably far apart over the Internet and have the same experience (such as between a bunch of musicians living in San Francisco, or all in the same region of Los Angeles).
  - Bud 6y ago
    Even about two orders of magnitude further than that, based on the work done so far. For instance, 22.5ms latency has been achieved at a distance of 900 miles (between the middle of Kentucky and Boston) with SoundJack and this setup: Both port-forwarded, both using Fast Music Boxes, both on fiber internet connections. First user had an MXL770 XLR mic and an Audient EVO4 interface. Second user also had an XLR mic and an audio interface. San Francisco is 7 miles wide.
  - pmiller2 6y ago
    I think you're underestimating the amount of latency that the human ear can detect. If the audio is as little as 10 ms out of sync, either way, someone will notice that.
    - kilnr 6y ago
      How does this impact large orchestras? Some of these musicians have got to be (way) over 10 m apart. Even 10 m at the speed of sound is almost 30 ms.
layoutIfNeeded 6y ago
You'll need faster-than-light communication actually.

It’s more or less workable for local (metropolitan) networks.

Over surprisingly short distances you run into issues with the speed of light. I did live sound engineering when I was younger, and worked in some studio recordings, and the conventional wisdom of the field is that musicians will begin to get distracted by latency greater than 10ms (which you have to care about when sending foldback to a musician, even when all of your equipment is in the same room). 10ms only gets light 2/3rds of the straight line distance between LA and NY. After you start accounting latency in infrastructure components, round trips, and the fact that cables aren't laid in straight lines between two users, the idea of live musical collaboration over even short distances starts to seem implausible.

bhj 6y ago
The threshold is more like 3ms in my experience. When tracking vocals with headphones (no acoustic latency) anything higher will start feeling weird. This is also why Thunderbolt is preferred over USB for recording; you can get in and out of the CPU with effects much faster.
- saurik 6y ago
  3ms by sound is only 3.36ft away.
  - AmericanChopper 6y ago
    Distance from the source of sound is actually very important in live performances. There have been some empirical studies on it, but it’s still something people often get rather opinionated about. There is a reason though that a small ensemble can play to the beat of something like a drum, but a large ensemble like an orchestra can’t. There’s plenty of writing about how orchestras lag behind conductors and how different sections of orchestras have to take their timing queues differently depending on where they’re situated. 10ms is what I was always told was the generally accepted threshold after which a musician start to experience increased difficulty as a result of the latency. Which largely aligns with my own experience with audio latency. From there the difficulty just keeps increasing, and the performance quality keeps degrading until at some point the musicians and the audience gives up.
    - pmiller2 6y ago
      Thanks for writing this. I made the exact same point in another subthread, that 10ms is enough to mess with your brain. My only experience of it is when the audio and the visual out of sync by only about 10ms while watching something (I forget what). I can't watch anything like that for any real length of time, and I'm not even trying to play along.
      - saurik 6y ago
        10ms by light is 1862.824 miles away (which is approximately the distance from Los Angeles to Chicago). I honestly just don't think you are truly appreciating that light is almost a million times faster than sound.
        AmericanChopper 6y ago
        The refractive index of single model fiber is ~1.5. So 10ms of fiber transit is closer to 1250 miles. The fact that no two real world internet users are connected by a straight line of fiber optic cabling is going to bring that in even further. If you were relying on round trip latency for some reason, that would cut the distance in half. And that’s just for an idealized communication network. A real world use case would have a lot more latency introduced by routing/switching, and all of the typical quality issues you’d expect from ISPs. I used to live within line of sight of the data centre that hosted an online video game that I played, and the lower bound of my ping was about 15ms, it was usually in the 20s. Those factors are why something like this is unlikely to ever provide a high quality experience over even relatively short distances. I’ve just always found it interesting how quickly you start to run into issues with the speed of light when you’re trying to optimize for network latency.
        saurik 6y ago
        I mean, the stated use case is for a music school. I actually live in a University town. We have tons of students all "working remotely" within a single mile radius of campus, and many of them are connected together by our campus intranet -- the University does wireless point-to-multipoint links to connect all of these random housing options (which include large buildings they simply bought in the community) -- which is something I think we could easily push to include more buildings. I bet we could easily pull off 2ms networking overhead. (And no: you are wrong about round trip. If you care about round trip time then 3ms of round trip by sound is so close that the violin player is going to be elbowing the singer. Sound is fundamentally slow, and you need to just accept that.)
Bud 6y ago
It's actually not implausible; it's already happening. No, you can't do it between LA and NY, but you definitely can do it across distances of, say, several hundred miles quite well. https://www.ianhowellcountertenor.com/soundjack-comprehensive-user-guide https://www.ianhowellcountertenor.com/soundjack-comprehensiv...
pmiller2 6y ago
Well, I don't know about how fast electrons travel in copper, but I imagine it's well below c. In optical fiber, the speed of light is about 2/3 c. So, you should probably at least multiply your numbers by 2/3. In that case, you can only get 4/9 of the way between LA and NYC.
- visarga 6y ago
  > how fast electrons travel in copper Signals are fast, electrons are slow - on the order of a quarter of a millimeter per second. > the drift speed through a copper wire of cross-sectional area 3.00 x 10-6 m^2, with a current of 10 A will be approximately 2.5 x 10-4 m/s https://www.physlink.com/education/askexperts/ae69.cfm https://www.physlink.com/education/askexperts/ae69.cfm
- daniel-cussen 6y ago
  Signal travels at roughly 2/3 c in copper as well.

smolder 6y ago

The established technique and one that many people have independently invented (myself included) is to chain the performers so one person hears only a metronome and each other person hears the previous members on the chain, forwarding along the audio in sync. As I've said elsewhere in the thread zoom is the wrong tree to bark up since these tools exist.

hunter2_ 6y ago
As a musician, audio engineer, and software engineer, this just blew my mind. I can't believe I've never heard of or thought of this until now. But my primary focus has been non-audio software through the pandemic, so I haven't been actively trying to find solutions to this. Thank you. What applications would you recommend? Can it also do video, if not each link in the chain, at least one camera for the conductor (the node providing the metronome audio)? I realize the conductor would hear nobody at all, until playing back a recording from the end of the chain. Playback of said recording could begin after the song, or one measure into the song if there are no tempo/time changes (NINJAM style).
- smolder 6y ago
  Someone else in this thread linked to software like I described: https://github.com/gwillen/solstice-audio-test https://github.com/gwillen/solstice-audio-test I will try to search around and get you some more examples. I can't remember off the top of my head what else worked that way.
- pas 6y ago
  There's a sea shanty called the Wellerman that recently blow up on TikTok, and listening to it I was amazed how in sync the whole thing was despite all the layers. (It uses the same chain technique, but not "live".)
  - hunter2_ 6y ago
    Not-live overdubbing with perfect sync has been going on since the dawn of multi-track recording, although the popularity of including video from each layer (and DIY without a tracking engineer running the session) surged in popularity during the pandemic. With modern editing in a DAW, quantizing every phrase, word, or even syllable to the grid (or other layers) isn't difficult at all, just a bit tedious.

MrStonedOne 6y ago

I'm not sure on the percentage of assuming latency is only coming from the network/internet layer.

Bud 6y ago

Latency can be radically improved with SoundJack and tweaks to router settings, combined with audio equipment that introduces as little latency as possible. (i.e., don't use cheap USB mics.) Do all of this, and you can collaborate really well with people in the same geographic area as you. Of course you can't get around the speed of light, but you can do a lot better than the default setup or Zoom will give you.

saurik 6y ago
Yeah: it is maybe worth noting explicitly for people that the speed of sound is sufficiently slower than the speed of light that two people laying instruments on opposite sides of a small stage can actually, if they have pickup mics and headphones using software carefully designed to minimize latency, be reasonably far apart over the Internet and have the same experience (such as between a bunch of musicians living in San Francisco, or all in the same region of Los Angeles).
- Bud 6y ago
  Even about two orders of magnitude further than that, based on the work done so far. For instance, 22.5ms latency has been achieved at a distance of 900 miles (between the middle of Kentucky and Boston) with SoundJack and this setup: Both port-forwarded, both using Fast Music Boxes, both on fiber internet connections. First user had an MXL770 XLR mic and an Audient EVO4 interface. Second user also had an XLR mic and an audio interface. San Francisco is 7 miles wide.
- pmiller2 6y ago
  I think you're underestimating the amount of latency that the human ear can detect. If the audio is as little as 10 ms out of sync, either way, someone will notice that.
  - kilnr 6y ago
    How does this impact large orchestras? Some of these musicians have got to be (way) over 10 m apart. Even 10 m at the speed of sound is almost 30 ms.

layoutIfNeeded 6y ago

You'll need faster-than-light communication actually.