3 ms·

What if Zoom had spatial audio?

3 points by troydavis 6y ago

Communitivity 6y ago
FYSA, High Fidelity is a virtual world startup / standards project created by the Philip Rosedale, the guy that founded Linden Labs (makers of Second Life).
SamBam 6y ago
This seems very interesting, and definitely something that I could see getting integrated into a product like Zoom eventually. Is the Spatial Audio function fast-enough to be done client-side, so it can deal with the way different clients might be rendering the talking heads in different orders? If I make my client really wide, so that all heads are in a line, it should sound different than if all the heads are stacked vertically. (Also, people on "page 2" of a big meeting could sound like they are further away -- indeed, that latter feature is something I've always thought should exist for a teacher to be able to listen to the soft hubub of breakout rooms). If the transfer function is fast, I could also see it being eventually paired with the camera that is already watching your face, so that it could tweak it subtly as you turn your head. I've noticed that people already naturally cock their heads towards someone they're straining to hear, even though they are wearing headphones so it makes no difference (except as a subtle hint to the person talking).
- PaulHoule 6y ago
  I listened carefully with my eyes closed and didn't feel there was a problem telling the speakers apart with Zoom. The Spatial Audio put them all through distinct filters which I thought had great localization in the left/right axis and they have mastery of far field vs near field effects so some are definitely closer than others. Some of the voices seem to be slightly behind my ear but because it doesn't try anything extreme it doesn't fail like the average video game HRTF demo. On top of that they put distinct reverb on the voices because a huge part of the 'surround' experience of sound is how the sound interacts with the space it is in. But with my eyes closed and listening hard I hear a lot of richness in the zoom recording too; the Spatial Audio is louder at times then the zoom recording and that is hard to control but unfair because you have to get terribly loud before most people don't prefer the louder sound (i either made the excuse I was afraid to blow the speakers or i planned ahead and palmed OSHA ear plugs -20db before you say Uncle)
PaulHoule 6y ago
I remember sometimes making stereo phone calls in the 1980s with two phones.
linsomniac 6y ago
I was just speculating a week or so ago that it might be much better if they would use stereo output for multiple speakers. That seems to be the thing that we've had the most problem with: two people start talking and the combined audio is harder to understand over videoconferencing than in person. But stereo output is pretty easy and nearly universally supported (one co-worker has a little puck-mic/speaker that's designed for conferencing and works pretty well, but has no stereo separation, for example). This seems to take it to the next level.

3 points by troydavis 6y ago

FYSA, High Fidelity is a virtual world startup / standards project created by the Philip Rosedale, the guy that founded Linden Labs (makers of Second Life).

SamBam 6y ago

This seems very interesting, and definitely something that I could see getting integrated into a product like Zoom eventually. Is the Spatial Audio function fast-enough to be done client-side, so it can deal with the way different clients might be rendering the talking heads in different orders? If I make my client really wide, so that all heads are in a line, it should sound different than if all the heads are stacked vertically. (Also, people on "page 2" of a big meeting could sound like they are further away -- indeed, that latter feature is something I've always thought should exist for a teacher to be able to listen to the soft hubub of breakout rooms). If the transfer function is fast, I could also see it being eventually paired with the camera that is already watching your face, so that it could tweak it subtly as you turn your head. I've noticed that people already naturally cock their heads towards someone they're straining to hear, even though they are wearing headphones so it makes no difference (except as a subtle hint to the person talking).

PaulHoule 6y ago
I listened carefully with my eyes closed and didn't feel there was a problem telling the speakers apart with Zoom. The Spatial Audio put them all through distinct filters which I thought had great localization in the left/right axis and they have mastery of far field vs near field effects so some are definitely closer than others. Some of the voices seem to be slightly behind my ear but because it doesn't try anything extreme it doesn't fail like the average video game HRTF demo. On top of that they put distinct reverb on the voices because a huge part of the 'surround' experience of sound is how the sound interacts with the space it is in. But with my eyes closed and listening hard I hear a lot of richness in the zoom recording too; the Spatial Audio is louder at times then the zoom recording and that is hard to control but unfair because you have to get terribly loud before most people don't prefer the louder sound (i either made the excuse I was afraid to blow the speakers or i planned ahead and palmed OSHA ear plugs -20db before you say Uncle)

PaulHoule 6y ago

I remember sometimes making stereo phone calls in the 1980s with two phones.

linsomniac 6y ago

I was just speculating a week or so ago that it might be much better if they would use stereo output for multiple speakers. That seems to be the thing that we've had the most problem with: two people start talking and the combined audio is harder to understand over videoconferencing than in person. But stereo output is pretty easy and nearly universally supported (one co-worker has a little puck-mic/speaker that's designed for conferencing and works pretty well, but has no stereo separation, for example). This seems to take it to the next level.