5 ms·
I just want to say that this is really actually kind of mind-blowing from an audio engineering perspective. Outputting audio from multiple laptops in the same
by crazygringo 2y ago
I just want to say that this is really actually kind of mind-blowing from an audio engineering perspective.
Outputting audio from multiple laptops in the same room is easy. Perfectly syncing it is harder. Implementing echo cancellation across all of that is quite a bit trickier than regular single-device echo cancellation.
But then treating all the laptop microphones as a kind of microphone array, having to deal with sync issues and phase issues and background noise issues... that's hard core.
Kudos to the engineering team on this one. This is actually pretty amazing.
- iandanforth 2y agoDo you think they are using some kind of inaudible-to-humans signal to coordinate this or is the unmodified audio good enough?
- vineyardmike 2y agoUnmodified audio is good enough, but using some signaling sound would improve coordination.
- dathery 2y agoI could see this happening. Zoom for example already uses ultrasonic sound to detect what conference room a laptop is in for one-click screen sharing.
- YZF 2y agoI wasn't aware audio hardware on laptops can emit ultrasound. Aren't those filtered to be below 22KHz?
- nomel 2y agoMost adults can’t hear > 18kHz. Related, there are devices you can buy to repel teenagers: https://www.today.com/news/controversial-mosquito-sonic-devices-deter-young-people-high-pitched-sounds-t157801 https://www.today.com/news/controversial-mosquito-sonic-devi...
- stavros 2y agoI can hear whatever my PC can output, but I can't hear anything in our Zoom meeting rooms. Maybe it's low volume, though.
- nomel 2y ago> I can hear whatever my PC can output This would require measuring equipment, since you can also only hear what you PC can output that you can hear. ;)
- hiimshort 2y agoAs someone who grew up with iPhones coming out around the time I was in middle school and apps producing noises like this were used as pranks in class I have to ask that people don't do this. The sound is painful.
- IshKebab 2y agoYeah I'm pretty sure there are plenty of adults that can hear those frequencies though. It's not like everyone reaches 18 and suddenly loses hearing. Stupid devices IMO. When they first came out I downloaded a sample audio file from the manufacturers website to see if I could hear it. I couldn't.... because they encoded it as MP3 and it was completely filtered out by the encoding! Literally an empty file.
- ercan 2y ago
- jorvi 2y agoChromecasts do it too, for pairing purposes. I’m sure other devices use it for clever purposes as well.
- whoknowswho 2y agoTeams does it too, I've had meetings where I've been able to audibly hear it coming from devices
- slac 2y agoNot great if you have dogs in the office unfortunately.
- jeffrallen 2y agoIf you were doing this kind of multipath and array signal processing on RF signals, it would be covered by ITAR. And yet Google can ship it anywhere on the planet via HTTPS and webasm... Things that make you go hmm.
- refulgentis 2y agoI have a very strong feeling this is more marketing BS than 100% solved technical achievement. (disclaimer: xoogler, no inside info, just familiar with recent Google and separately, audio processing)
- joshuamorton 2y agoIt would be....surprising if this was done client side and not serverside.
- varispeed 2y agoAlso imagine that 44.1kHz on one laptop will not equal 44.1kHz on another or one can run at 96kHz and others on 44.1kHz etc. that means everything has to be dynamically resampled in realtime whilst preserving quality and low latency.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- nytesky 2y agoTechnically very impressive, and meets a real need. My office is still running telecom hardware from a decade ago, all the wireless mics have dead batteries, and is reluctant to replace since so many meetings are completely virtual, so why have custom in office hardware. This essentially replaces that expensive proprietary hardware with a matrix of laptops, and essentially every user gets a mic. But it only works on laptops right, not phones?
- paul7986 2y ago[flagged]
- jauntywundrkind 2y agoI too am in awe of the audio engineering challenges and opportunities here. But I don't necessarily know that Meet is trying to tackle all this? Are they using the mics as a microphone array & processing signals across phases? Could be missing it but I don't see that they said so. Perhaps they're just picking the loudest mic for a given speaker? Or any of a dozen other simpler tactics?
- whazor 2y agoThe current baseline is to manually mute and unmute microphones. So picking the best microphone sounds like a better idea already. If other people make a sound, I think it would be acceptable of that sound was missed/softened.
- warble 2y agoI think this is much simpler than what you're suggesting. Careful microphone level management can handle this. No need for audio sync. I know they use the word "sync" but that's a very broad term.
- lomase 2y agoYou need sync to mitigate phase issues because the mics are in different positions on the room.
- jpc0 2y agoWe were taught the 1:3 rule in audio... If the distance from the microphone to an "unwanted source" is three times the distance asthat from the microphone to the source phasing likely wont be an issue. There's always caveats with engineering but it's a decent rule of thumb assuming equal volume sources... I can imagine it's not too hard to detect that anyway, weve been able to do realtime fft for a very long time.
- warble 2y agoNo, probably not there are no phase issues if you just don't transmit the signal. The hard part would be to determine who's in the room, and then who's talking and then mixing appropriately to eliminate feedback and optimize speaker sound quality. None of which requires signal phase accurate synchronicity. If they're actually able to "sync" (again a poorly defined term) given the problems associated with network latency and different hardware it would border on magic.
- makeitdouble 2y agoFrom the praise, I assume you have first hand experience with it ? Which part did you use and did you see any rough edges ?
- albertzeyer 2y agoI assume it's just some neural network doing the heavy lifting, or not?
- mzl 2y agoMy guess is that that would introduce too much latency (neural networks generally have bad latency).
- jasonjmcghee 2y agoDLSS has to run in some small fraction of 16ms (as everything else still needs to run in that time) to keep 60fps- granted it’s running on a GPU. But they can be quite fast. I wouldn’t be surprised if they were and audio is routed through a neural network before relay.
- weebull 2y agoAnd in audio, that meets the definition of really bad latency.
- everforward 2y agoIs a neural network even really necessary? This seems like something where careful application of some normal math would work. Find a loud event, correlate the loud event across devices to get offsets (by sequence ID, not clock time), do some fancy math to apply inverse sin waves or what not. That's not to diminish the accomplishment or say that it's easy (or that I could do it), but I don't think a neural network is necessary here.
- boffinAudio 2y agoI agree that this is amazing - but what I don't like about it, is the fact that a 3rd party is doing this, when it should really be a built-in feature of the operating system - or at least, be implemented as close to the device as possible. From what I can glean from this breathless press release, this functionality requires a fair bit of cloud ... anathema to audio professionals, but maybe not so, the professional management classes. Too many times these kinds of services are wrapped up at the application layer, where really they belong in the operating system. For example, wouldn't this be a perfect thing to implement as a plugin for Pulseaudio, or JACK, or even .. VST? (Disclaimer: I work on high end microphone and audio products at a well-known hardware manufacturer of such, where much more effort is being made to make the devices, themselves, smarter ..)
- jpc0 2y agoHow would you do this on the os? I would honestly try to sync audio output based on a shared time reference, something along the lines of what AES67/Ravenna/Dante does but you can be a little more lax and use ntp or system time since you don't need to be sample accurate. For the microphones that would be a little harder but you should be aware it's not that tough since a few high end manufacturers have phased microphone arrays for videoconferencing. You could probably get close though but the fact is you need the audio from all the sources in a single location for processing and do phase analysis on it and possibly find an optimal delay for each by checking the group delay. The advantage they have is some latency is acceptable and they don't need to do it on a low power device. I don't see what this has to do with Gemini but maybe that's just marketing...
- svnt 2y agoMy money is that outside of a couple of dog and pony demos with everyone on one well-administered LAN you could not make this work with system time and NTP on consumer devices. You will regularly see 100ms difference in NTP time. The fact that phased array microphones exist has nothing to do with the point we are discussing, which is audio coherence across heterogenous devices whose only real connection is a web browser.
- PaulDavisThe1st 2y agoIn a large room, perfect syncing is actually impossible since different listeners will be far enough from each other speakers to cause, at best, comb filtering, and at worst, audible delays.
- IsTom 2y agoI assume that if speakers are not set too loud then either you're close and the algorithm works or you're far and sound is quiet enough that it's not an issue.
- PaulDavisThe1st 2y agoIf the speakers are not set too loud, you don't need the algorithm at all.
- IsTom 2y agoIf you're sitting next to each other with regular volume then clearly you do, with the same volume, but across a room it's probably going to be ok.