2 ms·

Seems like the multiple microphone beamforming source separation algorithms are getting pretty good these days, maybe just adding a lot more mics would help? C

Seems like the multiple microphone beamforming source separation algorithms are getting pretty good these days, maybe just adding a lot more mics would help? Could you have an AI model that extracts some characteristics of the speaker's voice for each individual word, then translates that to color and font? If the model was not confident about a word it could show slightly blurred, if it was loud it could be bold, perhaps(Although there's some stereotype issues) you could use different fonts for different pitches, whispers could be grey, quiet could be transparent. Maybe there's a language model that can pick up overlapping words if you don't have the constraint of needing to sort them out into who said it, just show all the possibile words that could have been said by anyone stacked together, in a "not sure" color, and maybe the wearer would eventually learn to figure it out without much effort? You could also try to stay consistent so the same speaker gets the same colors I'd possible, and also not reuse colors for new speakers that have been recently used, to best make use of the limited bits of data in font and color. Maybe just by showing all the words from every speaker all together like that, the wearer would be able to figure it out even if it made mistakes in the speaker identification?

DrKeithDuggar 4y ago
Regarding additions mics, yes that enabled more advanced spatial processing, especially when arranged in a determined and calibrated geometry. One could imagine glasses with several mics placed at optimal locations on the frames. Such multichannel audio could then be processed into multichannel spatial audio streams. As for the rest, thank you for wonderful ideas! Everything you propose is technically possible. The difficulties arise first in assessing the increased benefit to users versus the increased complexity of user experience, and second in prioritizing the work versus other features. Over time we do hope to add additional selectable “skins”, which is to say different UI designs, that allow users to choose UI’s from a wide range. Everything from the simple to complex layouts, from accessible to exotic color palettes, from professional to playful themes, etc. I could definitely see more advanced visual representations of transcription uncertainty showing up in such optional skins.

DrKeithDuggar 4y ago

Regarding additions mics, yes that enabled more advanced spatial processing, especially when arranged in a determined and calibrated geometry. One could imagine glasses with several mics placed at optimal locations on the frames. Such multichannel audio could then be processed into multichannel spatial audio streams. As for the rest, thank you for wonderful ideas! Everything you propose is technically possible. The difficulties arise first in assessing the increased benefit to users versus the increased complexity of user experience, and second in prioritizing the work versus other features. Over time we do hope to add additional selectable “skins”, which is to say different UI designs, that allow users to choose UI’s from a wide range. Everything from the simple to complex layouts, from accessible to exotic color palettes, from professional to playful themes, etc. I could definitely see more advanced visual representations of transcription uncertainty showing up in such optional skins.