5 ms·

Speech recognition on edge devices is still very compute intensive—it's either too slow or inaccurate or constrained in what it can recognize. Alexa and Google

Speech recognition on edge devices is still very compute intensive—it's either too slow or inaccurate or constrained in what it can recognize. Alexa and Google both still stream audio to the cloud for processing, though the wake word detection is typically done locally. Modern phones are starting to do more processing locally and I could see that making its way to the smart speakers as well if they survive as a product category, but they're definitely paying for compute on most existing devices.

fbdab103 3y ago
Good to know. I realize Whisper is new, but I have been blown away by its performance. I would think a revamped device could do all of the processing locally. Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient.
- lolinder 3y ago
  Yeah, whisper is the closest thing we have, but even it requires more processing power than is present in most of these edge devices in order to feel smooth. I've started a voice interface project on a Raspberry Pi 4, and it takes about 3 seconds to produce a result. That's impressive, but not fast enough for Alexa. From what I gather a Pi 5 can do it in 1.5 seconds, which is closer, so I suspect it's only a matter of time before we do have fully local STT running directly on speakers. > Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient. Yes, this is the approach taken by a lot of streaming STT systems, like Kaldi [0]. Rather than use a fully capable model, you train a specialized one that knows what kinds of things people are likely to say to it. [0] http://kaldi-asr.org/ http://kaldi-asr.org/

fbdab103 3y ago

Good to know. I realize Whisper is new, but I have been blown away by its performance. I would think a revamped device could do all of the processing locally. Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient.

lolinder 3y ago
Yeah, whisper is the closest thing we have, but even it requires more processing power than is present in most of these edge devices in order to feel smooth. I've started a voice interface project on a Raspberry Pi 4, and it takes about 3 seconds to produce a result. That's impressive, but not fast enough for Alexa. From what I gather a Pi 5 can do it in 1.5 seconds, which is closer, so I suspect it's only a matter of time before we do have fully local STT running directly on speakers. > Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient. Yes, this is the approach taken by a lot of streaming STT systems, like Kaldi [0]. Rather than use a fully capable model, you train a specialized one that knows what kinds of things people are likely to say to it. [0] http://kaldi-asr.org/ http://kaldi-asr.org/