4 ms·

Do the current Alexa devices do their voice recognition on device on in cloud? Curious what are their operational costs for the service, because I naively thoug

Do the current Alexa devices do their voice recognition on device on in cloud? Curious what are their operational costs for the service, because I naively thought that the device handled the basic operations (timer, weather) with some API calls which should be minimal data. Switching out to a novel AI model just to stay hip sounds expensive.

pchristensen 3y ago
Alexa is almost entirely cloud-based. The devices are mostly just a speaker, a microphone, and a power supply. That s why they’re so cheap and why Alexa has been added to so many different form factors and device types.
lolinder 3y ago
Speech recognition on edge devices is still very compute intensive—it's either too slow or inaccurate or constrained in what it can recognize. Alexa and Google both still stream audio to the cloud for processing, though the wake word detection is typically done locally. Modern phones are starting to do more processing locally and I could see that making its way to the smart speakers as well if they survive as a product category, but they're definitely paying for compute on most existing devices.
- fbdab103 3y ago
  Good to know. I realize Whisper is new, but I have been blown away by its performance. I would think a revamped device could do all of the processing locally. Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient.
  - lolinder 3y ago
    Yeah, whisper is the closest thing we have, but even it requires more processing power than is present in most of these edge devices in order to feel smooth. I've started a voice interface project on a Raspberry Pi 4, and it takes about 3 seconds to produce a result. That's impressive, but not fast enough for Alexa. From what I gather a Pi 5 can do it in 1.5 seconds, which is closer, so I suspect it's only a matter of time before we do have fully local STT running directly on speakers. > Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient. Yes, this is the approach taken by a lot of streaming STT systems, like Kaldi [0]. Rather than use a fully capable model, you train a specialized one that knows what kinds of things people are likely to say to it. [0] http://kaldi-asr.org/ http://kaldi-asr.org/

pchristensen 3y ago

Alexa is almost entirely cloud-based. The devices are mostly just a speaker, a microphone, and a power supply. That s why they’re so cheap and why Alexa has been added to so many different form factors and device types.

lolinder 3y ago

Speech recognition on edge devices is still very compute intensive—it's either too slow or inaccurate or constrained in what it can recognize. Alexa and Google both still stream audio to the cloud for processing, though the wake word detection is typically done locally. Modern phones are starting to do more processing locally and I could see that making its way to the smart speakers as well if they survive as a product category, but they're definitely paying for compute on most existing devices.

fbdab103 3y ago
Good to know. I realize Whisper is new, but I have been blown away by its performance. I would think a revamped device could do all of the processing locally. Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient.
- lolinder 3y ago
  Yeah, whisper is the closest thing we have, but even it requires more processing power than is present in most of these edge devices in order to feel smooth. I've started a voice interface project on a Raspberry Pi 4, and it takes about 3 seconds to produce a result. That's impressive, but not fast enough for Alexa. From what I gather a Pi 5 can do it in 1.5 seconds, which is closer, so I suspect it's only a matter of time before we do have fully local STT running directly on speakers. > Probably anathema to the space, but if the devices leaned into the ~five tasks people use them for (timers, weather, todo list?) could probably tighten up the AI models to be more accurate and/or resource efficient. Yes, this is the approach taken by a lot of streaming STT systems, like Kaldi [0]. Rather than use a fully capable model, you train a specialized one that knows what kinds of things people are likely to say to it. [0] http://kaldi-asr.org/ http://kaldi-asr.org/