4 ms·

TL;DR Alexa will be cloud-only very soon because it'll be powered by an LLM, which is too big to run on a smart speaker.

gleenn 2y ago
While the reason you suggest might be a good one, the repercussions of this change are clearly large. Unknowing customers now have their voice recordings sent to the cloud whoch could be leaked, have law enforcement get a warrant to take it from Amazon, or for it to be abused directly by Amazon. Those are huge. They might just have a good scapegoat.
- jamiek88 2y ago
  The law enforcement point is an excellent one. They were already caught giving ring footage without warrants.
- _ink_ 2y ago
  Yeah, but why do people still trust corporations? It's not like this has happened for the first time. Quite the opposite actually. For profit corporations seem to have no moral compass whatsoever and will do anything do increase the margin. People should finally start to act accordingly.
  - acuozzo 2y ago
    > Yeah, but why do people still trust corporations? When most of your time is spent ensuring your material needs are met, then whatever time remains is precious. Telling someone hustling two different "gig economy" jobs sixteen hours per day to give a shit about the long term privacy implications of the stupid smart speaker he got his kid for Christmas is asking… a lot. This is why change through popular political action peaked in the US in the late 1960s/early 1970s. People, on average, had more leisure time on their hands than ever before. More leisure time ⇒ more time to think, organize, unify, and give a shit beyond the immediate here & now.
- lynx97 2y ago
  [dead]
mastazi 2y ago
and there is no way to prevent that update from taking effect even though you "own" (wink wink) the device.
mrheosuper 2y ago
Doesn't alexa (or smart speaker) always need network to transfer voice data to server ? I doubt they have ever ran machine learning model on those speakers. Mostly just some simple one like "hey alexa"
Dylan16807 2y ago
That doesn't explain anything. LLMs use text as input. This feature already transcribes locally and sends text to the cloud.
- pzo 2y ago
  LLM can take audio as input, e.g. Gemini 2.0 Flash can take audio as input and is very fast and cheap (costs 25 audio tokens for a second, and prices $0.1 for 1 million input tokens). But I get your point that e.g. whisper is also very good and can run fast locally on edge devices. We also have recently phi4-multimodal LLM that is 5.6B params and is good enough to do inference locally on beefy smartphones. Problem is probably those smart speakers have really low CPU/GPU. Also LLM with audio input probably can be better re WER and take your mood/emotion into account when doing inference.
  - Dylan16807 2y ago
    Is the specific component that takes the audio in gemini an LLM? If so, how does audio data become tokens? But my point is not just about the ease of running whisper, it's that the device is already doing transcription. Even if it's not optimal, it works.
    - pzo 2y ago
      don't know exactly how it works under the hood for LLM. Gemini provide dedicated API for live API where you stream those audio via WebSocket - I guess they probably use some audio specific tokenizer. If I would like to guess why amazon is moving to cloud it's: 1) They support only 8 languages right now - cloud LLM or even whisper can support like 50 languages pretty well. I was always dissapointed that couldn't buy Google Mini or Alexa or Apple Home for my mum because none of them speak Polish. 2) They want to provide good support or those less beefy smartspeakers that don't have much power and those still sell well. 3) They wanna move people to this new Alexa subscription that they recently announced or make people more subscribe to Prime. 4) Gather more voice samples so they can train as good multilingual TTS as elevenlabs.
sorokod 2y ago
There is nothing inevitable about this. It a business decision to use an LLM. It's a business decision not to beef up the Alexa hardware to support an LLM. These decisions are weighted against reputational damage by Amazon with the outcome discussed here.
alex_suzuki 2y ago
> too big to run on a smart speaker. how convenient.
numpad0 2y ago
It's depressing that this hadn't happened years ago, neither for legal nor ethical reasons.

gleenn 2y ago

While the reason you suggest might be a good one, the repercussions of this change are clearly large. Unknowing customers now have their voice recordings sent to the cloud whoch could be leaked, have law enforcement get a warrant to take it from Amazon, or for it to be abused directly by Amazon. Those are huge. They might just have a good scapegoat.

jamiek88 2y ago
The law enforcement point is an excellent one. They were already caught giving ring footage without warrants.
_ink_ 2y ago
Yeah, but why do people still trust corporations? It's not like this has happened for the first time. Quite the opposite actually. For profit corporations seem to have no moral compass whatsoever and will do anything do increase the margin. People should finally start to act accordingly.
- acuozzo 2y ago
  > Yeah, but why do people still trust corporations? When most of your time is spent ensuring your material needs are met, then whatever time remains is precious. Telling someone hustling two different "gig economy" jobs sixteen hours per day to give a shit about the long term privacy implications of the stupid smart speaker he got his kid for Christmas is asking… a lot. This is why change through popular political action peaked in the US in the late 1960s/early 1970s. People, on average, had more leisure time on their hands than ever before. More leisure time ⇒ more time to think, organize, unify, and give a shit beyond the immediate here & now.
lynx97 2y ago
[dead]

mastazi 2y ago

and there is no way to prevent that update from taking effect even though you "own" (wink wink) the device.

mrheosuper 2y ago

Doesn't alexa (or smart speaker) always need network to transfer voice data to server ? I doubt they have ever ran machine learning model on those speakers. Mostly just some simple one like "hey alexa"

Dylan16807 2y ago

That doesn't explain anything. LLMs use text as input. This feature already transcribes locally and sends text to the cloud.

pzo 2y ago
LLM can take audio as input, e.g. Gemini 2.0 Flash can take audio as input and is very fast and cheap (costs 25 audio tokens for a second, and prices $0.1 for 1 million input tokens). But I get your point that e.g. whisper is also very good and can run fast locally on edge devices. We also have recently phi4-multimodal LLM that is 5.6B params and is good enough to do inference locally on beefy smartphones. Problem is probably those smart speakers have really low CPU/GPU. Also LLM with audio input probably can be better re WER and take your mood/emotion into account when doing inference.
- Dylan16807 2y ago
  Is the specific component that takes the audio in gemini an LLM? If so, how does audio data become tokens? But my point is not just about the ease of running whisper, it's that the device is already doing transcription. Even if it's not optimal, it works.
  - pzo 2y ago
    don't know exactly how it works under the hood for LLM. Gemini provide dedicated API for live API where you stream those audio via WebSocket - I guess they probably use some audio specific tokenizer. If I would like to guess why amazon is moving to cloud it's: 1) They support only 8 languages right now - cloud LLM or even whisper can support like 50 languages pretty well. I was always dissapointed that couldn't buy Google Mini or Alexa or Apple Home for my mum because none of them speak Polish. 2) They want to provide good support or those less beefy smartspeakers that don't have much power and those still sell well. 3) They wanna move people to this new Alexa subscription that they recently announced or make people more subscribe to Prime. 4) Gather more voice samples so they can train as good multilingual TTS as elevenlabs.

sorokod 2y ago

There is nothing inevitable about this. It a business decision to use an LLM. It's a business decision not to beef up the Alexa hardware to support an LLM. These decisions are weighted against reputational damage by Amazon with the outcome discussed here.

alex_suzuki 2y ago

> too big to run on a smart speaker. how convenient.

numpad0 2y ago

It's depressing that this hadn't happened years ago, neither for legal nor ethical reasons.