4 ms·

I don't use Alexa. How much of their compute is in the cloud? Do they do all voice recognition in the cloud? If the voice recognition was done on local hardware

by jerf 4y ago

I don't use Alexa. How much of their compute is in the cloud? Do they do all voice recognition in the cloud? If the voice recognition was done on local hardware you could get Alexa down to the moral equivalent of a hobby for Amazon without much work. There are open source projects that are nearly the equivalent with nearly no server resources, or ones so minimal they essentially run for free. But if all the voice recognition is done in the cloud they're in big trouble, because even the minimum size the service can have is huge. Probably another victim of the end of 0% interest rates is the idea that we can force everything into "the cloud" and run it indefinitely, funded by the whiffs and vapors of whatever possible value we can extract (without running afoul of privacy laws or angry customers if they find out) in the fractions of a cent per month or year range. Compute has gotten cheap but it's still not free, and "the cloud" is still overwhelmingly outweighed in power by "not the cloud". Plus the compute tasks keeps getting larger and larger; it doesn't do "the cloud" any good if people smoothly transition from wanting voice recognition to wanting GPT-type chatbots and cloud video gaming and other things that you can't serve 50,000 people from a single beefy computer. I've been resisting a lot of cloud-level computation in my life, where even where I permit it I retain the ability to do my own thing, like serve my own videos in my house. I've really been more concerned about data ownership but this has factored in, too; I'm abundantly endowed with more computing resources at home than I can use, but all these free or cheap cloud services are remote (adding latency) and incentivized to short change the resources because they are expensive. The ad-based economy still doesn't pay enough to fund reasonable levels of performance; Facebook's profits couldn't afford a Raspberry Pi's level of performance dedicated to you in a year, let alone what I have sitting around the house. Even as computing becomes cheap in the cloud, the computing in my house is also becoming cheap, and I'm not even particularly trying to have a lot of it. Trying to keep up with it in a centralized system picking up a penny here and a penny there is not something that can be sustained without free money.

vishnugupta 4y ago
> How much of their compute is in the cloud? Do they do all voice recognition in the cloud? Their voice assistance is constantly learning so a good chunk of voice recognition related computation takes place in cloud. However, even if the voice recognition is done locally, the vast majority of responses are streamed from the cloud. Only trivial requests such as "Set timer" can be handled locally. As of 2019, Amazon had sold 100 million Alexa devices. Whichever way I look at it, those many devices incur a big running cost at the backend cloud to service.
- jerf 4y ago
  Thank you for the info. This post is just further noodling, not any sort of disagreement. "Responses" streamed from the cloud differ a lot in terms of whether or not it's some sort of "API" response that could be a couple of cheap kilobytes of JSON, or a full text-to-speech audio file. I can't speak to Alexa, but I do use Google Maps, and it obviously has two modes for its TTS; a slightly nicer mode that streams from the cloud if you're on the network, but a perfectly serviceable lower quality one generated from the device if it can't get to the cloud at the time. I've often wondered why they don't just put the nicer cloud model on my device. Maybe when Maps was first starting it was a big difference, but now I'm sure my cell can run that "nice" model just as well. It's not like they're running a massive multi-billion parameter thing that only runs on the latest graphics cards (or, if they are, they aren't getting that much additional value for it), I'm sure it's something we could run locally just fine. Learning could always be done by sampling some things to send up.
  - vishnugupta 4y ago
    For sure I agree with your overarching assessment though. The irony, as you noted, is even as consumer devices have gotten more and more powerful computation and processing is being pushed to the backend/cloud. For example; to get a EC2 that's comparable with my laptop one has to pay through the nose. But still most of my machine's CPU cycles is idling and is used for rendering webpages. To make matters worse, most of the tools now a days are web only. Even if I want to use there's no app. There are exceptions of course such as IntelliJ, Figma etc., but sadly they should be the norm but aren't. To me, it feels as if because bandwidth got cheaper engineers just got lazy and stopped having to think to take advantage of client device's capabilities. It's gotten so bad that most of the apps wouldn't even start without internet connection.

by jerf 4y ago

vishnugupta 4y ago

> How much of their compute is in the cloud? Do they do all voice recognition in the cloud? Their voice assistance is constantly learning so a good chunk of voice recognition related computation takes place in cloud. However, even if the voice recognition is done locally, the vast majority of responses are streamed from the cloud. Only trivial requests such as "Set timer" can be handled locally. As of 2019, Amazon had sold 100 million Alexa devices. Whichever way I look at it, those many devices incur a big running cost at the backend cloud to service.

jerf 4y ago
Thank you for the info. This post is just further noodling, not any sort of disagreement. "Responses" streamed from the cloud differ a lot in terms of whether or not it's some sort of "API" response that could be a couple of cheap kilobytes of JSON, or a full text-to-speech audio file. I can't speak to Alexa, but I do use Google Maps, and it obviously has two modes for its TTS; a slightly nicer mode that streams from the cloud if you're on the network, but a perfectly serviceable lower quality one generated from the device if it can't get to the cloud at the time. I've often wondered why they don't just put the nicer cloud model on my device. Maybe when Maps was first starting it was a big difference, but now I'm sure my cell can run that "nice" model just as well. It's not like they're running a massive multi-billion parameter thing that only runs on the latest graphics cards (or, if they are, they aren't getting that much additional value for it), I'm sure it's something we could run locally just fine. Learning could always be done by sampling some things to send up.
- vishnugupta 4y ago
  For sure I agree with your overarching assessment though. The irony, as you noted, is even as consumer devices have gotten more and more powerful computation and processing is being pushed to the backend/cloud. For example; to get a EC2 that's comparable with my laptop one has to pay through the nose. But still most of my machine's CPU cycles is idling and is used for rendering webpages. To make matters worse, most of the tools now a days are web only. Even if I want to use there's no app. There are exceptions of course such as IntelliJ, Figma etc., but sadly they should be the norm but aren't. To me, it feels as if because bandwidth got cheaper engineers just got lazy and stopped having to think to take advantage of client device's capabilities. It's gotten so bad that most of the apps wouldn't even start without internet connection.