6 ms·

But wont it be that in real life no one would want to run a voice command which consumes lot of CPU and battery as opposed to making a network call to a service

But wont it be that in real life no one would want to run a voice command which consumes lot of CPU and battery as opposed to making a network call to a service which has this model hosted ? Agreed that this can always be improved and hardware can get more efficient and better to but at the end of the day, would it ever be better then an API call ?

Tepix 4y ago
Privacy concerns are justified. It's not just that, this can also work completely offline.
- irusensei 4y ago
  I'm looking forward to run stuff like this online. Using bigtech corporate souls SaaS AI is just pure dystopia material. It's even better that we are talking about a relatively low power machine here. Maybe can operate offered.
  - Tepix 4y ago
    You mean offline?
sebzim4500 4y ago
Ultimately, no amount of technology will ever beat the speed of light. Running locally will always have a lower latency floor.
- brookst 4y ago
  Theoretically yes. But in the real world, no. Simple thought experiment: you want to know how many tons of copper are mined in the US each year. Lowest possible latency is calculating this in your head, most likely using data you don’t have. Looking it up online is a lot, lot faster. In some far future world maybe every transistor will include the sum total of human knowledge up to the nanosecond, but that’s a pretty far future. There are many things where running locally means a higher latency floor.
nmcfarl 4y ago
I live in eastern Oregon on a property with no cell service. I use Siri a lot, mainly to add reminders, and sometimes I try to use Siri when I'm out at the greenhouse, which is just past the edge of the mesh network. I would love for those reminders to get added - even if it burnt battery. And more generally I would love for people writing apps to consider that phones don't always have service - as would my neighbors.
brucethemoose2 4y ago
Its still cheaper to run a free model on a competitive "dumb" cloud host than buy a service only one company provides.
q7xvh97o2pDhNrh 4y ago
There are still a few people in the world who don't have always-on gigabit internet access everywhere they go.
grandiego 4y ago
"There is No Reason for Any Individual To Have a Computer in Their Home"

Tepix 4y ago

Privacy concerns are justified. It's not just that, this can also work completely offline.

irusensei 4y ago
I'm looking forward to run stuff like this online. Using bigtech corporate souls SaaS AI is just pure dystopia material. It's even better that we are talking about a relatively low power machine here. Maybe can operate offered.
- Tepix 4y ago
  You mean offline?

sebzim4500 4y ago

Ultimately, no amount of technology will ever beat the speed of light. Running locally will always have a lower latency floor.

brookst 4y ago
Theoretically yes. But in the real world, no. Simple thought experiment: you want to know how many tons of copper are mined in the US each year. Lowest possible latency is calculating this in your head, most likely using data you don’t have. Looking it up online is a lot, lot faster. In some far future world maybe every transistor will include the sum total of human knowledge up to the nanosecond, but that’s a pretty far future. There are many things where running locally means a higher latency floor.

nmcfarl 4y ago

I live in eastern Oregon on a property with no cell service. I use Siri a lot, mainly to add reminders, and sometimes I try to use Siri when I'm out at the greenhouse, which is just past the edge of the mesh network. I would love for those reminders to get added - even if it burnt battery. And more generally I would love for people writing apps to consider that phones don't always have service - as would my neighbors.

brucethemoose2 4y ago

Its still cheaper to run a free model on a competitive "dumb" cloud host than buy a service only one company provides.

q7xvh97o2pDhNrh 4y ago

There are still a few people in the world who don't have always-on gigabit internet access everywhere they go.

grandiego 4y ago

"There is No Reason for Any Individual To Have a Computer in Their Home"