3 ms·

Agreed. On the downside, we have to acknowledge that it is hugely inefficient for everyone to own expensive hardware that has to sit idle most of the time beca

Agreed. On the downside, we have to acknowledge that it is hugely inefficient for everyone to own expensive hardware that has to sit idle most of the time because it would otherwise drain the battery. Where low latency is not an absolute necessity, the economic pull of the cloud will be tremendous, especially if mobile networks become ubiquitous and fast.

vinay_ys 4y ago
That's a weak argument. Lots of hardware sits idle in the cloud as well. And on your phone its not expensive. In fact, the $/tflop is cheaper on phone than in the cloud – cloud has to deal with all kinds of complexity that you assume away in your local single-tenant phone context.
- fauigerzigerk 4y ago
  I wouldn't be so sure. A quick web search brings up average server utilisation numbers for large-scale cloud providers between 45% and 65%. That's probably an order of magnitude or two higher than what you could do on a mobile device without absolutely annihilating the battery.
nl 4y ago
Inactive silicon isn't inefficient. And it's not even particularly wasteful to produce. While there are lots of devices the amount of materials in each chip is incredibly small. In terms of financial cost the material cost is so low that most chip vendors include features in low end chips that are disabled but shipped anyway.
- moffkalast 4y ago
  Aren't they usually disabled because they don't perform up to spec?
- fauigerzigerk 4y ago
  That's an interesting point, but I wonder how the cost structure will change as AI becomes a bigger part of the feature set we expect. The battery is a key part as well. Doing a lot of on-device processing requires a bigger battery and/or more frequent battery replacements.
ClumsyPilot 4y ago
> On the downside, we have to acknowledge that it is hugely inefficient for everyone to own expensive hardware that has to sit idle most of the time Expensive hardware like millions of 5G/6G modems, stations and fiber optic cables nessesary to send vast quantity of photos and videos to the cloud for analysis? ait is more expensive to build out bandwidth than to give everyone compute Also average car is unused 95/% of the time, so by this principle everyone should take a bus
- fauigerzigerk 4y ago
  >ait is more expensive to build out bandwidth than to give everyone compute I'm not sure about that. Storing all data on all devices (not just user data but also the pretrained models) means higher data transfer volumes. On the other hand, syncing data doesn't require low latency and a lot of the data can be transferred over cheaper landline connections rather than mobile towers. Right now though the default appears to be to upload everything to the cloud right away, regardless of where the data is processed. And there's also communication and media streaming, which consumes the vast majority of network resources. Net net I'm not sure whether on-device vs cloud processing will make a big difference for networking costs one way or another. >Also average car is unused 95/% of the time, so by this principle everyone should take a bus Yes, and it is hugely inefficient. People who own a car (I don't) are doing it for the benefits it has, not because it's efficient.
- gurleen_s 4y ago
  > Also average car is unused 95/% of the time, so by this principle everyone should take a bus That’s exactly right.

vinay_ys 4y ago

That's a weak argument. Lots of hardware sits idle in the cloud as well. And on your phone its not expensive. In fact, the $/tflop is cheaper on phone than in the cloud – cloud has to deal with all kinds of complexity that you assume away in your local single-tenant phone context.

fauigerzigerk 4y ago
I wouldn't be so sure. A quick web search brings up average server utilisation numbers for large-scale cloud providers between 45% and 65%. That's probably an order of magnitude or two higher than what you could do on a mobile device without absolutely annihilating the battery.

nl 4y ago

Inactive silicon isn't inefficient. And it's not even particularly wasteful to produce. While there are lots of devices the amount of materials in each chip is incredibly small. In terms of financial cost the material cost is so low that most chip vendors include features in low end chips that are disabled but shipped anyway.

moffkalast 4y ago
Aren't they usually disabled because they don't perform up to spec?
fauigerzigerk 4y ago
That's an interesting point, but I wonder how the cost structure will change as AI becomes a bigger part of the feature set we expect. The battery is a key part as well. Doing a lot of on-device processing requires a bigger battery and/or more frequent battery replacements.

ClumsyPilot 4y ago

> On the downside, we have to acknowledge that it is hugely inefficient for everyone to own expensive hardware that has to sit idle most of the time Expensive hardware like millions of 5G/6G modems, stations and fiber optic cables nessesary to send vast quantity of photos and videos to the cloud for analysis? ait is more expensive to build out bandwidth than to give everyone compute Also average car is unused 95/% of the time, so by this principle everyone should take a bus

fauigerzigerk 4y ago
>ait is more expensive to build out bandwidth than to give everyone compute I'm not sure about that. Storing all data on all devices (not just user data but also the pretrained models) means higher data transfer volumes. On the other hand, syncing data doesn't require low latency and a lot of the data can be transferred over cheaper landline connections rather than mobile towers. Right now though the default appears to be to upload everything to the cloud right away, regardless of where the data is processed. And there's also communication and media streaming, which consumes the vast majority of network resources. Net net I'm not sure whether on-device vs cloud processing will make a big difference for networking costs one way or another. >Also average car is unused 95/% of the time, so by this principle everyone should take a bus Yes, and it is hugely inefficient. People who own a car (I don't) are doing it for the benefits it has, not because it's efficient.
gurleen_s 4y ago
> Also average car is unused 95/% of the time, so by this principle everyone should take a bus That’s exactly right.