5 ms·

So it’s safe to assume in the next 10 years - AI will be running locally on every device from phones, laptops and even many embedded devices. Even robots- from

by taf2 3y ago

So it’s safe to assume in the next 10 years - AI will be running locally on every device from phones, laptops and even many embedded devices. Even robots- from street cleaning bots to helpful human assistants?

caycep 3y ago
I mean, diffusion models tend to be less computationally expensive than say CNNs or LLMs, so probably? And before than ppl ran SVMs, random forest, and other forms of non-gpu intensive ML algorithms locally as well...
ukuina 3y ago
Yes, every processor will have an AI core or two.
ranting-moth 3y ago
Probably not even 10 years.
hereonout2 3y ago
Yeah, that's happening right now really? There has been loads of developments in the mobile space already, in many ways lower powered arm devices are way more optimised for AI applications than the current crop of intel machines. This example, whilst impressive, feels way more in the "Doom running on a calculator" vein of progress though.
fragmede 3y ago
Every Google Home device is already running an ML model to do speech recognition to recognize the "hey Google" wake word, so sooner than 10 years. The Raspberry Pi Zero is a particularly underpowered device for this. Doing it on the Coral TPU accelerator plugged into a pi zero would take less than 30 mins. Doing it on an iPhone 15 would take less time. Doing it on a Pixel 8 would be faster. Not to diminish getting it to work on a Pi Zero, but that future is already here, just as soon as we figure out what to do with them. https://coral.ai/products/accelerator/ https://coral.ai/products/accelerator/
- dontwearitout 3y ago
  There's an ocean of difference between optimizing for a single wakeword and the class of models that are taking off today. I'm excited for more on-board processing, because it will mean less dependency on the cloud.
  - hereonout2 3y ago
    Siri came out 12 years ago, wake words are probably a bad example. Better examples are the magic eraser on my 2 generation old pixel phone or the fact llama2 runs genuinely fast on a Mac mini
    - joegibbs 3y ago
      Siri’s wake word stuff is also terrible, she gets constantly activated whenever I have my Apple Watch near running water, frying food or anything else that makes a white noise-type sound.
  - fragmede 3y ago
    I'm not going to argue that there isn't a difference when going from 0 -> 1 and 1 -> 10, or in this case, from 1.5b (Whisper-large) -> 1.7t parameters (gpt-4). But it's not like we don't know how to do it, so it won't take 10 years to get there.
nextworddev 3y ago
Probably less than 5 years, maybe 2
mrtksn 3y ago
The bottleneck is probably the availability of lithography machines that can make ubiquitous chips for processing that much data quickly enough without heating or drawing electricity too much. Not too far in the future every device will have some 5nm or better tech LLM chip inside and devices understanding natural language will be the norm. By dumb machines, people will mean machines that have to be programmed by people using ancient techniques where everything the machine is supposed to do is written step by step in a low level computer language like JavaScript. Nerds will be making demos of doing something incredibly fast by directly writing the algorithms by hand, and will be annoyed by the fact that something that can be done in 20 lines of code on few hundred MB of RAM in NodeJS now requires a terabyte of RAM. Dumb phone will be something like iPhone 15 pro or Pixel 8 Pro where you have separate apps for each thing you do and you can't simply ask the device to do it for you.
- herval 3y ago
  I really don’t buy that last paragraph. It’s the idea the AI Pin founders keep saying - you don’t want apps, the device chooses it for you, etc. In a world where you have economical choice, that’s quite hard to imagine: would there be no competition? What good is it for you if you can’t even choose what music app you want? Without _something_ like an app, how do you even know what you can interact with?
  - mrtksn 3y ago
    > Without _something_ like an app, how do you even know what you can interact with? You just tell what you need or want and the AI will suggest you options. >What good is it for you if you can’t even choose what music app you want? You don't want a music app, you want to listen to music. The AI will simply play it to you and any additional features that you might like will be available. Feel like doing karaoke? Just tell to the AI. Want more songs like this? Tell it to the AI. Not sure what you want? Tell the AI that you are not sure and ask for suggestions, maybe vaguely describe how you feel. You get the idea. > would there be no competition? There would, but not in the current sense. Maybe there would be experience designers and knowledge brokers. The AI might tell you that you will need a weather info subscription if you need something specific that is not provided by your current subscriptions. Maybe you want some specific recipe from a home lighting style from a designer? The AI can help you purchase and apply that. Maybe you want to try dating? It wouldn't be an app but a subscription you can join and the AI will handle the UX. However, I think there will be games because games are apps whose value comes from interaction with the app itself.
godelski 3y ago
Yes and no. Context matters. Will models of similar quality to the current LLaMA, GPT, and Stable Diffusion be running locally on devices and edge systems? Very likely. Will much higher quality models that still require compute incapable by such edge or consumer devices be available, sold as a service, and in high use? Also very likely. So expect current quality to make it to your devices but don't expect to necessarily move everything to local because the whole ecosystem will improve too. Overton window will shift and it's like asking if gaming will move to phones. In some ways yes, in other ways you're still going to want to buy that Playstation/Xbox/PC.
VyseofArcadia 3y ago
That's optimistic. We don't even run the humble word processor locally anymore. Everything is cloud, cloud, cloud.
- herval 3y ago
  You don’t run it because you don’t want to, not because the device can’t run it
herval 3y ago
“AI” is already running on every single laptop and phone. If you mean diffusion models (like this), then I’d say it’s 100% guaranteed they’ll run everywhere too, since they’ll get faster and more refined, and processing power keeps growing (it doesn’t even have to grow fast)

by taf2 3y ago

caycep 3y ago

I mean, diffusion models tend to be less computationally expensive than say CNNs or LLMs, so probably? And before than ppl ran SVMs, random forest, and other forms of non-gpu intensive ML algorithms locally as well...

ukuina 3y ago

Yes, every processor will have an AI core or two.

ranting-moth 3y ago

Probably not even 10 years.

hereonout2 3y ago

Yeah, that's happening right now really? There has been loads of developments in the mobile space already, in many ways lower powered arm devices are way more optimised for AI applications than the current crop of intel machines. This example, whilst impressive, feels way more in the "Doom running on a calculator" vein of progress though.

fragmede 3y ago

Every Google Home device is already running an ML model to do speech recognition to recognize the "hey Google" wake word, so sooner than 10 years. The Raspberry Pi Zero is a particularly underpowered device for this. Doing it on the Coral TPU accelerator plugged into a pi zero would take less than 30 mins. Doing it on an iPhone 15 would take less time. Doing it on a Pixel 8 would be faster. Not to diminish getting it to work on a Pi Zero, but that future is already here, just as soon as we figure out what to do with them. https://coral.ai/products/accelerator/ https://coral.ai/products/accelerator/

dontwearitout 3y ago
There's an ocean of difference between optimizing for a single wakeword and the class of models that are taking off today. I'm excited for more on-board processing, because it will mean less dependency on the cloud.
- hereonout2 3y ago
  Siri came out 12 years ago, wake words are probably a bad example. Better examples are the magic eraser on my 2 generation old pixel phone or the fact llama2 runs genuinely fast on a Mac mini
  - joegibbs 3y ago
    Siri’s wake word stuff is also terrible, she gets constantly activated whenever I have my Apple Watch near running water, frying food or anything else that makes a white noise-type sound.
- fragmede 3y ago
  I'm not going to argue that there isn't a difference when going from 0 -> 1 and 1 -> 10, or in this case, from 1.5b (Whisper-large) -> 1.7t parameters (gpt-4). But it's not like we don't know how to do it, so it won't take 10 years to get there.

nextworddev 3y ago

Probably less than 5 years, maybe 2

mrtksn 3y ago

The bottleneck is probably the availability of lithography machines that can make ubiquitous chips for processing that much data quickly enough without heating or drawing electricity too much. Not too far in the future every device will have some 5nm or better tech LLM chip inside and devices understanding natural language will be the norm. By dumb machines, people will mean machines that have to be programmed by people using ancient techniques where everything the machine is supposed to do is written step by step in a low level computer language like JavaScript. Nerds will be making demos of doing something incredibly fast by directly writing the algorithms by hand, and will be annoyed by the fact that something that can be done in 20 lines of code on few hundred MB of RAM in NodeJS now requires a terabyte of RAM. Dumb phone will be something like iPhone 15 pro or Pixel 8 Pro where you have separate apps for each thing you do and you can't simply ask the device to do it for you.

herval 3y ago
I really don’t buy that last paragraph. It’s the idea the AI Pin founders keep saying - you don’t want apps, the device chooses it for you, etc. In a world where you have economical choice, that’s quite hard to imagine: would there be no competition? What good is it for you if you can’t even choose what music app you want? Without _something_ like an app, how do you even know what you can interact with?
- mrtksn 3y ago
  > Without _something_ like an app, how do you even know what you can interact with? You just tell what you need or want and the AI will suggest you options. >What good is it for you if you can’t even choose what music app you want? You don't want a music app, you want to listen to music. The AI will simply play it to you and any additional features that you might like will be available. Feel like doing karaoke? Just tell to the AI. Want more songs like this? Tell it to the AI. Not sure what you want? Tell the AI that you are not sure and ask for suggestions, maybe vaguely describe how you feel. You get the idea. > would there be no competition? There would, but not in the current sense. Maybe there would be experience designers and knowledge brokers. The AI might tell you that you will need a weather info subscription if you need something specific that is not provided by your current subscriptions. Maybe you want some specific recipe from a home lighting style from a designer? The AI can help you purchase and apply that. Maybe you want to try dating? It wouldn't be an app but a subscription you can join and the AI will handle the UX. However, I think there will be games because games are apps whose value comes from interaction with the app itself.

godelski 3y ago

Yes and no. Context matters. Will models of similar quality to the current LLaMA, GPT, and Stable Diffusion be running locally on devices and edge systems? Very likely. Will much higher quality models that still require compute incapable by such edge or consumer devices be available, sold as a service, and in high use? Also very likely. So expect current quality to make it to your devices but don't expect to necessarily move everything to local because the whole ecosystem will improve too. Overton window will shift and it's like asking if gaming will move to phones. In some ways yes, in other ways you're still going to want to buy that Playstation/Xbox/PC.

VyseofArcadia 3y ago

That's optimistic. We don't even run the humble word processor locally anymore. Everything is cloud, cloud, cloud.

herval 3y ago
You don’t run it because you don’t want to, not because the device can’t run it

herval 3y ago

“AI” is already running on every single laptop and phone. If you mean diffusion models (like this), then I’d say it’s 100% guaranteed they’ll run everywhere too, since they’ll get faster and more refined, and processing power keeps growing (it doesn’t even have to grow fast)