4 ms·

In 2-3 years, devices will run this locally.

• dingnuts 2y ago
  Oh, they will? Someone else on this site told me they would be running locally in 2-3 years, back in 2022. Someone on this site also assured me ten years ago that we'd have full self driving by 2020.
  • dbmnt 2y ago
    I mean, the iPhone 16 is already running LLMs locally. So yeah.
  • getwiththeprog 2y ago
    Firefox released a very easy to deploy local LLM around a year ago. https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile https://ai-guide.future.mozilla.org/content/running-llms-locally/ https://ai-guide.future.mozilla.org/content/running-llms-loc... It works.
  • kbelder 2y ago
    >locally in 2-3 years, back in 2022. That's pretty much accurate. Mine's been up and running on my clunky old home machine for 6 months, and I just this morning overheard a couple coworkers talking about the local llms they're running. Right now a substantial minority of PCs could run useful models. Usage is at the stage where it's small and growing rapidly. Early adapter phase, but all you need is a relatively modest years-old GPU to handle something like this model. What isn't done, and probably won't be for a while, is a nice generic framework so we can tie their local llm into all sorts of local apps and processes. The big players all want us to use cloud services.
• nullc 2y ago
  It's easy to run models of this size locally already.