3 ms·
I've a limited understanding of the topic, but would this allow to run an LLM in a mobile phone in offline mode? If that's feasible, it'd pave the way to lots o
by remoquete 3y ago
I've a limited understanding of the topic, but would this allow to run an LLM in a mobile phone in offline mode? If that's feasible, it'd pave the way to lots of interesting applications, such as AI-assisted content moderation without having to phone back confidential data.
- hobofan 3y agoYes, this may (significantly) improve that. Even without that, you can run LLMs already on mobile phones, the question is just how big of a model and how strongly quantized, and if the few models that remain produce good enough results. E.g. there was a GH Discussion about running LLMs on Apple A-series chips (iPhone) posted here yesterday: https://news.ycombinator.com/item?id=38703161 https://news.ycombinator.com/item?id=38703161
- frankthepickle 3y agoI'm not sure but I think that's a selling point of the new pixel
- keivanalizadeh 3y agoYes, the goal is at the end to run larger models on the phone as phones have very limited DRAM.