3 ms·
Current LLMs have vram and power requirements that mean it’s unlikely we’ll see a decent LLM running on phones without a step change in technology.
by michaelt 3y ago
Current LLMs have vram and power requirements that mean it’s unlikely we’ll see a decent LLM running on phones without a step change in technology.
- sigilis 3y agoI currently have mistral 7b running on my phone through the MLCChat app. The performance isn’t too bad for something running on phone hardware. I’d consider that decent, if not world beating.