0.6b - 1.5b models are surprisingly good for RAG, and should work reasonably well even on old toasters. Then there's gemma 3n which runs fine-ish even on mobile

0.6b - 1.5b models are surprisingly good for RAG, and should work reasonably well even on old toasters. Then there's gemma 3n which runs fine-ish even on mobile phones.