5 ms·
Not to mention the text-only 0.8GB version. Just crazy. You can have basic real-time conversations on-device that's video and audio aware now.
by reactordev 4mo ago
Not to mention the text-only 0.8GB version. Just crazy. You can have basic real-time conversations on-device that's video and audio aware now.
- simonw 4mo agoHave you seen a 0.8GB model file floating around yet? I couldn't find one earlier.
- reactordev 4mo agoI think this is the one but it’s 0.8GB VRAM not 0.8GB size. https://huggingface.co/google/gemma-4-E2B-it-qat-mobile-ct https://huggingface.co/google/gemma-4-E2B-it-qat-mobile-ct But they could be cooking up a smaller one because the model card lists the Q_4 quants as being bigger than the mobile or text-only so I think we’ll need to wait for the Q_2_Distilled_Mobile_Textformer version. Still, just amazing work.
- yalok 4mo ago0.8GB is for text only. It's more like ~1.1GB if you include video/audio encoder
- reactordev 4mo agoAnd your point is what? That’s more than 0.8GB text only if you include more than, text-only?
- ranguna 4mo agoTheir point is that OP used the same dot separated phrase to point out that there's a 0.8GB model and an audio/image model on device. Which reads weird.
- madduci 4mo agoWhere is it? On ollama I see only the bigger one
- reactordev 4mo agoI don’t use ollama, can you pull from HF?
- viccis 4mo agoI'll be honest with you. My main ask for on device AI is that when I am typing "Going out for a quick j" it corrects to "jog" and not "Jonathan". I don't think it needs that many gigabytes.
- taffydavid 4mo agoWho doesn't enjoy a quick Jonathan now and then. But seriously, wouldn't productive text on a 90s cell phone pass this test?
- reactordev 4mo agoThe autocomplete of a decade ago is better than what we have now. It’s harder now because emojis and draw-to-type as well as pen input. We didn’t have these things 14 years ago when “I’ll be right back” could be expanded from “I’ll b ri ba”