3 ms·
This is great, thanks for putting this together. Haven't followed it through yet, but does this model run successfully on an iPhone? My 9 year old ran a Qwen
by thedeep_mind 1y ago
This is great, thanks for putting this together.
Haven't followed it through yet, but does this model run successfully on an iPhone?
My 9 year old ran a Qwen 0.6B model using ollama quite well, anything else was too slow to offer a good UX.
- SparkyMcUnicorn 1y agoMLC[0] indicates that it can run models in the 8B range on iOS, but 1-3B sounds more reasonable to me. [0] https://llm.mlc.ai/docs/deploy/ios.html#bring-your-own-model https://llm.mlc.ai/docs/deploy/ios.html#bring-your-own-model
- parpfish 1y agoOh, a nine year old PHONE. I was thinking there was a fourth grader out there deploying models when at that age I was still learning multiplication tables.
- NetOpWibby 1y agoMy son just turned 9 today so I was like, "Wow! I wonder if my kid would be interested in doing this?"