4 ms·
well it was a long shot anyway but it doesn’t seem to work on mobile. (tried on iOS safari on iPhone 11 pro) a 1B model should be able to run in the RAM constr
by jedbrooke 2y ago
well it was a long shot anyway but it doesn’t seem to work on mobile. (tried on iOS safari on iPhone 11 pro)
a 1B model should be able to run in the RAM constraints of a phone(?) if this is supported soon this would actually be wild. Local LLMs in the palm of your hands
- nromiun 2y agoI don't know about this model but people have been running local models in Android phones for years now. You just need a large amount of ram (8-12 GB), ggml and Termux. I tried it once with a tiny model and it worked really well.
- kittikitti 2y agoThis is from Reddit, what were you expecting?
- bla3 2y agoThis needed a 4 GB renderer process and about that much additional memory use in the GPU process for me, in Chrome.
- rahimnathwani 2y agoLocal LLMs in the palm of your hands https://apps.apple.com/us/app/mlc-chat/id6448482937 https://apps.apple.com/us/app/mlc-chat/id6448482937