3 ms·
What apps can one currently use to run them on say an iPhone? Only aware of the MLC one which has literally 3 old models only
by Havoc 2y ago
What apps can one currently use to run them on say an iPhone? Only aware of the MLC one which has literally 3 old models only
- 5cott0 2y agowat https://huggingface.co/mlc-ai https://huggingface.co/mlc-ai
- Havoc 2y agoOn my iphone there doesn’t seem to be an option to download more. Vaguely recall there being a button initially but don’t see it anymore
- pickettd 2y agoThe Android apk for MLC is updated frequently with recent models built-in. And a Samsung S24+ can comfortably run 7-8B models at reasonable speeds (10ish tokens/sec). https://llm.mlc.ai/docs/deploy/android.html https://llm.mlc.ai/docs/deploy/android.html
- woadwarrior01 2y agoI have an (mlc-llm based) app on the App Store that supports over 2 dozen models, including some recent ones.
- ukuina 2y agocnvrs runs GGUFs on iOS: https://testflight.apple.com/join/ERFxInZg https://testflight.apple.com/join/ERFxInZg