6 ms·

Phi-3 Technical a Highly Capable Language Model Locally on Your Phone

7 points by EveryPizza 2y ago

deleted 2y ago
[deleted]
pogue 2y ago
Anyone made or working on a UI for this for Android (or iPhone)?
- trao 2y ago
  Yeah, it’s on Private LLM. https://privatellm.app/blog/phi-3-now-available-on-iphone-and-ipad https://privatellm.app/blog/phi-3-now-available-on-iphone-an...
- sottol 2y ago
  I've been trying this app but haven't had any luck getting it to actually generate text yet: https://github.com/Mobile-Artificial-Intelligence/maid https://github.com/Mobile-Artificial-Intelligence/maid The UI looks nice and includes a native compilation of llama.cpp. My main phone's screen broke so I'm on an old Pixel 4 until it's repaired but I've had no luck getting 2-3GB models to run so far.
  - pogue 2y ago
    The APK doesn't include any of the models, so I'm not sure where you get them, where you're supposed to put them on the phone (same directory? Do they need to be compiled?) When I ask it a question it doesn't respond & when you attempt to switch models it starts looking for nearby devices. It also appears to be role play for some characters, so it's really not a GenAI chat bot, as far as I can tell. https://ibb.co/DRZhZcH https://ibb.co/DRZhZcH
    - sottol 2y ago
      You can just get the GGUF off hugging-face. Usually googling for "<modelname> gguf" gets you to the site, then choose a quantization that works for you. But as I said, not working on Pixel 4. Should get a new screen tomorrow and can't wait to try on my Pixel 6 Pro. Eg "phi-3 gguf" eventually take you here [1]. and you can download the Q4 quantized model on that page. [1] https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/blob/main/Phi-3-mini-4k-instruct-q4.gguf https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf...
      - pogue 2y ago
        I see, ty. I'm not familiar with quantization. Can you give me a brief explainer for that or point me in the right direction?
tommy_axle 2y ago
Looks like HuggingFace is experiencing issues again (https://status.huggingface.co/ https://status.huggingface.co/) same 503 errors like yesterday morning.

7 points by EveryPizza 2y ago

deleted 2y ago

[deleted]

pogue 2y ago

Anyone made or working on a UI for this for Android (or iPhone)?

trao 2y ago
Yeah, it’s on Private LLM. https://privatellm.app/blog/phi-3-now-available-on-iphone-and-ipad https://privatellm.app/blog/phi-3-now-available-on-iphone-an...
sottol 2y ago
I've been trying this app but haven't had any luck getting it to actually generate text yet: https://github.com/Mobile-Artificial-Intelligence/maid https://github.com/Mobile-Artificial-Intelligence/maid The UI looks nice and includes a native compilation of llama.cpp. My main phone's screen broke so I'm on an old Pixel 4 until it's repaired but I've had no luck getting 2-3GB models to run so far.
- pogue 2y ago
  The APK doesn't include any of the models, so I'm not sure where you get them, where you're supposed to put them on the phone (same directory? Do they need to be compiled?) When I ask it a question it doesn't respond & when you attempt to switch models it starts looking for nearby devices. It also appears to be role play for some characters, so it's really not a GenAI chat bot, as far as I can tell. https://ibb.co/DRZhZcH https://ibb.co/DRZhZcH
  - sottol 2y ago
    You can just get the GGUF off hugging-face. Usually googling for "<modelname> gguf" gets you to the site, then choose a quantization that works for you. But as I said, not working on Pixel 4. Should get a new screen tomorrow and can't wait to try on my Pixel 6 Pro. Eg "phi-3 gguf" eventually take you here [1]. and you can download the Q4 quantized model on that page. [1] https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/blob/main/Phi-3-mini-4k-instruct-q4.gguf https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf...
    - pogue 2y ago
      I see, ty. I'm not familiar with quantization. Can you give me a brief explainer for that or point me in the right direction?

tommy_axle 2y ago

Looks like HuggingFace is experiencing issues again (https://status.huggingface.co/ https://status.huggingface.co/) same 503 errors like yesterday morning.