You can just get the GGUF off hugging-face. Usually googling for "<modelname> gguf" gets you to the site, then choose a quantization that works for you. But as

You can just get the GGUF off hugging-face. Usually googling for "<modelname> gguf" gets you to the site, then choose a quantization that works for you. But as I said, not working on Pixel 4. Should get a new screen tomorrow and can't wait to try on my Pixel 6 Pro. Eg "phi-3 gguf" eventually take you here [1]. and you can download the Q4 quantized model on that page. [1] https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/blob/main/Phi-3-mini-4k-instruct-q4.gguf https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf...