You can also target WASM. Depending on the model size, it can be really good. Here are some examples of quantized models. Vision Model https://huggingface.co

You can also target WASM. Depending on the model size, it can be really good. Here are some examples of quantized models. Vision Model https://huggingface.co/spaces/radames/Candle-Moondream-2 https://huggingface.co/spaces/radames/Candle-Moondream-2 Blip Image Captioning https://huggingface.co/spaces/radames/Candle-BLIP-Image-Captioning https://huggingface.co/spaces/radames/Candle-BLIP-Image-Capt... Microsoft Phi 2 https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-demo?model=phi_2_0_q4k https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-...