> That said, I’m still skeptical about how practical this really is for most people. I'm running Open WebUI for months now for me and some friends as a front-e

> That said, I’m still skeptical about how practical this really is for most people. I'm running Open WebUI for months now for me and some friends as a front-end to one of the API providers (deepinfra in my case, but there are many others, see https://artificialanalysis.ai/ https://artificialanalysis.ai/). Having 1.58-bit is very practical for me. I'm looking much forward to the API provider adding this model to their system. They also added a Llama turbo (also quantized) a few months back so I have good hopes.