I take it you never run models locally then? Local models that can run on reasonable consumer hardware have gotten to the point of being very useful, but they s

I take it you never run models locally then? Local models that can run on reasonable consumer hardware have gotten to the point of being very useful, but they still occasionally are tripped up with syntax errors. Especially if you quantize the model and KV.