I've been playing with this lately and it's been loads of fun (on OSX in particular, on Windows/WSL I don't think it's rigged up with GPTQ yet). Of note is the

I've been playing with this lately and it's been loads of fun (on OSX in particular, on Windows/WSL I don't think it's rigged up with GPTQ yet). Of note is the experimental "Modelfile" that allows you to ship around an AI character or "stack" like a Docker image. I can really see the future around this shaping up to be really exciting, since I'll be able to hand you something much more replicable in terms of model, prompt, etc, and once we are able to chain calls and bundle embeddings, LoRa, etc., it seems like it could be really powerful.