The problem is that it's not consistent enough for a good demo. Not even two different models, but even two different fine tunes of the same base model may be w

The problem is that it's not consistent enough for a good demo. Not even two different models, but even two different fine tunes of the same base model may be wildly differently affected by quantization. It can range from making hardly a difference to complete garbage output.