Thanks for the tip about exllama, I've been on the lookout for a readable python implementation to play with that is also fast and has support for quantized dat

Thanks for the tip about exllama, I've been on the lookout for a readable python implementation to play with that is also fast and has support for quantized datasets.