I've finally managed to download the model and it seems to be working well for me. There's been some updates to the quantization code, so maybe if you do a 'gi

I've finally managed to download the model and it seems to be working well for me. There's been some updates to the quantization code, so maybe if you do a 'git pull && make' and rerun the quantization script it will work for you. I'm getting about 350ms per token with the 30B model.