3 ms·
Yes, these models are identical in architecture to GPT-NeoX which has already been GPTQ quantized. So you can quantize these with GPT-NeoX quantization scripts
by MacsHeadroom 4y ago
Yes, these models are identical in architecture to GPT-NeoX which has already been GPTQ quantized. So you can quantize these with GPT-NeoX quantization scripts out of the box.
There's also support in llama.cpp already.