With exllama, the number is "flexible" because it supports completely arbitrary quantization levels. 7B fits comfortably in 6GB. 70B will squeeze, into 24GB, a

With exllama, the number is "flexible" because it supports completely arbitrary quantization levels. 7B fits comfortably in 6GB. 70B will squeeze, into 24GB, albeit with significant (but not catastrophic) quality loss.