2 ms·
what quantization did u try ? u can use Q4 quantization, im pretty sure that 48GB would be enough
by yassa9 4mo ago
what quantization did u try ? u can use Q4 quantization, im pretty sure that 48GB would be enough
- sgt101 4mo ago8bits is fine.... I was talking full bore.