30 ms·
And this will fit in 6GB of VRAM with very little quality loss: https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6-exl2 https://huggingface.co/LoneSt
by brucethemoose2 3y ago
And this will fit in 6GB of VRAM with very little quality loss:
https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6-exl2 https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6...