4 ms·
A 128GB card could run llama 3.1B 70B with FP8. It is a huge increase in quality over what the current 24GB cards can do.
by ryao 2y ago
A 128GB card could run llama 3.1B 70B with FP8. It is a huge increase in quality over what the current 24GB cards can do.