15 ms·
it doesn't matter how much compute you have if you don't have enough vram to run the model.
by coconut08 3y ago
it doesn't matter how much compute you have if you don't have enough vram to run the model.
- Zambyte 3y agoExactly. My friend was telling me that I was making a mistake for getting a 7900 XTX to run language models, when the fact of the matter is the cheapest NVIDIA card with 24 GB of VRAM is over 50% more expensive than the 7900 XTX. Running a high quality model at like 80 tps is way more important to me than running a way lower quality model at like 120 tps.
- mejutoco 3y agoI cannot parse this. The Radeon RX 7900 XTX also has 24GB of vram, so how does it help you run higher quality models? I would understand if it had more ram.
- Zambyte 3y agoOnly the RX 7900 XTX has 24 GB of VRAM at its price point. If I went with an NVIDIA card, I would either have to spend over 50% more on the card, or use much worse models to fit on their 16 GB cards.
- deleted 3y ago[deleted]