3 ms·
It's an LLM, older devices don't have the juice. Newer devices only barely have the juice.
by Legend2440 3y ago
It's an LLM, older devices don't have the juice.
Newer devices only barely have the juice.
- RockRobotRock 3y agoWhat does that mean, though? Is it a VRAM thing? I have a 20 series card with 11 GB and okay performance in CUDA for things like OpenAI Whisper. I think it could run it, albeit slowly.
- imtringued 3y agoIt's always a VRAM thing from this point on. Compute will always be abundant in relation to memory capacity and bandwidth. The only places were this doesn't count is I low power situations such as embedded, where you might intentionally choose a small model to save power.
- RockRobotRock 3y agoThanks for the insight. Time for an upgrade.