5 ms·
Why can't this run on older devices?
by RockRobotRock 3y ago
Why can't this run on older devices?
- Legend2440 3y agoIt's an LLM, older devices don't have the juice. Newer devices only barely have the juice.
- RockRobotRock 3y agoWhat does that mean, though? Is it a VRAM thing? I have a 20 series card with 11 GB and okay performance in CUDA for things like OpenAI Whisper. I think it could run it, albeit slowly.
- imtringued 3y agoIt's always a VRAM thing from this point on. Compute will always be abundant in relation to memory capacity and bandwidth. The only places were this doesn't count is I low power situations such as embedded, where you might intentionally choose a small model to save power.
- RockRobotRock 3y agoThanks for the insight. Time for an upgrade.