3 ms·
I'm sure this question has been asked before, but why not launch a GPU with more but slower ram? That would fit bigger models while still affordable...
by Gravityloss 2y ago
I'm sure this question has been asked before, but why not launch a GPU with more but slower ram? That would fit bigger models while still affordable...
- varelse 2y ago[dead]
- ChocolateGod 2y agoBecause then you would have less motivation to buy the more expensive GPUs.
- fleischhauf 2y agothey absolutely can build gpus with larger vram, they just don't have the competition to have to do so. it's much more profitable this way.
- TeMPOraL 2y agoWhy would you need it for? Not gaming for sure. AI, you say? Then fork up the cash. That's Nvidia's current MO. There's more demand for GPUs for AI than there are GPUs available, and most of that demand still has stupid amounts of money behind it (being able to get grants, loans or investment based on potential/hype) - money that can be captured by GPU vendors. Unfortunately, VRAM is the perfect discriminator between "casual" and "monied" use. (This is not unlike the "SSO tax" - single sign-on is pretty much the perfect discriminator between "enterprise use" and "not enterprise use".)
- andrewstuart 2y agoDid you miss the news about AMD Halo Strix? More than twice as fast as Nvidia 4090 for AI. Launched last week.
- Gravityloss 2y agoI indeed was not aware, thanks
- coolspot 2y ago> More than twice as fast as Nvidia 4090 for AI. Not in memory bandwidth which is all that matter for LLM inference.