3 ms·
So why does anyone care much at all about GPU RAM size for AI?
by andrewstuart 3y ago
So why does anyone care much at all about GPU RAM size for AI?
- smoldesu 3y agoYour guess is as good as mine. I'm running an AI-powered Discord bot off a free ARM server from Oracle, I cringe when I hear founders complaining about the "GPU shortage" or "Nvidia monopoly". I'm convinced most people just buy whatever can fit the 70b model because... apparently that's what they think they need. Unless you're running a multitenant GPU instance or aim to beat OpenAI at their own game, you probably can get by with consumer grade hardware.