3 ms·
No gamers need such high VRAM, if you're buying Gaming cards for ML work you're doing it wrong.
by vonneumannstan 2y ago
No gamers need such high VRAM, if you're buying Gaming cards for ML work you're doing it wrong.
- riskable 2y agoIt's Nvidia that considers them, "gaming cards". The market decides their use in reality though. Their strategy is to sell lower-VRAM cards to consumers with the understanding that they can make more money on their more expensive cards for professionals/business. By doing this, though they're creating a gap in the market that their competitors could fill (in theory). Of course, this assumes their competitors have half a brain cell (I'm looking at YOU, Intel! For fuck's sake give us a 64GB ARC card already!).
- vonneumannstan 2y agoAnd if you buy the cards that Nvidia says are for gaming and then complain that they don't have good specs for ML, who is the fool exactly?
- epolanski 2y agoGames already exceed 16 GBs at 4k from years.
- throwaway48476 2y agoI exceed 16GB in Chrome.
- brokenmachine 2y agoThat says more about Chrome than anything else. I use Firefox and have an 8Gb card and only encounter problems when I have more than about 125 windows with about 10-20 tabs each. Yes, I am a tab hoarder. And yes, I am going to buy a 16Gb card soon. :P
- throwaway48476 2y ago1200 tabs isn't that many
- brokenmachine 2y agoThat's why I need to get a 16Gb card. :P
- vonneumannstan 2y agoSystem Ram != GPU VRAM
- throwaway314155 2y ago> Gaming cards for ML work you're doing it wrong lol okay. "doing it wrong" for a tenth of the cost.
- moogly 2y agoAnd screwing gamers over by raising the prices by 2x. Fuck that.
- throwaway314155 2y agoBelieve it or not, it's possible to be interested in both machine learning and videogames. That's ignoring the notion that it's somehow how screwing over gamers. Buy a fucking AMD card. They're great at gaming and you don't need CUDA anyways. Enjoy the long-term acceleration of GPU performance increases you're getting by the way. All that stuff comes from innovations made for workstation/DL setups.
- moogly 2y agoGet an AMD GPU? Said no one ever.
- muchosandwich 2y agoIt seems like the 90-series cards are going to be targeting prosumers again. People who play games but may use their desktop for work as well. Some people are doing AI training on some multiple of 3090/4090 today but historically the Titan cards that preceded the 90s cards were used by game developers, video editors and other content developers. I think NVIDIA is going to try to move the AI folks onto Digits and return the 90-series back to its roots but also add in some GenAI workloads.
- sfmike 2y agoforget the post but some dude had a startup piping his 3090 to use via cloudflare tunnels for his ai saas making 5 figures a month off of his 1k gpu that handled the work load, I'd say he was doing it more then right.
- vonneumannstan 2y agoAnd if his volume grows 100x should we expect him to run his company off gaming gpus? Just because you can do something doesn't mean you should or that it's ideal. There's a reason large companies are buying H100s and not 4090s. Despite what you guys think, serious ML work isn't done on the consumer cards for many reasons: FP16/FP8 TFLOPS, NVLINK, power consumption, physical space, etc.
- Sohcahtoa82 2y agoMS Flight Simulator 2024 can consume...who knows how much. I know my 10 GB 3080 ran out of VRAM playing it on Ultra, and i was getting as low as 2 fps because I'm bottlenecked by the PCI-Express bus as it has to constantly page the entire working set of textures and models in and out. I'm getting a 5090 for that, plus I want to play around with 7B parameter LLMs and don't want to quantize below 8 bits if I can help it.
- thecosmicfrog 2y agoI've regularly exceeded 24 GiB of VRAM in Microsoft Flight Simulator 2024. Imagine a huge airport environment with high levels of detail, plus AI aircraft in the ground and sky. Then, on top of that, terrain and textures of the surrounding environment. And that's at 1440p, not even 4K. The resulting stutters are... not pretty.
- segmondy 2y agotell us how to do it right.
- vonneumannstan 2y agoGet your daddy's credit card and buy H100s like a serious person.