2 ms·
It's true that a lot of ML development has taken place on gaming GPUs, over the past decade and a half. And you can certainly get a long way with gaming GPUs in
by michaelt 3y ago
It's true that a lot of ML development has taken place on gaming GPUs, over the past decade and a half. And you can certainly get a long way with gaming GPUs in a lot of areas of ML - 8GB is enough to run things like stable diffusion, just about.
However, some ML models (such as LLMs) demand a lot of vram to train. Simultaneously, a lot of gamers have been unimpressed by the latest generations of gaming GPUs due to the limited vram.
nvidia's GTX-1070 released in 2016 with 8 GB of vram and an MSRP of $379.
Then the RTX-2060 Super, with 8 GB of vram and an MSRP of $399.
Then the RTX-3060 Ti, with 8 GB of vram and an MSRP of $399.
Then the RTX-4060 Ti with - you guessed it - 8 GB of vram and an MSRP of $399.
Some people think nvidia is deliberately being miserly with vram on gaming cards, to try and force ML users onto the $$$$$ data centre GPUs.
I suspect this is what andrewstuart means by "Nvidia is pushing upmarket" - that they're deliberately letting their gaming products languish, in pursuit of the data centre market.
- dragonwriter 3y ago> 8GB is enough to run things like stable diffusion, just about. 4GB (from personal experience, 2GB from what I’ve heard) is enough to run stable diffusion 1.x/2.x with the major current consumer UIs and the optimizations they make under the hood. Not sure about SDXL. The original first-party inferencing code takes more. > Some people think nvidia is deliberately being miserly with vram on gaming cards, to try and force ML users onto the $$$$$ data centre GPUs. And, sure, as long as there is no real competition, that kind fo segmentation makes sense. OTOH, if there is competition and desktop ML demand, it will make progressively less sense. So, the question seems to be, will there be competition that matters?