5 ms·
You over estimate the 'semi-pro' market for graphics cards. Gamers are barely willing to pay for 20GB. There's no market for consumer cards with an order of mag
by Brananarchy 3y ago
You over estimate the 'semi-pro' market for graphics cards. Gamers are barely willing to pay for 20GB. There's no market for consumer cards with an order of magnitude more RAM until games are built to use that memory.
- cinntaile 3y ago48GB cards would sell like hotcakes. The problem is that they would sell way less cards aimed at professionals, where they have much higher margins.
- AnthonyMouse 3y agoIntel doesn't sell a lot of graphics cards whatsoever though. Be the first to offer 64GB of VRAM for under $1000 and that could change pretty fast.
- xadhominemx 3y agoNot without CUDA unfortunately.
- KeplerBoy 3y agoDon't underestimate the amount of shit people would be willing to deal with to make stuff work. A capable GPU with 24+ GB would sell if it significantly undercuts Nvidia. Just look at geohot building his tinyboxes with AMD cards.
- xadhominemx 3y agoI would personally love that project but there are already so many versioning issues in the space it would be a nightmare if ROCm randomly broke things all the time.
- KeplerBoy 3y agoI agree, ROCm seems to be a mess from the outside, but I'm glad people are putting in the effort.
- AnthonyMouse 3y agoAnd we're talking about Intel here. AMD is going to price competitively against Nvidia but they'd still rather you buy a $20,000 MI300 than a hypothetical 128GB Radeon for $2000. Intel could very easily just put a buttload of VRAM on their existing GPUs to stick it to their competitors and make out like bandits. All they'd have to do is charge a Big markup instead of an Enterprise markup. And Intel has a better history of not making broken libraries.
- cyanydeez 3y agoa lot of the true AI value is context window size limited, not compute limited.
- imtringued 3y agoAssuming 50 input tokens per second, you could still be waiting ten minutes for a full 32k token prompt. What you are talking about is highly optimized inference using accelerators, batching and speculative decoding to achieve high throughout. Once you have that then compute is irrelevant except in terms of cost, but if all you have is a small consumer grade GPU you will be compute limited at the extreme limits of your context window.
- cyanydeez 3y agoI'm taking about context in, not out. reports I have and the knowledge base I want answers from are 500-1000k tokens. I don't need long answers, I need by site specific knowledge base
- ianbutler 3y agoThis is for ML, not gamers. There is an entirely different market here.
- usrusr 3y agoBut "basement ML" is a thing, the market of people who are interested in PC gaming but not to the point of being lifestyle gamers who throw every cent they can spare at that altar. The GPU they bought long before the pandemic is still running every game they throw at it, but they never completely stop eyeing the new stuff. Dipping their toes in ML, even if it's just getting through 80% of some stable diffusion setup tutorial, can be a very welcome excuse to upgrade their gaming. A card sold for gaming but with generously overprovisioned VRAM (ideally in the range of the lowest bin of the biggest or second-biggest chip I think) could match that market segment very well - and it would not only compete with other price points, it would actually increase the market by some buyers (those who would not upgrade without the "ML excuse").