3 ms·
I was looking into this for LLMs but it's clearly a graphics-processing focused card. The memory bandwidth is too low for that much RAM to be useful in an LLM c
by arjie 6mo ago
I was looking into this for LLMs but it's clearly a graphics-processing focused card. The memory bandwidth is too low for that much RAM to be useful in an LLM context. The 5090 I have has the same amount of RAM but far more bandwidth and that makes it much more useful.
- Mindless2112 6mo agoCompared to a B70, a 5090 is 1x the memory with 3x the bandwidth at 4x the price. Yeah, the 5090 is better, but you're paying for it.
- arjie 6mo agoOn actual market it’s $1100 vs $3200 now, right? I actually got mine at $2200 at cost in the before days.
- Mindless2112 6mo agoCurrent lowest price for a new card on Newegg: $949.99 vs $3,699.99.
- arjie 6mo agoWow, 5090 prices have exploded. Thanks for looking. I should have known by hardware price intuition is broken.
- cmxch 6mo agoIt’s 32gb for people who can’t go for scalped 5090s but have a 3090 budget. I have a pair of them with a 9480 and the only thing I have to do is keep the cache happy.
- fluoridation 6mo agoEh. Trading CUDA for 8 more gigs seems like bad deal, unless you know absolutely for certain what you want to run will run on it.
- cmxch 6mo agoUntil NVidia prices get better, I’ll build out with the Intel stack and keep the cache (and prompt processing speeds) happy. As for software, anything that has a SYCL or Vulkan backend, and/or can be Intel optimized (especially to the same degree as llama.cpp) can run well.
- arjie 6mo ago[dead]
- girvo 6mo agoOh wow, I really would've expected higher memory bandwidth. That's only ~2-3x the little DGX Spark-alike I have to play with. Would've expected more.
- to11mtm 6mo agoIt would have been nice but I'm not mad about it; this is what I've been asking for since the 16GB Arc cards from earlier gens were so cheap; i.e. okay now do 'cheap but enough memory to run fancy models'. IMO if this would have come out a year or so earlier, I feel like it would have been more like 750$ due to DRAM pricing and thus more compelling... [0] [0] Yes It's likely that companies had secured DRAM contracts before the launch but I am also willing to bet that they also priced the card's launch price with at least some padding for increases in mind...
- askl 6mo ago> it's clearly a graphics-processing focused card. Yes, that's what the G in GPU stands for. It's great to see that there are still manufacturers that understand this.