7 ms·
Nvidia RTX 5000 Ada 32GB Workstation GPU Review
- prvc 3y agoAny explanation for lacking NVLink other than having been purposefully crippled?
- YetAnotherNick 3y agoNVLink is essential for training large neural networks, which NVidia now earn majority of revenue from. Their sales of more expensive GPUs will be affected if they have NVLink in cheaper GPUs.
- chriskanan 3y agoThe issue is that this has caused severe shortages. The only new card with NVLink is at the very highest end and when trying to get a quote recently, I was told there was a 13 month delay in shipping. But if I don't need NVLink, just a few months. They keep removing features
- YetAnotherNick 3y agoThey are available at 3-7 day delivery here: https://www.newegg.com/nvidia-900-21001-0000-000/p/N82E16814132090 https://www.newegg.com/nvidia-900-21001-0000-000/p/N82E16814...
- deleted 3y ago[deleted]
- RcouF1uZ4gsC 3y agoAt this pricing level, with this amount of RAM, I suspect a lot of use cases will be with ML and GenAI. Benchmarks for these use cases would have been interesting.
- janalsncm 3y agoTo be honest, the best benchmark you can run is your own training code. Everything else is a guess. When I tested the A6000 against the H100, there wasn’t that big of a boost from the newer card. Perhaps GPU operations weren’t the bottleneck in that case.
- RcouF1uZ4gsC 3y ago> To be honest, the best benchmark you can run is your own training code. Everything else is a guess. Yes, but the point of a review with benchmarks is that it is expensive and time-consuming for a customer to acquire the hardware just to run their own benchmarks on. Stable Diffusion and various LLMs are available pretty easily. A simple benchmark that this version of stable diffusion/llm was used with these settings and this is how long it took to produce image/we got this many tokens/sec would be a nice comparison that you are in a good position to do with access to all the hardware.
- ex3ndr 3y agoThis would end up memory-bound for LLMs (and you would better get x3 4090 for this) or by compute for diffusion models (and again 4090 is like 10% slower than H100 for 1% of the price).
- janalsncm 3y agoI get what you’re saying, but I think too much complexity hides in those numbers. Some of the optimizations for LLMs (flash attention) are hardware specific. And anyways, it’s not that expensive. You can rent the same gpu from cloud providers for a few dollars per hour. If you are serious about buying a GPU it is an extremely small cost in terms of time and money compared to the price of the gpu itself. The alternative is trusting an online review running some training or inference code which is likely not comparable to what most people are doing.
- michaelt 3y ago> You can rent the same gpu from cloud providers for a few dollars per hour. The big three cloud providers don't offer things like the RTX 5000 32GB or the 4090 do they? AWS will happily rent me a H100, A100, V100, K80, A10G, T4, or M60. Not to mention a Trainium, Inferentia, Inferentia2, Gaudi or Qualcomm AI 100. And don't forget to benchmark each one of those in a 1, 2, 4, and 8 GPU configuration; and with a variety of batch sizes; and with and without distributed training. Remember to work out the performance-per-dollar for on-demand, reserved, and spot instances, times three different cloud providers. Now, to compare to on-premise pricing we start by calling our air conditioning and backup generator vendors...
- ex3ndr 3y agoVery disappointing, much less cuda cores than in 4090, no nvlink (32G won't be enough anyway), price x3 to 4090.
- jakderrida 3y ago>much less cuda cores than in 4090 "fewer", not "less".
- malermeister 3y agoWhile that's true, it feels a bit pedantic to correct parent like that.
- tcptomato 3y agoWhat would be the right way to correct him?
- sushibowl 3y agoI think the right way is, don't. This kind of "mistake" can not really result in any misunderstanding so there is not really any benefit gained from a correction. Correcting serves only to disrupt the flow of the conversation.
- prmoustache 3y agoIt benefits me, the non native english speaker passing by in the comments and who learnt something today.
- _a_a_a_ 3y agoI'm a native English speaker. I don't personally like corrections like this, and I would never criticise anyone for using the 'wrong' term. There is too much convention in English which isn't useful because it bleeds into pedantry and I think this is such a case. Using either word gets the job done cleanly, and at the bottom that's all that counts.
- merb 3y agoWhy do they have this naming… it’s just insane. RTX 5000 Ada… do they just put letters in front or names at the back these days? So confusing and the consumer cards will also be rtx 50xx
- mgaunard 3y agoNvidia always named their microarchitectures after famous scientists: Ada, Ampere, Turing, Pascal, Maxwell, Kepler, Fermi, Tesla. I guess what's unique with Ada is that they're using her first name? Though most official sources call it Ada Lovelace in full.
- NolF 3y agoI believe the issue with Lovelace is that you may find less than PG results typing that on a search engine. Hence using Ada primarily on the marketing. I think the complaint is more with the consumer card being 4xxx but this is 5000 both on the same architecture.
- Tsiklon 3y agoYeah they have different naming conventions on the workstation cards; Quadro RTX 4000 RTX A4000 RTX 4000 Ada Unfortunately they’ve had 3 separate naming conventions in 3 successive generations. Those 4000 series cards are in the same position in the lineup for each generation.
- washadjeffmad 3y agoI just went through this with our Dell rep. The generations aren't totally successive, if you count the non-Quadro RTX 4000 series, which is Ada generation but not part of the RTX 4000 Ada series. Add to it the card variants, and there's a chance that you might still end up with the wrong part if your purchaser isn't careful.
- usrusr 3y agoWas about to post something questioning the "forever" part because my memory only starts to link Nvidia generations with scientist somewhere around Kepler. And that's despite having followed GPU tech a lot more in the years before. But according to Wikipedia it goes back to the days of the Riva TNT: the wiki seems undecided about Fahrenheit-ness of earlier generations, but I'd consider that close enough for "forever".
- KeplerBoy 3y agoWeird timing, reviewing it months after it was released.