3 ms·
Very interesting, thanks! 32gb for the 5090 vs 128gb for digits might put a nasty cap on unleashing all that power for interesting models. Several 5090s toget
by lz400 2y ago
Very interesting, thanks!
32gb for the 5090 vs 128gb for digits might put a nasty cap on unleashing all that power for interesting models.
Several 5090s together would work but then we're talking about multiple times the cost (4x$2000+PC VS $3000)
- ryao 2y agoInference presumably will run faster on a 5090. If the 5x memory bandwidth figure holds, then token generation would run 5 times faster. That said, people in the digits discussion predict that the memory bandwidth will be closer to 546GB/sec, which is closer to 1/3 the memory bandwidth of the 5090, so a bunch of 5090 cards would only run 3 times faster at token generation.