4 ms·
I think it's actually really cool to focus on efficiency over just raw performance! The page for the cards themselves goes into more detail and has a pretty nic
by KronisLV 10mo ago
I think it's actually really cool to focus on efficiency over just raw performance! The page for the cards themselves goes into more detail and has a pretty nice graph: https://furiosa.ai/rngd https://furiosa.ai/rngd
You can see them admit that RNGD will be slower than a setup with H100 SXM cards, but at the same time the tokens per second per watt is way better!
Actually, I wonder how different that is from Cerebras chips, since they're very much optimized for speed and one would think that'd also affect the efficiency a whole bunch: https://www.cerebras.ai/ https://www.cerebras.ai/
- bradfa 10mo agoHaving only 48GB of RAM per card seems low. The full server system with 8 cards barely has enough RAM to run modern large open models. And batching together user requests eats quite a lot of memory, too. Curious to see how these machines and cards are received by the market.