4 ms·
Do the math. A dense 100GB model will be awfully slow at 273GB/s mem bandwidth.- So far, project digits looks disappointing.
by Tepix 2y ago
Do the math. A dense 100GB model will be awfully slow at 273GB/s mem bandwidth.-
So far, project digits looks disappointing.
- fancyfredbot 2y agoA 100GB model running on one of these would be a disaster, you'd probably run out of space for the KV cache apart from anything else. But if you had two of them performance should be acceptable. I think they would be canabalising their other product lines if they had more memory bandwidth.