4 ms·
For comparison, my 7900 XTX, which is the most powerful card that AMD offers, hits a (specced) peak of 120 tflops with fp16 operations, at 300W. However, when
by FeepingCreature 2y ago
For comparison, my 7900 XTX, which is the most powerful card that AMD offers, hits a (specced) peak of 120 tflops with fp16 operations, at 300W.
However, when they say "TOPS" instead of "TFLOPS" that usually means something like int8, and it's unclear if this chip will support any float format, so with most networks you'll have to quantize first. Not sure what overhead that adds to get the same quality.
- atq2119 2y agoTwo points about the GPU comparison. Modern GPUs, including the 7900 XTX, have tensor cores which give higher TOPS, especially for smaller data formats like int8. Desktop GPUs are also driven to power levels and voltages are beyond peak efficiency, so TOPS/watt will be worse than an NPU by design. (I'm sure the overall architecture is also a factor, but don't underestimate the effect of those comparatively simple power/frequency/voltage decisions.) And of course, software support is still very much in "I'll believe it when I see it" territory.