2 ms·

I’m not sure this is true. Nvidia ships hardware today that is specialized for transformers, and presumably AMD needs something similar on their side. An arms r

by oivey 3y ago

I’m not sure this is true. Nvidia ships hardware today that is specialized for transformers, and presumably AMD needs something similar on their side. An arms race of specific hardware like that is probably not to their advantage. It makes the interoperability story even harder. They still need a usable GPGPU language to glue all that hardware together, too.

dhruvdh 3y ago
There is nothing special about Nvidia hardware outside of them having significantly better lower precision FLOPs before anyone else. Their marketing chose to call this a "transformer engine" but there is no actual "engine" in silicon. Like AMD announced today, and is shown in the article we're commenting on, AMD now is 1.3 times faster at FP8, INT8 flops with and without sparsity, meaning their "transformer engine" is 1.3 times faster. The rest of your comment doesn't make sense. See the other comments on software in this thread.