4 ms·

I know it’s only FP4, but 1.4 exaflops in one rack is still crazy.

• alecco 3y ago
  With scaling and some tricks, FP4 inference can be very close to 16 bit. And most software out there is still missing on FP8 lol
  • cyanydeez 3y ago
    Doesn't that suggest the training method is flawed?
    • sp332 3y ago
      For training, you need to follow a gradient, and usually the gradient is small enough that you need the precision. Training in 8-bit is fairly new but Google and now Grok seem to have gotten good results.
      • alecco 3y ago
        And Microsoft.
      • cyanydeez 3y ago
        So these cards are geared towards inference ? There's a register article showing amds new instinct accelerators have much better.FP64
        • sp332 3y ago
          The article has numbers for both training, I think in FP8, and inference in FP4.
  • deleted 3y ago
    [deleted]