4 ms·

With scaling and some tricks, FP4 inference can be very close to 16 bit. And most software out there is still missing on FP8 lol

• cyanydeez 3y ago
  Doesn't that suggest the training method is flawed?
  • sp332 3y ago
    For training, you need to follow a gradient, and usually the gradient is small enough that you need the precision. Training in 8-bit is fairly new but Google and now Grok seem to have gotten good results.
    • alecco 3y ago
      And Microsoft.
    • cyanydeez 3y ago
      So these cards are geared towards inference ? There's a register article showing amds new instinct accelerators have much better.FP64
      • sp332 3y ago
        The article has numbers for both training, I think in FP8, and inference in FP4.
• deleted 3y ago
  [deleted]