4 ms·

Are there benefits to using FP32 vs FP16? I’ve been dabbling with deep learning but not really sure how much affect higher precision is having. Though more prec

• bitL 8y ago
  Traditionally Deep Learning frameworks were all using FP32. With FP16 one can theoretically get 2x speed and 2x larger models with the same VRAM capacity. For inferencing with INT8/INT4 it can be even way better (good for embedded stuff). The downside is that sometimes more complex/deep models don't converge (or converge less often than FP32). Sometimes there are framework issues with some advanced FP16 stuff.
• visionscaper 8y ago
  From experience I know that models using RNNs have trouble training with FP16 precision. The common solution is to do training in FP32 and inference in FP16. To make this happen you often have to implement custom code (e.g. using Tensorflow or Keras as a meta framework)