4 ms·
Question: Does fp16 provide more accuracy than mixed-precision? If so, any reason for this to be happening? Looking at the discussion, everybody is agreeing to
by mrv_asura 3y ago
Question: Does fp16 provide more accuracy than mixed-precision? If so, any reason for this to be happening?
Looking at the discussion, everybody is agreeing to the fact that it is already well-known that fp32 is overkill and fp16(or bf16) is already industry standard(for most cases at least). But any opinions on mixed precision floating point is seems to be missing. Has anybody seen benchmarks that seem to indicate that mixed-precision fp performs worse than fp16 and fp32,(other than the paper)?