2 ms·
Because it's 32-bits wide in memory. The effective mantissa is like FP16 but it's padded out to be the same size as FP32. In other words, there's 1 sign bit,
by 37ef_ced3 5y ago
Because it's 32-bits wide in memory.
The effective mantissa is like FP16 but it's padded out to be the same size as FP32.
In other words, there's 1 sign bit, 8 exponent bits, 10 mantissa bits that are USED, and 13 mantissa bits that are IGNORED.
1 + 8 + 10 + 13 = 32
The 13 ignored mantissa bits are part of the memory image: they pad the number out to 32-bit alignment.
- cjbgkagh 5y agoBut the user never sees that memory right? Doesn't it go in FP32 and come out FP32? I still think it's deceptive marketing.
- bcatanzaro 5y agoThe user does see 32-bits and all bits are used because all the additions (and other operations besides the multiply in matrix ops) are in FP32. So the bottom bits are populated with useful information.