21 ms·
It's not just changing the exponent bias, but also adjusting the center of the representable range for gradients to be closer to the current value of the parame
by yorwba 9y ago
It's not just changing the exponent bias, but also adjusting the center of the representable range for gradients to be closer to the current value of the parameter (which is stored in full precision).