This is actually really cool article, though it could be a bit better written for sure. The new Tesla 8 and 16 bit floating number formats in the Dojo system s

This is actually really cool article, though it could be a bit better written for sure. The new Tesla 8 and 16 bit floating number formats in the Dojo system supports this kind of rounding up/down using a PRNG when compressing neural network parameters from higher precision floats (it is specified in the floating point paper). The random rounding is needed to not have a bias in the neural network weight updates, but this article improves the rounding method to be more accurate (closer to the real rounding than uniform sampling) without reintroducing bias.