All of this is fantastic if you are a deep learning Jedi Master. But there's a very small supply of those sorts. I really worry about all these reduced floati

All of this is fantastic if you are a deep learning Jedi Master. But there's a very small supply of those sorts. I really worry about all these reduced floating-point representations when they are made use of by people who mostly understand deep learning through tinkering with existing tensorflow tutorials. FP32 seems like a relative sweet spot with sufficient dynamic range to let most amateurs avoid getting trapped in the weeds. I could probably be persuaded to believe that FP24 is sufficient as well. But I suspect that once you get down to 16 bits or so you have to do all sorts of stochastic rounding / dithering that is beyond the skill set of most data scientists. And that's because I suspect many of them don't really get dynamic range. Which then leads me to believe that what we really need here is the equivalent of R for machine learning.