5 ms·
> for some reason When training a model you're typically following a gradient which may be very small so you want high precision. One a model is trained and yo
by redcalx 9y ago
> for some reason
When training a model you're typically following a gradient which may be very small so you want high precision. One a model is trained and you're just using it you can generally get by with very low precision, ultimately we're usually talking about placing some input vector in one or more binary categories and the affect of small signal variations higher up in the model generally don't amount to much w.r.t the final categorisations.
- pps43 9y agoThe folk theorem of statistical computing [1] says that when you have computational problems, often there’s a problem with your model. So chances are that if your gradient vanishes with 32 bit floats, switching to 64 bit floats won't help much. [1] http://andrewgelman.com/2008/05/13/the_folk_theore/ http://andrewgelman.com/2008/05/13/the_folk_theore/