21 ms·
This article confuses me. Machine learning needs less precision (for some reason), but the workloads that CERN performs are scientific computing, not ML. Or,
by supahfly_remix 9y ago
This article confuses me. Machine learning needs less precision (for some reason), but the workloads that CERN performs are scientific computing, not ML. Or, are they doing ML and so need less precision?
- redcalx 9y ago> for some reason When training a model you're typically following a gradient which may be very small so you want high precision. One a model is trained and you're just using it you can generally get by with very low precision, ultimately we're usually talking about placing some input vector in one or more binary categories and the affect of small signal variations higher up in the model generally don't amount to much w.r.t the final categorisations.
- pps43 9y agoThe folk theorem of statistical computing [1] says that when you have computational problems, often there’s a problem with your model. So chances are that if your gradient vanishes with 32 bit floats, switching to 64 bit floats won't help much. [1] http://andrewgelman.com/2008/05/13/the_folk_theore/ http://andrewgelman.com/2008/05/13/the_folk_theore/
- nabla9 9y agoCern does lots of data analysis where measurement precision is not essential at least in evaluation phase. Switching into higher precision may be needed only for the final calculations It's also possible to scale and normalize data to lower precision.