3 ms·
> float16 parameters vs 4bit parameters What does that mean?
by starik36 4y ago
> float16 parameters vs 4bit parameters
What does that mean?
- arbol 4y agoIt's referring to the precision with which the weights are stored in the models. Models are basically giant matrices of decimals in the range 0-1 and they represent the strengths of the paths between the neurons. Greater precision (float 16) will give better predictions. You lose accuracy in predictions when you reduce the precision (float 4) but it makes the model much easier to run on lower spec hardware, as it requires less RAM to store lower precision floats. https://en.m.wikipedia.org/wiki/Half-precision_floating-point_format https://en.m.wikipedia.org/wiki/Half-precision_floating-poin...