5 ms·
Thank you. So F16 would be 16 bits per weight, and F32 would be 32? Next question, if you don't mind, what are the tradeoffs in choosing between a model with mo
by cratermoon 2y ago
Thank you.
So F16 would be 16 bits per weight, and F32 would be 32?
Next question,
if you don't mind,
what are the tradeoffs in choosing between a model with more parameters quantized to smaller values vs fewer parameters full-precision models?
My current understanding is to prefer smaller quantized models over larger full-precision.