3 ms·
It's pretty common to convert to bfloat16 nowadays since that's much faster. e.g. bitsnbytes now defaults to bf16 compute for 4bit quant.
by jph00 3y ago
It's pretty common to convert to bfloat16 nowadays since that's much faster. e.g. bitsnbytes now defaults to bf16 compute for 4bit quant.