4 ms·
it's very common to run local models in 8 bit int.
by fzzzy 2y ago
it's very common to run local models in 8 bit int.
- qwertox 2y agoYes, but it's not common for the original model to be 8 bit int. The community can downgrade any model to 8 bit int, but it's always linked to quality loss.