2 ms·
Quantization and other optimizations are more for productionizing models. You start with something accurate and then you start making tradeoffs to get the infer
by charcircuit 4y ago
Quantization and other optimizations are more for productionizing models. You start with something accurate and then you start making tradeoffs to get the inference time to fit into your compute, memory, and time budgets.