State of the art is basically 8 bit weights. Anything below that doesn't really work. You will see lots of benchmarks and figures saying that it does, but nearl

State of the art is basically 8 bit weights. Anything below that doesn't really work. You will see lots of benchmarks and figures saying that it does, but nearly all of those neglect the absolute accuracy, meaning, they compare deeply quantized models against the _shitty_ variants of full-precision or half-precision models which are not useful in practice. Another trick is to deeply quantize an overly redundant model that's no longer state of the art, and show a few percent degradation in accuracy on top of an already barely acceptable number. IMO, we need to pay attention to absolute accuracy if any of this is to become actually practical. I.e. I don't care how fast or small your compressed network is if its top5 accuracy on ImageNet is below 80%, or some other such criterion. Now granted, this is not perfect, because such models might still be useful for a smaller number of classes, but then maybe come up with a separate metric for that, too. A pedestrian detector is not very useful if it misses or misplaces 30% of pedestrians.