I came to the comments to say the same thing. Quantization and hashing tricks for embeddings are cool and all, but not really important for model compression.

I came to the comments to say the same thing. Quantization and hashing tricks for embeddings are cool and all, but not really important for model compression. Rather, training companion models to prune away whole subnetworks of weight and layer combinations can allow you to remove tens of thousands of parameters from the model entirely— not wasting space on their quantized weights when they end up not being a contributing pathway to predictions.