2 ms·

If I'm following correctly, does this mean that with this change along with a model being quantized, we could see models that are 5% the size (on file system) a

If I'm following correctly, does this mean that with this change along with a model being quantized, we could see models that are 5% the size (on file system) and memory usage but almost identical in output?

zamalek 3y ago
The vales are selected were arbitrary. The size reduction will be 32bits/8bits - so it will be 4 times smaller.