I think quantization for large language models already do something like this - they compress the parameters in memory and then decompress when performing the f

I think quantization for large language models already do something like this - they compress the parameters in memory and then decompress when performing the forward passes