3 ms·
I’m surprised there’s no mention of sparse formats to store the weights of the model.
by posharma 6y ago
I’m surprised there’s no mention of sparse formats to store the weights of the model.
- antixk 6y agoBecuase unless you have a certain (high) level of sparsity, sparse formats are infact ineffective in storing. There are cases where sparse formats take more memory than storing dense tensors.
- posharma 6y agoSure. It’s still one of the methods to compress models. You might be surprised how often the weights are really quite sparse.