CNN layer's parameters are not interesting for compression, these are very small comparing to the massive full connect layers. In retrospective, the full conne

CNN layer's parameters are not interesting for compression, these are very small comparing to the massive full connect layers. In retrospective, the full connect layer's regularity probably just reflects that some neurons are more dead than others. You always probably want to have a dictionary-based compression method (LZ-like) for the last step to see how much more you can squeeze. Before that, quantization, residual quantization, some clever transformations probably can carry much longer way in terms of compressing the full connect layer parameters. I haven't explored literally any of these fancy methods yet. Just finished running xzip / gzip on best setting on my quantized full connect layer output to see if pngcrush's gain is my illusion. For the last two full connect layers, both pngcrush and xzip and gzip produce the about same size files, which suggests very limited compression ratio. On the first full connect layer, pngcrush (8.2M) is marginally better than xzip (8.6M). It is a reasonable choice as last step if you don't want to have library reference to xzip ;)