There's a huge amount of work on model pruning, especially with an eye towards model reduction for on-device deployments. I've done a bunch of work in this spac

There's a huge amount of work on model pruning, especially with an eye towards model reduction for on-device deployments. I've done a bunch of work in this space focused on getting speech synthesis working in real time on phones. It works and can be automated. There's a lot of nuance though. What has typically worked best are variants on iterative magnitude weight pruning rather than activation pruning. This can often get rid of 90% of the weights with minimal impact on quality. Structured pruning lets you remove blocks of weights while retaining good SIMD utilization... and so on.