I thought something as simple as this would have been tried out long ago. Here are some related ideas. Instead of pruning channels, one should try to prune con

I thought something as simple as this would have been tried out long ago. Here are some related ideas. Instead of pruning channels, one should try to prune connections. Say with a 64x64x3x3 conv layer, it might be that the matrix (or tensor) can be brought to "block diagonal" form with a "change of basis", similar to singluar value decomposition. Except here we have 9 matrices of size 64x64 that we want to diagonilize simultaneously. This process itself might be formulated as an optimization problem that one solves by gradient descent.