It turns out a convolution can be thought of like a special type of matrix https://en.wikipedia.org/wiki/Toeplitz_matrix#Discrete_convolution https://en.wikiped

It turns out a convolution can be thought of like a special type of matrix https://en.wikipedia.org/wiki/Toeplitz_matrix#Discrete_convolution https://en.wikipedia.org/wiki/Toeplitz_matrix#Discrete_convo... You could imagine do the backprop through the matrix multiply, and you are going to get several terms with the same coefficent, since the diagonals are the same - and you could just sum up all the gradients from the relevant terms and use that.