The symmetry is mostly an effect of 'natural' images not favoring a specific axis (i.e., vertical vs horizontal lines are still distinguished as the same 'type'

The symmetry is mostly an effect of 'natural' images not favoring a specific axis (i.e., vertical vs horizontal lines are still distinguished as the same 'type' of lines, in some sense). Another way of seeing why the symmetry emerges is that mirroring the image over an axis shouldn't change the edges you observe (this is the case for the edge detection). So, mirroring the underlying image shouldn't change the operation we wish to apply to detect edges.