Images themselves have very little correlation beyond a few pixels in any direction. We might think of them as complex because we are interpreting 3D objects, l

Images themselves have very little correlation beyond a few pixels in any direction. We might think of them as complex because we are interpreting 3D objects, light and building a scene from our understanding of reality, but none of this actually exists in the image itself. Audio on the other hand has a very high degree of correlation across multiple time scales. There is tone and timbre and harmonic structure that is information on a very small time scale, then there is transients like percussive sounds that are slightly longer, then there is changing pitch that is on a longer time scale, then there are note envelopes, rhythmic phrases, bars of notes, and longer musical phrases. There is still a lot of interpretation done by your brain to build meaning from all this information, but there is much more actual physical correlation with phase and amplitude of physical waveforms evolving in time that has to be inferred by a neural net. Another way to think of this, is what happens when you take a 'snapshot' of music? If you freeze time and take a snapshot you can capture all the phases, amplitudes, and stochastic properties of the music for an instant, but you basically lose all meaning. Compared to an image, a snapshot can represent an entire story, not because there is actually anything significant recorded in any of the individual pixels, but because your brain is able to construct a story out of simple symbols and shapes. The lack of correlation lets machine learning algorithms use relatively simple operations like small 2D convolution filters and divide an image into small regions and still a neural net is able to infer a useful new image. I think it is much more difficult to break apart audio into small representative parts, then reassemble into something that would be aesthetically appealing.