3 ms·
> It's the way in which audio and spectrogram are both representational while being encoded by the same information. They're not, really. If you've spent much
by wizzwizz4 2y ago
> It's the way in which audio and spectrogram are both representational while being encoded by the same information.
They're not, really. If you've spent much time reading spectrograms, you can look at the images and see which part is the sound. Occasionally the picture and sound are "chosen" to line up (notably the birdsong, disguised as distant blurry flower petals – notably not the large flowers, which don't really sound like birdsong at all), but usually the sound appears as clear, visually-distinct banding (e.g. the kittens meowing), and the image shows up as very audible distortion (e.g. the tigers).
Choosing sounds and images that line up properly might not be something a human has tried before, but I can already see how I'd do it, if I were better at drawing. It's like a multimedia ambigram: https://en.wikipedia.org/wiki/Ambigram https://en.wikipedia.org/wiki/Ambigram. The diffusion model is missing a lot of tricks: I could get similar quality by making a collage of clip art.
Edit: just saw the third "painting of a farm" one. That one is actually quite good, the way it uses the solid space in the barns. The first "painting of a farm" has the right idea (if I may anthropomorphise the system), working the birdsong in as clouds and trees, but its execution is severely lacking.