3 ms·

My understanding is music generation is more like stable diffusion. It generates a waveform as an image, then turns it into an audio file.

• cubefox 6mo ago
  They do use diffusion models, but I don't think they would make a detour via images. They can just generate audio directly with audio diffusion rather than image diffusion.
  • corysama 6mo ago
    There technically was one experiment early on to trick Stable Diffusion into generating spectrograms that could be converted into audio. And, it worked surprisingly well. https://web.archive.org/web/20230314190913/https://www.riffusion.com/about https://web.archive.org/web/20230314190913/https://www.riffu... https://huggingface.co/riffusion/riffusion-model-v1 https://huggingface.co/riffusion/riffusion-model-v1 But, I'd expect everything in the past 3 years to diffuse the audio waveform directly.
    • Kye 6mo ago
      That's probably what I was thinking of. I haven't kept up as much on non-text generative AI.