3 ms·
Some poking around the authors of the paper brought me to: AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models [1] with demos [2] This
by zoogeny 4y ago
Some poking around the authors of the paper brought me to: AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models [1] with demos [2]
This sparks my interest so much since the last few days I was wondering if it was possible to use diffusion models on spectrograms to do audio effects editing. Here is a paper submitted a couple of weeks ago doing just that. And the demo examples are exceptional.
I want all of this to start slowing down a bit so I have a chance to catch up. I was just watching Andrej Karpathy's excellent Zero to Hero syllabus [3] trying to wrap my head around LLMs and now I feel I absolutely must catch up on diffusion models.
1. https://arxiv.org/abs/2304.00830 https://arxiv.org/abs/2304.00830
2. https://audit-demo.github.io/ https://audit-demo.github.io/
3. https://karpathy.ai/zero-to-hero.html https://karpathy.ai/zero-to-hero.html
- sroussey 4y agoDoesn’t riffusion create music based on spectrographs? https://github.com/riffusion/riffusion https://github.com/riffusion/riffusion
- ShamelessC 4y agoYou might be interested in Riffusion, which was able to do text-to-audio by finetuning Stable Diffusion on spectrograms late last year. https://www.riffusion.com/about https://www.riffusion.com/about