16 ms·
People working on this problem: have diffusion models taken off in this field too?
by blt 3y ago
People working on this problem: have diffusion models taken off in this field too?
- p1esk 3y agoYes: https://github.com/riffusion/riffusion https://github.com/riffusion/riffusion
- brcmthrowaway 3y agoAny dummies explanation of what diffusion is?
- benanne 3y agoI've since moved on to work primarily on diffusion models, so I have a series of blog posts about that topic as well! - https://sander.ai/2022/01/31/diffusion.html https://sander.ai/2022/01/31/diffusion.html is about the link between diffusion models and denoising autoencoders, IMO the easiest to understand out of all interpretations; - https://sander.ai/2023/07/20/perspectives.html https://sander.ai/2023/07/20/perspectives.html covers a slew of different perspectives on diffusion models (including the "autoencoder" one). In a nutshell, diffusion models break up the difficult task of generating natural signals (such as images or sound) into many smaller partial denoising tasks. This is done by defining a corruption process that gradually adds noise to an input until all of the signal is drowned out (this is the "diffusion"), and then learning how to invert that process step-by-step. This is not dissimilar to how modern language models work: they break up the task of generating text into a series of easier next-word-prediction tasks. In both cases, the model only solves a small part of the problem at a time, and you apply it repeatedly to generate a signal.
- zaptrem 3y agoyeah, we're training a diffusion model for this here: https://sonauto.ai/ https://sonauto.ai/ example 70s dolly parton song made by a rando earlier today: https://sonauto.ai/songs/t4XGcHqsYwS6B73MjtV5 https://sonauto.ai/songs/t4XGcHqsYwS6B73MjtV5
- xotesos 3y agoI hope this isn't a sign of what is to come in music. We can do anything imaginable with AI and instead we just make really shitty Dolly Parton music reboots. Maybe we can do AI Tom Jones too and form a whole genre of shitty AI 70s music.
- zaptrem 3y agoAgreed. The above is essentially a tech demo while we work on the more exciting/fulfilling uses we're planning for these models.