8 ms·
Text-to-Audio Generation Using Instruction Tuned LLM and Latent Diffusion Model
- adt 4y agoExcellent. Some of the theory here goes back to Oct/2021 and beyond [1]. The riffusion.com [2] guys made this practical. Also, my video showing high-level overview and examples [3]. 1. SpecVQGAN: https://github.com/v-iashin/SpecVQGAN https://github.com/v-iashin/SpecVQGAN 2. Riffusion: https://www.riffusion.com/ https://www.riffusion.com/ 3. Riffusion high-level overview: https://youtu.be/olkLVGcvib8 https://youtu.be/olkLVGcvib8
- leecarraher 4y agointeresting, seems like it could take the job of a movie looper with very little additional tuning.
- leecarraher 4y agoloopers role on a movie set - https://www.npr.org/2015/02/19/385237280/never-seen-and-sometimes-barely-heard-loopers-fill-in-hollywoods-soundtrack https://www.npr.org/2015/02/19/385237280/never-seen-and-some...
- brianjking 4y agoAnyone have the actual Gradio space link? The one that is on the site is broken.
- brucethemoose2 4y agoCould it generate speech with enough training? And what is SOTA for TTS these days?
- cheald 4y agoTortoiseTTS is pretty impressive: https://github.com/neonbjb/tortoise-tts https://github.com/neonbjb/tortoise-tts
- GaggiX 4y agohttps://google-research.github.io/seanet/audiolm/examples/ https://google-research.github.io/seanet/audiolm/examples/ is capable of generating speech that is syntactically correct and semantically coherent (without ever being conditioned on text).