5 ms·

I tested StyleTTS2 last month, my step-by-step notes that might be useful for people doing local setup (not too hard): https://llm-tracker.info/books/howto-guid

by lhl 3y ago

I tested StyleTTS2 last month, my step-by-step notes that might be useful for people doing local setup (not too hard): https://llm-tracker.info/books/howto-guides/page/styletts-2 https://llm-tracker.info/books/howto-guides/page/styletts-2 Also I did a little speed/quality shootoff with the LJSpeech model (vs VITS and XTTS). StyleTTS2 was pretty good and very fast: https://fediverse.randomfoo.net/notice/AaOgprU715gcT5GrZ2 https://fediverse.randomfoo.net/notice/AaOgprU715gcT5GrZ2

rahimnathwani 3y ago
Thanks. Following the instructions now. BTW mamba is no longer recommended (for those like me who aren't already using it), and the #mambaforge anchor in the link didn't work.
- lhl 3y ago
  I switched from conda to mamba a while ago and never looked back (it's probably saved dozens of hours from waiting for conda's slow as molasses package resolution). I'm looking at the latest docs and it doesn't look like there's any deprecation messages or anything (it does warn against installing mamba inside of conda, but that's been the case for a long time): https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html https://mamba.readthedocs.io/en/latest/installation/mamba-in... It looks like miniforge is still the recommended install method, but also the anchor has changed in the repo docs, which I've updated, thx. FWIW, I haven't run into any problems using mamba. While I'm not a power user, so there are edge cases I might have missed, but I have over 35 mamba envs on my dev machine atm, so it's definitely been doing the job for me and remains wicked fast (if not particularly disk efficient).
  - rahimnathwani 3y ago
    I had somehow missed the introduction of mamba, and have been using the default conda solver (which I think is the 'classic' one). Apparently conda now supports using the mamba solver: https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community https://www.anaconda.com/blog/a-faster-conda-for-a-growing-c... conda update -n base conda conda install -n base conda-libmamba-solver conda config --set solver libmamba
kelseyfrog 3y ago
> inferences at up to 15-95X (!) RT on my 4090 That's incredible! Are infill and outpainting equivalents possible? Super-RT TTS at this level of quality opens up a diverse array of uses esp for indie/experimental gamedev that I'm excited for.
- JonathanFly 3y ago
  >Are infill and outpainting equivalents possible? Do you mean outpainting as in you still what words to do, or the model just extends the audio unconditionally the way some image models just expand past an image borders without a specific prompt (in audio like https://twitter.com/jonathanfly/status/1650001584485552130 https://twitter.com/jonathanfly/status/1650001584485552130)
- refulgentis 3y ago
  Not sure what you mean: If you mean could inpainting and out painting with image models be faster, its a "not even wrong" question, similar to asking if the United Airlines app could get faster because American Airlines did. (Yes, getting faster is an option available to ~all code) If you mean could you inpaint and outpaint text...yes, by inserting and deleting characters. If you mean could you use an existing voice clip to generate speech by the same speaker in the clip, yes, part of the article is demonstrating generating speech by speakers not seen at training time
  - pedrovhb 3y ago
    I'm not sure I understand what you mean to say. To me it's a reasonable question asking whether text to speech models can complete a missing part of some existing speech audio, or make it go on for longer, rather than only generating speech from scratch. I don't see a connection to your faster apps analogy. Fwiw, I imagine this is possible, at least to some extent. I was recently playing with xtts and it can generate speaker embeddings from short periods of speech, so you could use those to provide a logical continuation to existing audio. However, I'm not sure it's possible or easy to manage the "seams" between what is generated and what is preexisting very easily yet. It's certainly not a misguided question to me. Perhaps you could be less curt and offer your domain knowledge to contribute to the discussion? Edit: I see you've edited your post to be more informative, thanks for sharing more of your thoughts.

by lhl 3y ago

rahimnathwani 3y ago

Thanks. Following the instructions now. BTW mamba is no longer recommended (for those like me who aren't already using it), and the #mambaforge anchor in the link didn't work.

lhl 3y ago
I switched from conda to mamba a while ago and never looked back (it's probably saved dozens of hours from waiting for conda's slow as molasses package resolution). I'm looking at the latest docs and it doesn't look like there's any deprecation messages or anything (it does warn against installing mamba inside of conda, but that's been the case for a long time): https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html https://mamba.readthedocs.io/en/latest/installation/mamba-in... It looks like miniforge is still the recommended install method, but also the anchor has changed in the repo docs, which I've updated, thx. FWIW, I haven't run into any problems using mamba. While I'm not a power user, so there are edge cases I might have missed, but I have over 35 mamba envs on my dev machine atm, so it's definitely been doing the job for me and remains wicked fast (if not particularly disk efficient).
- rahimnathwani 3y ago
  I had somehow missed the introduction of mamba, and have been using the default conda solver (which I think is the 'classic' one). Apparently conda now supports using the mamba solver: https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community https://www.anaconda.com/blog/a-faster-conda-for-a-growing-c... conda update -n base conda conda install -n base conda-libmamba-solver conda config --set solver libmamba

kelseyfrog 3y ago

> inferences at up to 15-95X (!) RT on my 4090 That's incredible! Are infill and outpainting equivalents possible? Super-RT TTS at this level of quality opens up a diverse array of uses esp for indie/experimental gamedev that I'm excited for.

JonathanFly 3y ago
>Are infill and outpainting equivalents possible? Do you mean outpainting as in you still what words to do, or the model just extends the audio unconditionally the way some image models just expand past an image borders without a specific prompt (in audio like https://twitter.com/jonathanfly/status/1650001584485552130 https://twitter.com/jonathanfly/status/1650001584485552130)
refulgentis 3y ago
Not sure what you mean: If you mean could inpainting and out painting with image models be faster, its a "not even wrong" question, similar to asking if the United Airlines app could get faster because American Airlines did. (Yes, getting faster is an option available to ~all code) If you mean could you inpaint and outpaint text...yes, by inserting and deleting characters. If you mean could you use an existing voice clip to generate speech by the same speaker in the clip, yes, part of the article is demonstrating generating speech by speakers not seen at training time
- pedrovhb 3y ago
  I'm not sure I understand what you mean to say. To me it's a reasonable question asking whether text to speech models can complete a missing part of some existing speech audio, or make it go on for longer, rather than only generating speech from scratch. I don't see a connection to your faster apps analogy. Fwiw, I imagine this is possible, at least to some extent. I was recently playing with xtts and it can generate speaker embeddings from short periods of speech, so you could use those to provide a logical continuation to existing audio. However, I'm not sure it's possible or easy to manage the "seams" between what is generated and what is preexisting very easily yet. It's certainly not a misguided question to me. Perhaps you could be less curt and offer your domain knowledge to contribute to the discussion? Edit: I see you've edited your post to be more informative, thanks for sharing more of your thoughts.