4 ms·

From a quick try results aren't good. Sounds bland, and the text I type isn't exactly equal to the text that is spoken. Didn't try with voice cloning though. W

From a quick try results aren't good. Sounds bland, and the text I type isn't exactly equal to the text that is spoken. Didn't try with voice cloning though. Why is good TTS so expensive and why are there no good open source options? Is it just from the need for high quality training data? I don't imagine these models are more expensive to run compared to SOTA LLMs, yet they cost so much more.

modeless 2y ago
There are a lot of options. StyleTTS2 is pretty good, XTTSv2 is pretty good, the new E2 TTS and F5 TTS also seem decent.
amrrs 2y ago
Commercially available high quality training dataset is the key. Open search libraries don't get the luxury of working with voice actors to record voices.
- Aeolun 2y ago
  Would it be hard to create such a training dataset? Seems like you’d just need a lot of people to say a bunch of stuff for you?
  - wahnfrieden 2y ago
    needs a crowdsourced model
    - huggingmouth 2y ago
      Ideally, Mozilla would step up here given their mission statement, but they won't, probably because their CEO needs another bonus.
      - IshKebab 2y ago
        Yeah there's no chance Mozilla would do anything like this: https://commonvoice.mozilla.org/ https://commonvoice.mozilla.org/
        mgkimsal 2y ago
        That's the first thing I thought of! I wonder how used these are. Are there any sources or data points indicating that this commonvoice data is being used, and if so, where/how? I think I may have contributed to this a few times back years ago. Nice to see it's still going, would be better to know it's being used.
        willwade 2y ago
        It was used quite a bit of speech to text - but tts it’s not that great.
        Aeolun 2y ago
        It costs a million dollar a year to host 32k hours of audio?
em-bee 2y ago
a few weeks ago i used piper to create an acceptable translation of a book. i didn't listen to it all, but the result sounded better than anything i was able to listen to before. good enough to listen to a book if a human read one is not available. just a few years ago, this was not the case. in other words, while FOSS TTS lags behind commercial options, it does get better and i expect within a few years it will produce results that are at least as good as the commercial options today if not fully caught up.
- asaddhamani 2y ago
  Piper seems roughly equivalent to old-school TTS outputs that sound flat, jumpy with the concatenative approach. Listen to this first example I tried: https://rhasspy.github.io/piper-samples/samples/en/en_GB/alan/medium/speaker_0.mp3 https://rhasspy.github.io/piper-samples/samples/en/en_GB/ala... Of all the TTS APIs I have tried, I like OpenAI voices the best. Haven't considered things like elevenlabs because I find them ridiculously expensive. I love voice to voice interfaces, but only when they sound natural to my ears, and the current pricing for good ones is prohibitive for a huge number of use cases.
  - em-bee 2y ago
    well, i was comparing it to the free tools available a few years ago, and against that, this example is a markable improvement. it's the first that i could actually bear to listen to over a longer period of time. i expect just another few years and this will actually be good.
sjnair96 2y ago
Have you tried VoiceCraft?
- asaddhamani 2y ago
  Yeah all these seem hyper focused on "voice cloning" so on replicate VoiceCraft doesn't even let you try normal TTS unless you provide a reference voice so I noped out.
miki123211 2y ago
From what I'm seeing, most of the open source TTS models are trained on the same few voices, mostly in 16Khz, mostly from Librivox books I think. Eleven Labs is most likely trained on stolen audiobooks, they've published a few Youtube videos in Polish, now taken down, of AI renditions of famous Polish audiobook narrators. This was all before they became popular, and before their voice cloning models were publicly available I think.
- generalizations 2y ago
  > mostly from Librivox books That probably explains a lot. I've tried listening to some of those audiobooks - very hit and miss, mostly miss. Definitely amateur hour and mostly bad quality.
sandreas 2y ago
I had pretty good results with coqui-tts and a VITS model, I trained myself with an open dataset and later with one I extracted from audiobooks / epub and therefore can't publish (german) The dataset and video tutorials are all available and linked on (also english): https://www.thorsten-voice.de/en/motivation-vision/ https://www.thorsten-voice.de/en/motivation-vision/
- thorsten-voice 2y ago
  Thanks for mentioning my Thorsten-Voice project, dear sandreas :)
  - sandreas 2y ago
    You're very welcome.

modeless 2y ago

There are a lot of options. StyleTTS2 is pretty good, XTTSv2 is pretty good, the new E2 TTS and F5 TTS also seem decent.

amrrs 2y ago

Commercially available high quality training dataset is the key. Open search libraries don't get the luxury of working with voice actors to record voices.

Aeolun 2y ago
Would it be hard to create such a training dataset? Seems like you’d just need a lot of people to say a bunch of stuff for you?
- wahnfrieden 2y ago
  needs a crowdsourced model
  - huggingmouth 2y ago
    Ideally, Mozilla would step up here given their mission statement, but they won't, probably because their CEO needs another bonus.
    - IshKebab 2y ago
      Yeah there's no chance Mozilla would do anything like this: https://commonvoice.mozilla.org/ https://commonvoice.mozilla.org/
      - mgkimsal 2y ago
        That's the first thing I thought of! I wonder how used these are. Are there any sources or data points indicating that this commonvoice data is being used, and if so, where/how? I think I may have contributed to this a few times back years ago. Nice to see it's still going, would be better to know it's being used.
        willwade 2y ago
        It was used quite a bit of speech to text - but tts it’s not that great.
      - Aeolun 2y ago
        It costs a million dollar a year to host 32k hours of audio?

em-bee 2y ago

a few weeks ago i used piper to create an acceptable translation of a book. i didn't listen to it all, but the result sounded better than anything i was able to listen to before. good enough to listen to a book if a human read one is not available. just a few years ago, this was not the case. in other words, while FOSS TTS lags behind commercial options, it does get better and i expect within a few years it will produce results that are at least as good as the commercial options today if not fully caught up.

asaddhamani 2y ago
Piper seems roughly equivalent to old-school TTS outputs that sound flat, jumpy with the concatenative approach. Listen to this first example I tried: https://rhasspy.github.io/piper-samples/samples/en/en_GB/alan/medium/speaker_0.mp3 https://rhasspy.github.io/piper-samples/samples/en/en_GB/ala... Of all the TTS APIs I have tried, I like OpenAI voices the best. Haven't considered things like elevenlabs because I find them ridiculously expensive. I love voice to voice interfaces, but only when they sound natural to my ears, and the current pricing for good ones is prohibitive for a huge number of use cases.
- em-bee 2y ago
  well, i was comparing it to the free tools available a few years ago, and against that, this example is a markable improvement. it's the first that i could actually bear to listen to over a longer period of time. i expect just another few years and this will actually be good.

sjnair96 2y ago

Have you tried VoiceCraft?

asaddhamani 2y ago
Yeah all these seem hyper focused on "voice cloning" so on replicate VoiceCraft doesn't even let you try normal TTS unless you provide a reference voice so I noped out.

miki123211 2y ago

From what I'm seeing, most of the open source TTS models are trained on the same few voices, mostly in 16Khz, mostly from Librivox books I think. Eleven Labs is most likely trained on stolen audiobooks, they've published a few Youtube videos in Polish, now taken down, of AI renditions of famous Polish audiobook narrators. This was all before they became popular, and before their voice cloning models were publicly available I think.

generalizations 2y ago
> mostly from Librivox books That probably explains a lot. I've tried listening to some of those audiobooks - very hit and miss, mostly miss. Definitely amateur hour and mostly bad quality.

sandreas 2y ago

I had pretty good results with coqui-tts and a VITS model, I trained myself with an open dataset and later with one I extracted from audiobooks / epub and therefore can't publish (german) The dataset and video tutorials are all available and linked on (also english): https://www.thorsten-voice.de/en/motivation-vision/ https://www.thorsten-voice.de/en/motivation-vision/

thorsten-voice 2y ago
Thanks for mentioning my Thorsten-Voice project, dear sandreas :)
- sandreas 2y ago
  You're very welcome.