4 ms·

XTTS: Open-source Foundation TTS model

59 points by osanseviero 3y ago

moeffju 3y ago
This is pretty great. Is anyone aware of a model that can apply this to timestamped text, such as in subtitles?
osanseviero 3y ago
XTTS, the production-quality TTS model from Coqui, is released - Multilingual: Generates speech in 13 different languages - Voice cloning with 3 seconds of audio - Cross language voice cloning as well - 24khz quality - Blog: https://coqui.ai/blog/tts/open_xtts https://coqui.ai/blog/tts/open_xtts - Demo: https://huggingface.co/spaces/coqui/xtts https://huggingface.co/spaces/coqui/xtts - Model: https://huggingface.co/coqui/XTTS-v1 https://huggingface.co/coqui/XTTS-v1
- antman 3y ago
  Runtime error for the demo
- totetsu 3y ago
  This is great https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/9d7e196f-f74f-4190-9bba-e1db9e586023.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon... https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/66e7ae5a-f7bc-4200-8dc4-6693e73692e1.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon... https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/e649013b-6715-4eb9-bc67-a087c9f9e71d.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon... https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/f0d4b66d-ad22-4255-b1a8-d9c13fd71785.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon...
mianos 3y ago
This whole thing is so easy to use. Pip install or docker installer, start little flask app and you are good to go. The speech output of this tool is as good as I have ever heard. I am looking forward to sending this to my ESP32 remote audio player. What a world we live in!
sneak 3y ago
> Working with Heather Meeker, world-leading expert on open source licenses, Coqui has created a new, innovative model license, the Coqui Public Model License (CPML), and XTTS will be the first ever model released under the CPML! You can read more about the Coqui Public Model License (CPML) here. Followed the link. The CPML has restrictions that make it a proprietary license, decidedly not open source/free software. Using the term "open source" in that paragraph is deceptive. Combined with the GitHub link, this leads me to believe this is just more open source cosplay.
- nmfisher 3y ago
  The code is open source (Mozilla Public Licence), it's only the licence for the model weights which prohibits commercial use. I don't think the concept of open source translates very well to model weights. The "source" is effectively the model architecture (which is freely available) and the training scripts/data (I don't know if these are available or not). With access to those, you can reproduce the weights yourself. IMO the situation is closer to "source is open, but if you want to use our published binaries commercially, pay us". It's not free/libre, but it's also not unreasonable. Coqui is a small company, and freely releasing the weights for commercial use would deprive them of one of the few revenue streams they have.
  - JoshTriplett 3y ago
    > IMO the situation is closer to "source is open, but if you want to use our published binaries commercially, pay us". In such a circumstance you can still compile the source yourself. In this case, you cannot. Also: > Coqui is also innovating in open source model licensing. "innovating" by making something that isn't open and falsely calling it "open source". And even that isn't "innovating" because a few others are already engaging in the same false advertising.
    - nmfisher 3y ago
      In this case, you can train it yourself. That’s why I think it’s a fair comparison.
olivierduval 3y ago
The french language is not as good as the english language seem to be
lee101 3y ago
[dead]
ilaksh 3y ago
The examples of that latest model sound very close to as good as Eleven Labs. Anyone compared them thoroughly?

59 points by osanseviero 3y ago

This is pretty great. Is anyone aware of a model that can apply this to timestamped text, such as in subtitles?

XTTS, the production-quality TTS model from Coqui, is released - Multilingual: Generates speech in 13 different languages - Voice cloning with 3 seconds of audio - Cross language voice cloning as well - 24khz quality - Blog: https://coqui.ai/blog/tts/open_xtts https://coqui.ai/blog/tts/open_xtts - Demo: https://huggingface.co/spaces/coqui/xtts https://huggingface.co/spaces/coqui/xtts - Model: https://huggingface.co/coqui/XTTS-v1 https://huggingface.co/coqui/XTTS-v1

antman 3y ago
Runtime error for the demo
totetsu 3y ago
This is great https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/9d7e196f-f74f-4190-9bba-e1db9e586023.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon... https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/66e7ae5a-f7bc-4200-8dc4-6693e73692e1.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon... https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/e649013b-6715-4eb9-bc67-a087c9f9e71d.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon... https://coqui-prod-creator-app-synthesized-samples.s3.amazonaws.com/samples/f0d4b66d-ad22-4255-b1a8-d9c13fd71785.wav https://coqui-prod-creator-app-synthesized-samples.s3.amazon...

mianos 3y ago

This whole thing is so easy to use. Pip install or docker installer, start little flask app and you are good to go. The speech output of this tool is as good as I have ever heard. I am looking forward to sending this to my ESP32 remote audio player. What a world we live in!

sneak 3y ago

> Working with Heather Meeker, world-leading expert on open source licenses, Coqui has created a new, innovative model license, the Coqui Public Model License (CPML), and XTTS will be the first ever model released under the CPML! You can read more about the Coqui Public Model License (CPML) here. Followed the link. The CPML has restrictions that make it a proprietary license, decidedly not open source/free software. Using the term "open source" in that paragraph is deceptive. Combined with the GitHub link, this leads me to believe this is just more open source cosplay.

nmfisher 3y ago
The code is open source (Mozilla Public Licence), it's only the licence for the model weights which prohibits commercial use. I don't think the concept of open source translates very well to model weights. The "source" is effectively the model architecture (which is freely available) and the training scripts/data (I don't know if these are available or not). With access to those, you can reproduce the weights yourself. IMO the situation is closer to "source is open, but if you want to use our published binaries commercially, pay us". It's not free/libre, but it's also not unreasonable. Coqui is a small company, and freely releasing the weights for commercial use would deprive them of one of the few revenue streams they have.
- JoshTriplett 3y ago
  > IMO the situation is closer to "source is open, but if you want to use our published binaries commercially, pay us". In such a circumstance you can still compile the source yourself. In this case, you cannot. Also: > Coqui is also innovating in open source model licensing. "innovating" by making something that isn't open and falsely calling it "open source". And even that isn't "innovating" because a few others are already engaging in the same false advertising.
  - nmfisher 3y ago
    In this case, you can train it yourself. That’s why I think it’s a fair comparison.

olivierduval 3y ago

The french language is not as good as the english language seem to be

lee101 3y ago

[dead]

ilaksh 3y ago

The examples of that latest model sound very close to as good as Eleven Labs. Anyone compared them thoroughly?