4 ms·

Between this and Lyrebird there seem to be a high number of cutting edge TTS solutions being worked on in the private sector. Does anyone know why there haven't

Between this and Lyrebird there seem to be a high number of cutting edge TTS solutions being worked on in the private sector. Does anyone know why there haven't been much advancement with the FOSS libraries?

vianneychevalie 6y ago
I’m convinced that the barrier to entry in this field in terms of technologic and financial investment is too high for FOSS projects to compete with the commercial solutions We don’t see FOSS pharmaceutical research for instance, I believe for the same reason. The amount of coordination needed and the impossibility to separate TTS projects into sub-parts could also factors.
- sgk284 6y ago
  https://voice.mozilla.org/en https://voice.mozilla.org/en “Common Voice is Mozilla's initiative to help teach machines how real people speak.”
  - nmfisher 6y ago
    The problem with Common Voice is the same resource problem plaguing open source efforts in general - for TTS data, it's not just the size that matters, it's also the quality. For something like Sonantic, you need clean recordings from professional actors in proper recording environments (not to mention the in-house expertise to then filter these down to curate the training/test datasets). That costs money. A million people with laptop microphones will just never get there.
- ani-ani 6y ago
  Sure, open source TTS seems to be lagging behind recent commercial offerings, but pharmaceutical research is actually an excellent example of a field with massive FOSS software usage. TTS is also "purely virtual" which makes it significantly different, and I would say significantly more approachable to open source collaboration.
jonas21 6y ago
There have been quite a few recent advances in open-source TTS, such as: https://github.com/NVIDIA/tacotron2 https://github.com/NVIDIA/tacotron2 https://github.com/CorentinJ/Real-Time-Voice-Cloning https://github.com/CorentinJ/Real-Time-Voice-Cloning https://github.com/mozilla/TTS https://github.com/mozilla/TTS I think a lot of the remaining gap is due to a lack of high-quality training data -- most of the open-source models are trained on public-domain audiobooks (e.g. LJ Speech). However, good training data (large amounts of annotated recordings by professional voice actors) is expensive to create, and unlike code, there's not a tradition of people sharing it.
reubenmorais 6y ago
We have done Mean Opinion Score tests on Mozilla TTS [0] and gotten similar scores to real humans. The main problem for open sourcing higher quality models is licensing the dataset. [0] https://github.com/mozilla/TTS https://github.com/mozilla/TTS
- terrycody 6y ago
  Can we use mozilla tts in production now?
  - kouohhashi 6y ago
    I think so https://github.com/mozilla/TTS/blob/master/LICENSE.txt https://github.com/mozilla/TTS/blob/master/LICENSE.txt
klodolph 6y ago
I think there are a couple factors here. This is an incredibly difficult problem space. The solutions going forward involve ML techniques, which require ML experts (currently being hired away by industry) and the resources to create models, which includes not only a large amount of computational resources but a big chunk of training data which needs to be sourced somehow. I’m not really an expert. From what I understand, the “cutting-edge” stuff requires pushing past the point where we are splicing segments of speech together. Splicing segments together is hard enough. There are a couple open-source efforts like Mozilla’s, but if you want something like Lyrebird, well, that technology isn’t even really productized commercially yet.

vianneychevalie 6y ago

I’m convinced that the barrier to entry in this field in terms of technologic and financial investment is too high for FOSS projects to compete with the commercial solutions We don’t see FOSS pharmaceutical research for instance, I believe for the same reason. The amount of coordination needed and the impossibility to separate TTS projects into sub-parts could also factors.

sgk284 6y ago
https://voice.mozilla.org/en https://voice.mozilla.org/en “Common Voice is Mozilla's initiative to help teach machines how real people speak.”
- nmfisher 6y ago
  The problem with Common Voice is the same resource problem plaguing open source efforts in general - for TTS data, it's not just the size that matters, it's also the quality. For something like Sonantic, you need clean recordings from professional actors in proper recording environments (not to mention the in-house expertise to then filter these down to curate the training/test datasets). That costs money. A million people with laptop microphones will just never get there.
ani-ani 6y ago
Sure, open source TTS seems to be lagging behind recent commercial offerings, but pharmaceutical research is actually an excellent example of a field with massive FOSS software usage. TTS is also "purely virtual" which makes it significantly different, and I would say significantly more approachable to open source collaboration.

jonas21 6y ago

There have been quite a few recent advances in open-source TTS, such as: https://github.com/NVIDIA/tacotron2 https://github.com/NVIDIA/tacotron2 https://github.com/CorentinJ/Real-Time-Voice-Cloning https://github.com/CorentinJ/Real-Time-Voice-Cloning https://github.com/mozilla/TTS https://github.com/mozilla/TTS I think a lot of the remaining gap is due to a lack of high-quality training data -- most of the open-source models are trained on public-domain audiobooks (e.g. LJ Speech). However, good training data (large amounts of annotated recordings by professional voice actors) is expensive to create, and unlike code, there's not a tradition of people sharing it.

reubenmorais 6y ago

We have done Mean Opinion Score tests on Mozilla TTS [0] and gotten similar scores to real humans. The main problem for open sourcing higher quality models is licensing the dataset. [0] https://github.com/mozilla/TTS https://github.com/mozilla/TTS

terrycody 6y ago
Can we use mozilla tts in production now?
- kouohhashi 6y ago
  I think so https://github.com/mozilla/TTS/blob/master/LICENSE.txt https://github.com/mozilla/TTS/blob/master/LICENSE.txt

klodolph 6y ago

I think there are a couple factors here. This is an incredibly difficult problem space. The solutions going forward involve ML techniques, which require ML experts (currently being hired away by industry) and the resources to create models, which includes not only a large amount of computational resources but a big chunk of training data which needs to be sourced somehow. I’m not really an expert. From what I understand, the “cutting-edge” stuff requires pushing past the point where we are splicing segments of speech together. Splicing segments together is hard enough. There are a couple open-source efforts like Mozilla’s, but if you want something like Lyrebird, well, that technology isn’t even really productized commercially yet.