6 ms·
TTS: Text-to-Speech for All
- uniqueid 6y agoWhat's the plan with this? Is it to incorporate it into Firefox to improve its Web Speech API implementation?
- gxqoz 6y agoHrmm the link to an example from Pocket leads me to hope that these are coming to that app. The current TTS for listening to saved articles is decent but certainly not state of the art.
- adkadskhj 6y agoIn an example[1], it sounds decent but i noticed a fuzzy white noise whenever the voice is talking. Is this the algorithm, or compression? If it's the algorithm, why? [1]: https://soundcloud.com/user-565970875/pocket-article-wavernn-and-tacotron2 https://soundcloud.com/user-565970875/pocket-article-wavernn...
- throwawaysea 6y agoI actually don't hear the fuzzy white noise, but maybe it's because of my tinnitus. Is it during a certain part of the recording? To my ears this sounds surprisingly high fidelity and natural sounding.
- adkadskhj 6y agoIt's only during when the .. "person" talks. Which makes it quite noticeable to me because it starts and stops. It is rather faint, so i might not even notice it if it was consistent.
- eddyg 6y agoI hear it as well, even when using the speaker on my phone and not headphones (where it seems like it would be even more noticeable).
- xcodevn 6y agoThis is a well known problem. The noise is due to mu-law compression. The 16 bit audio samples are compressed to 8, 9, or 10 bits before feeding to the neutral net. The reason is because predicting a categorical distribution of 2^16 values requires too many parameters. The noise was also in samples from the famous Wavenet from Deepmind (they used 8 bit mu law). There are two ways to avoid this: 1. predict 8 high (coarse) bits, 8 low (fine) bits separately as in the original waveRNN paper. 2. use a mixture of logistic distributions as the predictive output as in the recent Lyra vocoder from Google.
- Tade0 6y agoHow does the number of parameters scale with resolution? Specifically, how much slower this would be if the audio was, say, 10 bits? I recall a lab exercise in college where we were supposed to increase the resolution of a quantizer until we reached a decent tone and 10 bits were the point at which we reached satisfying quality.
- 6y ago
- Isn0gud 6y agoIt seems like this is another dead mozilla project now, given that the people who worked on this started a new project: https://github.com/coqui-ai https://github.com/coqui-ai
- echelon 6y agoI work on TTS (created https://vo.codes https://vo.codes) and my impression of the Mozilla project was that it was incredibly understaffed. Unrealistically so to ever lead to any kind of product or platform. Maybe this new organization can accomplish the goal of easy and open trainable TTS. I'd really like to see it.
- erogol 6y agounderstaffed -> it was only me and still only me but with a great list of contributors :)
- karussell 6y agoThanks for doing this! I got it running immediately and I'm impressed and will try it in a project I have in mind plus spread the good word.
- echelon 6y agoI would really love to see you get funding for developing an open platform for TTS that can offer commercial options that fuel growth. I really want to see your work scale. Easy TTS from custom data needs to be a thing. Good job with the stuff you've built, and good luck!
- kdavis 6y agoYou can see some Coqui[0] TTS examples here[1]. [0] https://coqui.ai/ https://coqui.ai/ [1] https://erogol.github.io/ddc-samples/ https://erogol.github.io/ddc-samples/
- banana_giraffe 6y agoIt's a cool demo. Though, to my ears it's still a bit far from my dream of having something cheap or free I can feed more niche books I like and use to create an audiobook version of them. https://vocaroo.com/1oOjiLNCagur https://vocaroo.com/1oOjiLNCagur
- Causality1 6y agoThat's still my personal TTS dream as well. Google's Read Aloud voice blows everything else out of the water but I've found by experimentation that it will only read the first three and a half hours of web page text.
- kdavis 6y agoYou should check out newer examples[1]. The quality has improved significantly. [1] https://erogol.github.io/ddc-samples/ https://erogol.github.io/ddc-samples/
- Abishek_Muthian 6y agoTTS tech is so accessible that product developers should consider integrating it within their product for the sake of visually impaired and not leave the product to the mercy of operating system's accessibility features. I feel the biggest causality of online advertisement is accessibility, Those with eyes (eye-sight) are more valuable to the mega corps than those without and so Internet is full of rich graphics; making the lives of those without proper vision miserable.
- suyash 6y agoAlways be 100% compatible with OS provided accessibility guidelines before adding additional support. Most disabled users are familiar with native accessibility tools and that should come first.
- Abishek_Muthian 6y agoTrue, I meant native TTS as an addition to current accessibility specs to all apps with content focused on reading. e.g. News apps should have a play button on the top both for accessibility and convenience. Here's a discussion[1] regarding that on my problem validation platform with participation of someone with accessibility needs. [1] https://needgap.com/problems/200-human-voice-summary-of-news-news-voice https://needgap.com/problems/200-human-voice-summary-of-news...
- sandreas 6y agoMaybe interesting: https://colab.research.google.com/drive/1SPl226SwzrfMZltrVagIXya_ax4CsMh-?usp=sharing https://colab.research.google.com/drive/1SPl226SwzrfMZltrVag... https://github.com/keithito/tacotron https://github.com/keithito/tacotron https://www.youtube.com/watch?v=ijhZR43TOwc https://www.youtube.com/watch?v=ijhZR43TOwc https://heartbeat.fritz.ai/a-2019-guide-to-speech-synthesis-with-deep-learning-630afcafb9dd https://heartbeat.fritz.ai/a-2019-guide-to-speech-synthesis-...
- snakers41 6y agoYou should also totally check out silero-models, which are also available in colab with 10 speakers: - https://github.com/snakers4/silero-models#text-to-speech https://github.com/snakers4/silero-models#text-to-speech - https://habr.com/ru/post/549482/ https://habr.com/ru/post/549482/ Disclaimer, this is my independent project
- synesthesiam 6y agoLarynx TTS has a similar goal: https://rhasspy.github.io/larynx/ https://rhasspy.github.io/larynx/ It was originally based on Mozilla TTS, but I've since moved to exporting models to Onnx for speed.
- marcodiego 6y agoFLOSS TTS and STT is badly needed right now. Being able to use voice recognition and speech synthesis should not be restricted to a small oligoply.
- synesthesiam 6y agoShameless plug for Rhasspy: https://rhasspy.readthedocs.io/en/latest/ https://rhasspy.readthedocs.io/en/latest/
- deleted 6y ago[deleted]
- Raed667 6y agoIs there a way to get this working in Firefox?
- monkeydust 6y agoOne of my products involves providing a lot of dense data to traders overlayed with performance measures based on proprietary models. We are working on automatically extracting some insights for the user and using NLP to present them like news articles. It wouldn't take a huge lift from that to use TTS to provide another way for user to digest the data. Would make for a cool demo but wonder how sticky it would be.
- cromwellian 6y agoNVidia pimped this at GTC21 as "state of the art TTS" which is why I think it's getting renewed attention, , but to my ears, it doesn't sound anywhere near WaveNet (Google), Siri, or Alexa.
- swiley 6y agoI'm personally very suspicious of any software coming from NVidia at this point.
- deleted 6y ago[deleted]
- kdavis 6y agoNewer examples[0] are, to my ears, comparable or better than WaveNet (Google), Siri, or Alexa. Also, Mean Opinion Score (MOS)[1] tests[2] of the TTS engine have it coming out ahead of other commercial engines[3]. [0] https://erogol.github.io/ddc-samples/ https://erogol.github.io/ddc-samples/ [1] https://en.wikipedia.org/wiki/Mean_opinion_score https://en.wikipedia.org/wiki/Mean_opinion_score [2] https://dl.acm.org/doi/abs/10.1145/3313831.3376789 https://dl.acm.org/doi/abs/10.1145/3313831.3376789 [3] https://github.com/coqui-ai/TTS#-tts-performance https://github.com/coqui-ai/TTS#-tts-performance
- mileycyrusXOXO 6y agoThe example is very impressive! Sounds very natural.
- ancarda 6y agoWill there be a wide choice of accents? The link in the README <https://erogol.github.io/ddc-samples/ https://erogol.github.io/ddc-samples/> seemed to only have a single voice
- erogol 6y agoYep. The aim is to solve TTS for all languages one at a time. You can check out the released models page for the other models and languages. https://github.com/coqui-ai/TTS/releases https://github.com/coqui-ai/TTS/releases
- ftyers 6y agoWhen are you going to do Chuvash ? ;)
- synesthesiam 6y agoIf you're willing to record a public domain dataset, I'll help train a voice :)
- mnemotronic 6y agoThe acronym would be more fun if the product was Text-InTo-Speech. Yea.... -1
- erogol 6y agoCheck out Coqui TTS where we continue the work. https://github.com/coqui-ai/TTS https://github.com/coqui-ai/TTS Mozilla TTS is not maintained anymore (at least ATM). Disclaimer: I've created both of the projects.
- jadbox 6y agoThe examples are really impressive! Are multiple voice tones/genders supported?
- erogol 6y agowe are working on it. Check our latest work https://edresson.github.io/SC-GlowTTS/ https://edresson.github.io/SC-GlowTTS/ You can also check other released models here https://github.com/coqui-ai/TTS/releases https://github.com/coqui-ai/TTS/releases
- kstrauser 6y agoThat's mind blowingly amazing. It's far and away the best TTS I've ever heard in my life! We've come a long from from SAM and my Amiga, I tell ya.
- bravura 6y agoHow does this compare to ESPNET2? (https://colab.research.google.com/github/espnet/notebook/blob/master/espnet2_tts_realtime_demo.ipynb https://colab.research.google.com/github/espnet/notebook/blo...) Do you support multiple speakers? Also, do you mind if I email you and ask you a few questions about Coqui TTS?
- erogol 6y agoWe support multi speaker models and working on even multi lingual models. Come and join our gitter room. https://gitter.im/coqui-ai/TTS https://gitter.im/coqui-ai/TTS
- sirius87 6y ago
- whatrocks 6y agoI used one of the TTS pre-trained models to turn the Frankenstein e-book from Project Gutenberg into a podcast, and it worked pretty darn well (especially when I compared it with the terminal "say" command). Here's my write-up: https://www.charlieharrington.com/flow-and-creative-computing https://www.charlieharrington.com/flow-and-creative-computin... And the podcast RSS feed: https://whatrocks.github.io/castellan/podcastjr.xml https://whatrocks.github.io/castellan/podcastjr.xml It's great when these ML models link to a Google Colab notebook. It makes it super easy and dare-I-say fun to try them out.
- snakers41 6y agoYou should also totally check out silero-models, which are also available in colab with 10 speakers: - https://github.com/snakers4/silero-models#text-to-speech https://github.com/snakers4/silero-models#text-to-speech - https://habr.com/ru/post/549482/ https://habr.com/ru/post/549482/ Disclaimer, this is my independent project
- disabled 6y agoWow, that neural voice sounds so much better than the TTS that I use with my screenreader for reading books with my print-related disability. Thank you for the writeup! :-)