7 ms·
Coqui.ai TTS: A Deep Learning Toolkit for Text-to-Speech
- spacemanspiff01 2y agoI believe the company behind this shit down at the end of 2023
- giancarlostoro 2y agoOne of my favorite typos. ;) Also coqui is a frog in Puerto Rico (that wound up in Hawaii, sneaking into someone's luggage or something to that effect), when you hear them at night, what you are hearing is their mating call if I remember correctly.
- Jayakumark 2y agoIts good except for license.
- sa-code 2y agoIs the license still relevant if the company has shut down?
- cal85 2y agoYes
- marcooliv 2y agohow?
- dlx 2y agoThe license forbids commercial use unless you buy a license. The problem is, no one seems to be selling one ;)
- 2Gkashmiri 2y agoSo if you violate the license, who will try to sue you?
- Bilal_io 2y agoYou never know, and you don't want to find out
- pabs3 2y agoThe license is the MPL, which allows commercial use? https://github.com/coqui-ai/TTS/blob/dev/LICENSE.txt https://github.com/coqui-ai/TTS/blob/dev/LICENSE.txt
- eginhard 2y agoThis is just the code license. Parents are referring to the XTTS model (their best one).
- nishithfolly 2y agoThis was a great team. Sad to see they had to shut down.
- ks2048 2y agoI don't know anything about the startup/VC world, but does anyone have insight on why this failed? It seemed to be one of the highest profile TTS projects and I thought money was just pouring into AI startups.
- eginhard 2y agoSome insights from one former lead: https://erogol.com/2024/01/09/goodsandbadsofopensource https://erogol.com/2024/01/09/goodsandbadsofopensource TLDR: Making money from open-source is hard.
- modeless 2y agoXTTSv2 is only slightly behind StyleTTS 2 near the top of the TTS Arena leaderboard, though they are both far behind Eleven Labs: https://huggingface.co/spaces/TTS-AGI/TTS-Arena https://huggingface.co/spaces/TTS-AGI/TTS-Arena Personally I prefer StyleTTS 2, and it has a better license. But XTTSv2 has a streaming mode with pretty low latency which is nice. I did run into hallucination issues though. It will hallucinate nonsense words or insert extra syllables in words, pretty frequently. As others mentioned they shut down so there won't be any updates to XTTS.
- eginhard 2y agoThey just shared the paper for XTTS, which got accepted to Interspeech and might be the reason for this being posted now: https://arxiv.org/abs/2406.04904 https://arxiv.org/abs/2406.04904
- jsemrau 2y agoInteresting. I got quite good results for my longform substack by combining xTTS2 with Nvidia's Nemo.
- WhitneyLand 2y agoAnyone have a sense for how these compare to OpenAI’s TTS?
- jonahx 2y agoSomewhat unrelated, but given that anyone can vote anonymously, how is the TTS-Arena protecting itself against bots or even rings of humans gaming the system?
- modeless 2y agoLow stakes, I guess
- Grimblewald 2y agoproblem is that low stakes divided by low cost of bots is still an acceptable return.
- vessenes 2y agoNB: Coqui is no longer actively maintained. I’m not sure what the team is up to now. The open market is definitely in need of an upgraded TTS offering; eleven labs is far ahead at the moment.
- eginhard 2y agoWe do maintain a fork, mostly with bug fixes for now: https://github.com/idiap/coqui-ai-TTS https://github.com/idiap/coqui-ai-TTS PRs welcome :)
- dlx 2y agoAny progress on the license situation? I'd love to work more on it, but worried about it being a bit of a dead end due to uncertainty about the future of the license and not being able to use it in any commercial projects.
- CaptainOfCoit 2y ago> Any progress on the license situation? What is the situation exactly? Seems to be licensed MPL at a glance, so you're able to use it in commercial projects.
- eginhard 2y agoThe licenses of the code (MPL 2.0, allowing commercial use) and the available pretrained models (https://github.com/idiap/coqui-ai-TTS/blob/dev/TTS/.models.json https://github.com/idiap/coqui-ai-TTS/blob/dev/TTS/.models.j...) are all clearly stated and won't change unless the model owners decide to do so. So the XTTS model is still under CPML, which doesn't allow commercial use.
- ritonlajoie 2y agoAre there any project which would make TTS with my own voice with some training on my voice ?
- mttpgn 2y agoYes, elevenlabs can.
- eginhard 2y agoYes, you can train/fine-tune models on your own voice with Coqui
- willwade 2y agoElevenlabs, coqui, piper, Microsoft, Google, Apple. Seriously. They all can these days. Don’t forget acapela or nuance.
- probably_wrong 2y agoWhile the other commenters provided several voice cloning projects, I would like to point out that I haven't been able to find one that works well for South-American Spanish.
- Kerbonut 2y agoI really like Parler TTS on the TTS Arena.
- phyce 2y agoCoqui is great, but another fantastic tool for TTS I recommend checking out is Piper. The voice quality is great, it's extremely lightweight, and it's fast enough to generate TTS in realtime https://github.com/rhasspy/piper https://github.com/rhasspy/piper
- roskoez 2y agoAnyone knows a modern TTS program for Windows? Something you can feed a text file to and have it read while it's on screen? I've been using Dimio's Speech for a decade now, but it seems silly now that much better voices exist.
- nextworddev 2y agoIs there anything that’s self hostable that’s on par with Elevenlabs?
- satvikpendem 2y agoHow does it compare to this recent Show HN, MARS5 [0]? Coqui is not maintained anymore so I'd be interested in what the SOTA is for open source TTS. [0] https://news.ycombinator.com/item?id=40616438 https://news.ycombinator.com/item?id=40616438
- SubiculumCode 2y agoI have a pet ML project that I am doing for fun. I am trying to build a custom transcription and diarizer model for a friend's podcast[1]. My initial solution involved a straight forward implementation using Whisper medium for transcription, and Nemo for diarizing, based on [1]. The results are not bad generally, but since my application involves a fixed set of five known speakers, I thought surely I could fine tune the nemo (or pyannote) diarizer model on their voices to improve accuracy. Audio samples are easily obtained from their podcast, but manual data labeling is painful for a hobby activity. Further, from what I understand, the real difficulty in performant diarizer models is not speaker recognition generally, but specifically speaker recognition while there is overlapping speech between multiple speakers. I am not even sure how to best implement a labeling procedure for segments with overlapping speech. I started to wonder whether I might bootstrap a decent sample by leveraging TTS vocal cloning models to simulate the five speakers in dialogues with overlapping speech segments. So I ask HN, is this hopelessly naive, or potentially useful technique? Also, any other advice? [1] https://www.3d6downtheline.com/ https://www.3d6downtheline.com/ [2] https://github.com/MahmoudAshraf97/whisper-diarization/ https://github.com/MahmoudAshraf97/whisper-diarization/
- tarasglek 2y agoUnclear from docs, does your solution support inferring number of speakers from audio? Found it a bit frustrating that this wasn't automatic in diarization algos I tried last year
- SubiculumCode 2y agoThe solution that this GitHub provided automatically determines the number of speaker labels, but will often create extra speaker classes for a few exerpts in the stream. You can prespecify the number of speakers I believe for better performance.
- robotburrito 2y agoI like this project. I used it to create a website that lets me turn a list of articles on the web into a podcast I can subscribe to via my phone.
- ackprakhack 2y agoWe've just opensourced MARS5 and are bullish about it's ability to capture very hard prosody -- hopefully you can validate the results and grow alongside its community. We tend to agree, the time for just one company to be seriously doing speech is over. It needs to be more diverse, and needs to be opensource https://github.com/Camb-ai/MARS5-TTS https://github.com/Camb-ai/MARS5-TTS
- BenRacicot 2y agoIf we could run this locally (Win and Mac) it could reset the standard for accessibility.
- vijucat 2y agoI absolutely love how good the voices are in the VCTK-VIS dataset (109 of them!). I found it easy to install Coqui on WSL and it is able to use CUDA + the GPU quite effectively. p236 male and p237 female are my choices, but holy cow, 109 quality voices still blows my mind. Crazy how you had to pay for a good TTS just a year ago, but now, it's commoditized. Hope you find this useful: CUDA_VISIBLE_DEVICES="0" python TTS/server/server.py --model_name tts_models/en/vctk/vits --use_cuda True def play_sound(response): #learning : you have to use a semaphore to serialize calls to winsound.PlaySound(), which freaks out with "Failed to play sound" if you try to play 2 clips at once semaphore.acquire() try: winsound.PlaySound(response.content, winsound.SND_MEMORY | winsound.SND_NOSTOP) finally: # Always release the permit, even if PlaySound raises an exception semaphore.release()