6 ms·
Play 3.0 mini – A lightweight, reliable, cost-efficient Multilingual TTS model
- Asjad 2y agoPlay 3.0 mini sounds like a game-changer for real-time multilingual TTS with its speed and voice cloning capabilities
- treesciencebot 2y agoMuch faster than OpenAI's real-time mode, wow! Quality seems to be on par if not better as well.
- samsepi0l121 2y agoDid we watch the same video? OpenAI's model is faster, and the quality is far better.
- steego 2y agoForget the video. Try it. I use OpenAI's voice models a lot and I have access to them all and I'm honestly more impressed with the ease at which one can conduct a conversation with this voice model. Honestly, this feels like the first voice model I would pilot as a customer service rep in a hospitality setting.
- dulldata 2y agodemo video if you don't want to go through the announcement - https://www.youtube.com/watch?v=DusTj5NLC9w https://www.youtube.com/watch?v=DusTj5NLC9w Good with numbers mostly!
- nickthegreek 2y agoThe live test on https://play.ai/ https://play.ai/ didn't work for me in firefox. swapped to chrome and it worked quickly. I cloned my voice in 30s and was instantly talking to myself. This would easily fool most people who know me. Wild stuff.
- legofan94 2y agoFirefox is a known issue, we're working on that :x
- ktosobcy 2y agoUhm... was it a known issue when you released it or you didn't even try it on Firefox before release? :(
- joeross 2y agoI still use FF, for now anyway, so I’m not trying to be a dick here, but we’re talking less than 4% market share, so it’s hard to fault a small team for prioritizing the 82% they get with Chrome+Safari Source: https://en.wikipedia.org/wiki/File:StatCounter-browser-ww-monthly-202312-202312-bar.png https://en.wikipedia.org/wiki/File:StatCounter-browser-ww-mo...
- ktosobcy 2y agoI quite often wonder about those stats... I mean - most of Firefox users are quire conscious about privacy/tracking so most likely they have it blocked which... would "disappear" them from the stats? Chrome/Safari users mostly don't give a darn (and blocking is getting more difficult) so they would usually balloon the stats? Not to mention sites usually working just fine in Firefox but doing dumb detection hence users often hiding UserAgent?
- wkat4242 2y agoYes Firefox user here. I hide my useragent too because of stupid sites like Microsoft 365 that disable a lot of functionality for Firefox but everything works totally fine if they think I'm using edge. The same skulduggery that Google used on Gmail to make chrome big.
- drcongo 2y agoSide note, Safari ad-blocking is in a perfectly fine state and I haven't seen an ad online in years.
- phkahler 2y agoSounds quite good, but this prompt is NOT what I'd expect an automated system to feed into it: “I’ve successfully processed your order and I’d like to confirm your product ID. It is A as in Alpha, 1, 2, 3, B as in Bravo, 5, 6, 7, Z as in Zulu, 8, 9, 0, X as in X-ray.“ Phone numbers and others were read nicely, but apparently a string of alphanumerics for an order number aren't handled well yet.
- amrrs 2y agoSorry, Do you mean to the audio for this text is not good? “I’ve successfully processed your order and I’d like to confirm your product ID. It is A as in Alpha, 1, 2, 3, B as in Bravo, 5, 6, 7, Z as in Zulu, 8, 9, 0, X as in X-ray.“ I thought this was included in the demo, it seemed okay!
- mrkstu 2y ago'Alpha' is kind of swallowed and Bravo is mispronounced.
- phkahler 2y ago>> Sorry, Do you mean to the audio for this text is not good? No, the audio was OK or even good. The example seems to be an automated response from some system where a human has just placed an order. The order number is A123B567Z890X but if we want our system to "read back" the order number we apparently have to specially format the text. I suppose for the clarifying stuff "Alpha Bravo" that's a good idea, but separating digits and all those commas?
- BoorishBears 2y agoMost of these prompts come from LLMs, so it's trivial to instruct them to provide a string that's broken out like that. Also not the end of the world to process stuff like this with a regex. Most of these newer TTS models require this type of formatting to reliably state long strings of numbers and IDs
- diggan 2y ago> Phone numbers and others were read nicely The phone numbers were not naturally read at all. A human would have read a grouping of 123-456-789 like "123", "456", "789", but instead the model generated something like "123", "45", "6789". Listen to the RVSP example again and you'll know what I mean. The pacing is generally off for normal text too, but extra noticeable for the numbers. My hunch would be that it's because of tokenization, but I wouldn't be able to say that's the issue for sure. Sounds like it though :)
- gorkemyurt 2y agowow! latency is insane
- c0brac0bra 2y agoThis is similar: https://deepgram.com/agent https://deepgram.com/agent
- codetrotter 2y agoHey Alexa, Google “Play”!
- DevX101 2y agoHas anyone done a comparison of combined speech to text and TTS vs speech-to-speech for create audio only interfaces? Particularly curious around latency, and quality of audio output.
- amrrs 2y agoHugging Face has got a TTS leaderboard (arena like lmsys) - https://huggingface.co/spaces/TTS-AGI/TTS-Arena https://huggingface.co/spaces/TTS-AGI/TTS-Arena
- yavorgiv 2y agoLatency from the announcement https://x.com/_mfelfel/status/1846025183993511965/photo/1 https://x.com/_mfelfel/status/1846025183993511965/photo/1
- stuxyz 2y agoalso here: https://x.com/play_ht/status/1846240712125452469 https://x.com/play_ht/status/1846240712125452469
- lostmsu 2y agoIs this one open in any way? If no, why would anyone use it over OpenAI?
- gyre007 2y agoThis is awesome! Over the summer I wrote API clients for both Go [1] and Rust [2] as we were using Play in my job at the time but there was only Python and Node SDK. [1] https://github.com/milosgajdos/go-playht https://github.com/milosgajdos/go-playht [2] https://github.com/milosgajdos/playht_rs https://github.com/milosgajdos/playht_rs
- Mizza 2y agoWhat's SOTA for open source or on-device right now? I tried building a babelfish with o1, but the transcription in languages other than English are useless. When it gets it correct, the translations are pretty perfect and the voice responses are super fast, but without good transcription it's kind of useless. So close!
- amrrs 2y agohave you tried Moshi - https://huggingface.co/collections/kyutai/moshi-v01-release-66eaeaf3302bef6bd9ad7acd https://huggingface.co/collections/kyutai/moshi-v01-release-...
- refulgentis 2y agoI'm not sure what you mean fully, this is TTS, but it sounds like you're expecting an answer about transcription So its both hard to know what category you'd like to hear about, as well as if you do mean transcription, what your baseline is. Whisper is widely regarded the best in the free camp, but I wouldn't be surprised to see a paper of a model claiming better WER, or a much bigger model. If you meant you tried realtime 4o from OpenAI, and not o1*, it uses whisper for transcription on server, so I don't think you'll see much gain from trying whisper. my next try would be the Google Cloud APIs, but they're paid and with regard to your question re: open source SOTA, the underlying model isn't open. But also if you did mean 4o, the transcription shouldn't matter for output transcription quality, the model is taking in voice (I verified their claim by noticing when there's errors in the transcription, it answers correctly) * I keep messing these two up when talking about it, and it seems unlikely you meant o1 because it has a long synchronous delay before any part of the answer is available, and doesn't take in audio. If you did mean o1, then, I'd use realtime 4o for TTS, and have it natively do the translation, as it will be unaffected by errors in transcription like you're facing now
- krageon 2y agoGP said local / on-device. Most of what you mentioned is cloud shit.
- 2y ago
- Yenrabbit 2y agoQuite disconcerting to have a low-latency chat with something that sounds like you! Can recommend the experience, very thought-provoking.
- lyjackal 2y agoIs there any way to use the TTS on its own? I maintain an obsidian TTS plug-in, and am starting to add new TTS providers (its just been OpenAI thus far). From the documentation at https://docs.play.ai/documentation/get-started/introduction https://docs.play.ai/documentation/get-started/introduction, it looks like their API seems to couple it to an LLM for building conversational agents. Seems like it might be nice to use standalone as just TTS.
- amrrs 2y agoYou can use Play HT (the TTS powering Play AI) on its own - https://docs.play.ht/reference/api-getting-started https://docs.play.ht/reference/api-getting-started Do you have link to your obsidian TTS plugin?
- BoppreH 2y agoIn the video demo, Play 3.0 mini (on the left) incorrectly claims that the other AI missed a word. How does that end up in an announcement? Do people not notice, or not care? Or are they trying to show realistic mistakes?
- KaoruAoiShiho 2y agoIs this better than 11labs?
- scotty79 2y agoIf you go by hugging face leaderboards then no. https://huggingface.co/spaces/TTS-AGI/TTS-Arena https://huggingface.co/spaces/TTS-AGI/TTS-Arena
- sigmar 2y agoI don't see "playht 3.0 mini" on there, only play.ht 2.0 (released in august of 2023)
- yavorgiv 2y agoIt depends on the use case. If you are looking for a stable model with great voices and very low latency, Play 3.0 mini is as good as or better than 11labs. https://x.com/_mfelfel/status/1846025183993511965/photo/1 https://x.com/_mfelfel/status/1846025183993511965/photo/1
- siscia 2y agoI honestly wanted to try to use it, but their pricing was quite off-putting.
- c0brac0bra 2y agoYes. I think $0.05/min is a high multiple of what other agent-oriented realtime TTS products are charging.
- ilrwbwrkhv 2y agoTTS is a loss making business.. Somehow the VC funding has to be returned.
- Aeolun 2y agoThat’s 12 times cheaper than the OpenAI models though. Those are already very good, so I can’t really see myself using this. I really want a good on-device model though.
- CommanderData 2y agoIs there a way to train this on common AI voices from video games/movies, I'd very much like a voice assistant to sound like Father/Mother from Alien or Dead Space.
- mlboss 2y agoOn related note a very good open source TTS model was released 2 days back: https://github.com/SWivid/F5-TTS https://github.com/SWivid/F5-TTS Very good voice cloning capability. Runs under 10G vram nvidia gpu.
- stavros 2y agoThanks! Would "under 10G" also include 8 GB, by any chance? Although I do die inside a little every time I see "install Torch for your CUDA version", because I never managed to get that working in Linux.
- mlboss 2y agoI bought a 10 Tb drive just for these kind of experiments
- linotype 2y agoTry out PopOS. They make it really easy. Though it’s named Tensorman it helps with Torch as well. https://support.system76.com/articles/tensorman/ https://support.system76.com/articles/tensorman/
- stavros 2y agoThanks, but I don't think I'm going to reinstall my entire OS to run these. I'll see if I can get Docker working, it's been more reliable with CUDA for me.
- __MatrixMan__ 2y agoI haven't tried it, but I notice that it's also in nixpkgs: https://search.nixos.org/packages?channel=24.05&show=tensorman&from=0&size=50&sort=relevance&type=packages&query=tensorman https://search.nixos.org/packages?channel=24.05&show=tensorm... That might be a less invasive way to use it, though you'd still have to install nix.
- 2y ago
- throwaway48476 2y agoI would love a browser extension that does high quality TTS on arbitrary web articles.
- jasonjmcghee 2y agoThere's "Reader" by Eleven Labs. Now with iphone mirroring, you can use it without much trouble from your laptop too
- Palmik 2y agoThis is still four times more expensive than Cartesia (https://cartesia.ai/ https://cartesia.ai/) and three times more expensive than OpenAI's TTS API. In general, TTS APIs seem to run with much higher margins than LLMs from what I know.
- jnsaff2 2y agoThey are all expensive but I'm not so sure about margins. Them being VC funded makes me question how much loss are they eating even with these prices and hope to recoup with some future improvement/home run.
- ks2048 2y agoAny good TTS (open or not) allow finetuning for a new language?
- antman 2y agoDoes anyone know of a TTS mod that could convey feeling? E.g. ebook reading for novels? Or can one request feeling in any of the models of this discussion?
- Zababa 2y agoWhat's the state of the art for voice cloning in another language (here French)?
- yavorgiv 2y agoIt should do pretty good. You can try it here https://play.ht/playground/ https://play.ht/playground/
- lynx23 2y agoFirst question, does it pronounce numbers > 9 correctly? At least OpenAI's model doesn't perform at all, marking garbage out of almost every number it finds. I actually dont remember if I checked with EleventLabs... But I was shocked enough that in 2024, someone could release a TTS model that doesn't do numbers correctly. As if the AI industry was approaching Xerox level of failings. However, the TTS models are way worse then the Xerox compression algo ever was. I believe verifying numbers up to at least 100000 should be a requirement for new TTS models.
- nature556 2y agoI think it's important to have high quality TTS on arbitrary web articles. reply
- nutanc 2y agoThis is really good. Tried out the cloning. It sounded very similar to my voice. But then I did a blind test with 5 people. All of them didn't recognise it as my voice. So is there a bias when we listen to our own voice?
- tkgally 2y agoI wondered about the same thing. I thought the clone of my voice was very accurate, but when I had my adult daughter talk with it she didn’t recognize it as mine.
- emursebrian 2y agoI didn't check to see if Thai was supported, but it hangs when I try to perform TTS on the text "ฉันพูดภาษาไทย" and then comes back with an error message several minutes later.
- wkat4242 2y agoI have to say even something really low-resource like Piper (pure CPU) sounds amazing these days. TTS really appears to be a solved problem now.
- causal 2y agoAlways a little disappointing when someone announces they're releasing a model when they mean they're releasing an API