9 ms·
Show HN: Dia, an open-weights TTS model for generating realistic dialogue
- LarsDu88 1y agoFantastic model. I'm going to write a Unity plugin for this. Have been using ElevenLabs for my VR game side project, but this appears to be better
- isoprophlex 1y agoIncredible quality demo samples, well done. How's the performance for multilingual generation?
- toebee 1y agoThank you for the kind words! We only support English at the moment.. Hope to add more languages in the future.
- toebee 1y agoHey HN! We’re Toby and Jay, creators of Dia. Dia is 1.6B parameter open-weights model that generates dialogue directly from a transcript. Unlike TTS models that generate each speaker turn and stitch them together, Dia generates the entire conversation in a single pass. This makes it faster, more natural, and easier to use for dialogue generation. It also supports audio prompts — you can condition the output on a specific voice/emotion and it will continue in that style. Demo page comparing it to ElevenLabs and Sesame-1B https://yummy-fir-7a4.notion.site/dia https://yummy-fir-7a4.notion.site/dia We started this project after falling in love with NotebookLM’s podcast feature. But over time, the voices and content started to feel repetitive. We tried to replicate the podcast-feel with APIs but it did not sound like human conversations. So we decided to train a model ourselves. We had no prior experience with speech models and had to learn everything from scratch — from large-scale training, to audio tokenization. It took us a bit over 3 months. Our work is heavily inspired by SoundStorm and Parakeet. We plan to release a lightweight technical report to share what we learned and accelerate research. We’d love to hear what you think! We are a tiny team, so open source contributions are extra-welcomed. Please feel free to check out the code, and share any thoughts or suggestions with us.
- new_user_final 1y agoEasily 10 times better than recent OpenAI voice model. I don't like robotic voices. Example voices seems like over loud, over excitement like Andrew Tate, Speed or advertisement. It's lacking calm, normal conversation or normal podcast like interaction.
- toebee 1y agoThank you! You can add audio prompts of calm voices to make them a bit smoother. https://huggingface.co/spaces/nari-labs/Dia-1.6B https://huggingface.co/spaces/nari-labs/Dia-1.6B you can try it here!
- gfaure 1y agoAmazing that you developed this over the course of three months! Can you drop any insight into how you pulled together the audio data?
- isoprophlex 1y ago+1 to this, amazing how you managed to deliver this, and iff you're willing to share i'd be most interested in learning what you did in terms of train data..!
- nickthegreek 1y agoAre there any examples of the audio differences between the this and the larger model?
- strobe 1y agojust in case, another opensource project using same name https://wiki.gnome.org/Apps/Dia/ https://wiki.gnome.org/Apps/Dia/ https://gitlab.gnome.org/GNOME/dia https://gitlab.gnome.org/GNOME/dia
- toebee 1y agoThanks for the heads-up! We weren’t aware of the GNOME Dia project. Since we focus on speech AI, we’ll make sure to clarify that distinction.
- Magma7404 1y agoI know it's a bit ridiculous to see that as some kind of conspiracy, but I have seen a very long list of AI-related projects that got the same name as a famous open-source project, as if they wanted to hijack the popularity of those projects, and Dia is yet another example. It was relatively famous a few years ago and you cannot have forgotten it if you used Linux for more than a few weeks. It's almost done on purpose.
- teddyh 1y agoThe generous interpretation is that the AI hype people just didn’t know about those other projects, i.e. that they are neither open source developers, nor users.
- 1y ago
- stuartjohnson12 1y agoImpressive project! We'd love to use something like this over at Delfa (https://delfa.ai https://delfa.ai). How does this hold up from the perspective of stability? I've spoken to various folks working on voice models, and one thing that has consistently held Eleven Labs ahead of the pack from my experience is that their models seem to mostly avoid (while albeit not being immune to) accent shifts and distortions when confronted with unfamiliar medical terminology. A high quality, affordable TTS model that can consistently nail medical terminology while maintaining an American accent has been frustratingly elusive.
- toebee 1y agoInteresting. I haven't thought of that problem before. I'm guessing a large enough audio dataset for medical terminology does not exist publicly. But AFAIK, even if you have just a few hours of audio containing specific terminology (and correct pronunciation), fine-tuning on that data will significantly improve performance.
- IshKebab 1y agoWhy does it say "join waitlist" if it's already available? Also, you don't need to explicitly create and activate a venv if you're using uv - it deals with that nonsense itself. Just `uv sync`.
- flakiness 1y agoSeek back a few tens of bytes which states "Play with a larger version of Dia"
- toebee 1y agoWe're envisioning a platform with a social aspect, so that is the biggest difference. Also, bigger models! We are aware of the fact that you do not need to create a venv when using pre-existing uv. Just added it for people spinning up new GPUs on cloud. But I'll update the README to make that a bit clearer. Thanks for the feedback :)
- ivape 1y agoDarn, don't have the appropriate hardware. The full version of Dia requires around 10GB of VRAM to run. If you have a 16gb of VRAM, I guess you could pair this with a 3B param model along side it, or really probably only 1B param with reasonable context window.
- toebee 1y agoWe will work on a quantized version of the model, so hopefully you will be able to run it soon! We've seen Bark from Suno go from 16GB requirement -> 4GB requirement + running on CPUs. Won't be too hard, just need some time to work on it.
- sarangzambare 1y agoImpressive demo! We'd love to use this at https://useponder.ai https://useponder.ai time to first audio is something that is crucial for us to reduce the latency - wondering if dia works with output streaming? the python code snippet seems to imply that the entire audio bytes are generated directly?
- toebee 1y agoSounds awesome! I think it won't be very hard to run it using output streaming, although that might require beefier GPUs. Give us an email and we can talk more - nari.ai.contact at gmail dot com. It's way past bedtime where I live, so will be able to get back to you after a few hours. Thanks for the interest :)
- sarangzambare 1y agono worries, i will email you
- xienze 1y agoHow do you declare which voice should be used for a particular speaker? And can it created a cloned speaker voice from a sample?
- toebee 1y agoYou can add an audio prompt and prepend text corresponding to it in the script. You can get a feel for it by trying the second example in the Gradio interface!
- brumar 1y agoImpressive! Is it english only at the moment?
- toebee 1y agoUnfortunately yes at the moment
- toebee 1y agoIt is way past bedtime here, will be getting back to comments after a few hours of sleep! Thanks for all the kind words and feedback
- pzo 1y agoSounds great. Hope more language support in the future. In comparison Sesame CSM-1B sounds like trained on stoned people.
- film42 1y agoVery very impressive.
- notdian 1y agomade a small change and got it running on M2 Pro 16GB Macbook pro, the quality is amazing. https://github.com/nari-labs/dia/pull/4 https://github.com/nari-labs/dia/pull/4
- noiv 1y agoCan confirm, runs straight forward on 15.4.1@M4, THX.
- rahimnathwani 1y agoThank you for this! My desktop GPU has only 8GB VRAM, but my MacBook has plenty of unified RAM.
- toebee 1y agoThank you for the contribution! We'll be merging PRs and cleaning code up very soon :)
- emmelaich 1y agoThanks, works well but slowly on a Mac Air M3 with 24gb. Will have to try it again after freeing up more ram as it was doing a bit of swapping with Chrome running too. (later). It did nicely for the default example text but just made weird sounds for a "hello all" prompt. And took longer?!
- Versipelle 1y agoThis is really impressive; we're getting close to a dream of mine: the ability to generate proper audiobooks from EPUBs. Not just a robotic single voice for everything, but different, consistent voices for each protagonist, with the LLM analyzing the text to guess which voice to use and add an appropriate tone, much like a voice actor would do. I've tried "EPUB to audiobook" tools, but they are really miles behind what a real narrator accomplishes and make the audiobook impossible to engage with
- mclau157 1y agoRealistic voice acting for audio books, realistic images for each page, realistic videos for each page, oh wait I just created a movie, maybe I can change the plot? Oh wait I just created a video game
- hleszek 1y agoNow do it in VR and make it fully interactive.
- azinman2 1y agoWouldn’t it be more desirable to hear an actual human on an audiobook? Ideally the author?
- senordevnyc 1y agoHonestly, I’d say that’s true only for the author. Anyone else is just going to be interpreting the words to understand how to best convey the character / emotion / situation / etc., just like an AI will have to do. If an AI can do that more effectively than a human, why not? The author could be better, because they at least have other info beyond the text to rely on, they can go off-script or add little details, etc.
- DrSiemer 1y agoAs somebody who has listened to hundreds of audiobooks, I can tell you authors are generally not the best choice to voice their own work. They may know every intent, but they are writers, not actors. The most skilled readers will make you want to read books _just because they narrated them_. They add a unique quality to the story, that you do not get from reading yourself or from watching a video adaptation. Currently I'm in The Age of Madness, read by Steven Pacey. He's fantastic. The late Roy Dotrice is worth a mention as well, for voicing Game of Thrones and claiming the Guinness world record for most distinct voices (224) in one series. It will be awesome if we can create readings automatically, but it will be a while before TTS can compete with the best readers out there.
- tyrauber 1y agoHey, do yourself a favor and listen to the fun example: > [S1] Oh fire! Oh my goodness! What's the procedure? What to we do people? The smoke could be coming through an air duct! Seriously impressive. Wish I could direct link the audio. Kudos to the Dia team.
- jinay 1y agoFor anyone who wants to listen, it's on this page: https://yummy-fir-7a4.notion.site/dia https://yummy-fir-7a4.notion.site/dia
- mrandish 1y agoWow. Thanks for posting the direct link to examples. Those sound incredibly good and would be impressive for a frontier lab. For two people over a few months, it's spectacular.
- DoctorOW 1y agoA little overacted, it reminds me of the voice acting in those flash cartoons you'd see in the early days of YouTube. That's not to say it isn't good work, it still sounds remarkably human. Just silly humans :)
- selimthegrim 1y agoReminded me of the Fenslerfilm G.I. Joe sketch where the kids have something on the stove burning
- wisemang 1y agoStop all the downloading!
- Cthulhu_ 1y ago"flash cartoons in the early days of Youtube" Wouldn't those be straight from Newgrounds?
- 999900000999 1y agoDoes this only support English? I would absolutely love something like this for practicing Chinese, or even just adding Chinese dialogue to a project.
- buttercrab 1y agoHi! I'm Dia's developer. We currently only support English.
- verghese 1y agoHow does this compare with Spark TTS? https://github.com/SparkAudio/Spark-TTS https://github.com/SparkAudio/Spark-TTS
- youssefabdelm 1y agoAnyone know if possible to fine-tune for cloning my voice?
- toebee 1y agoWe're adding guides for Zero-shot voice cloning. You can try it using the second example on Gradio: https://huggingface.co/spaces/nari-labs/Dia-1.6B https://huggingface.co/spaces/nari-labs/Dia-1.6B
- youssefabdelm 1y agoWill give it a shot but I feel like fine-tuning will be more reliable, any way to do that?
- xbmcuser 1y agoWow first time I have felt that this could be the end of voice acting/audio book narration etc. The speed with with the ways things are changing how soon before you can make any book any novel into a complete audio video / movie or tv show.
- a2128 1y agoWhat's the training process like? I have some data in my language I'd love to use train it on my language seeing as it's English-only
- toebee 1y agoWe'll try to give a high-level overview when we publish the technical report!
- popalchemist 1y agoThis looks excellent, thank you for releasing openly.
- hemloc_io 1y agoVery cool! Insane how much low hanging fruit there is for Audio models right now. A team of two picking things up over a few months can build something that still competes with large players with tons of funding
- toebee 1y agoThank you for the kind words <3
- kreelman 1y agoThis is amazing. Is it possible to build in a chosen voice, a bit like Eleven Labs does? ...This may be on the git summary, being lazy and asking anyway :=) Thanks for your work.
- JonathanFly 1y agoYes, see: https://github.com/nari-labs/dia/blob/main/example/voice_clone.py https://github.com/nari-labs/dia/blob/main/example/voice_clo...
- miki123211 1y agoYeah, Eleven Labs must be raking it in. You can get hours of audio out of it for free with Eleven Reader, which suggests that their inference costs aren't that high. Meanwhile, those same few hours of audio, at the exact same quality, would cost something like $100 when generated through their website or API, a lot more than any other provider out there. Their pricing (and especially API pricing) makes no sense, not unless it's just price discrimination. Somebody with slightly deeper pockets than academics or one guy in a garage needs to start competing with them and drive costs down. Open TTS models don't even seem to utilize audiobooks or data scraped off the internet, most are still Librivox / LJ Speech. That's like training an LLM on just Wikipedia and expecting great results. That may have worked in 2018, but even in 2020 we knew better, not to mention 2025. TTS models never had their "Stable Diffusion moment", it's time we get one. I think all it would take is somebody doing open-weight models applying the lessons we learned from LLMs and image generation to TTS models, namely more data, more scraping, more GPUs, less qualms and less safety. Eleven Labs already did, and they're profiting from it handsomely.
- lostmsu 1y agoDoes this only work for two voices? Can I generate an entire conversation between multiple people? Like this HN thread.
- toebee 1y agoOnly two voices at the moment... We will need to upgrade the dataset to make that happen, and are considering that as one of the next steps.
- mclau157 1y agoWill you support the other side with AI voice detection software to detect and block malicious voice snippets?
- rustc 1y agoIs this Apache licensed or a custom one? The README contains this: > This project is licensed under the Apache License 2.0 - see the LICENSE file for details. > This project offers a high-fidelity speech generation model *intended solely for research and educational use*. The following uses are strictly forbidden: > Identity Misuse: Do not produce audio resembling real individuals without permission. > ... Specifically the phrase "intended solely for research and educational use".
- montroser 1y agoHmm, the "strictly forbidden" part seems more important than whatever are their stated intentions... Either way, it seems like it needs clarifying.
- deleted 1y ago[deleted]
- toebee 1y agoSorry for the confusion. the license is plain Apache 2.0, and we changed the wording to "intended for research and educational use." The point was, users are free to use it for their use cases, just don't do shady stuff with it. Thanks for the feedback :)
- zhyder 1y agoV v cool: first time I've seen such expressiveness in TTS for laughs, coughs, yelling about a fire, etc! What're the recommended GPU cloud providers for using such open-weights models?
- toebee 1y agoThanks you!! We personally used Quickpod and Runpod the most. But you can try it now on HF Spaces without spinning up GPUs yourself! https://huggingface.co/spaces/nari-labs/Dia-1.6B https://huggingface.co/spaces/nari-labs/Dia-1.6B
- JonathanFly 1y ago> first time I've seen such expressiveness in TTS for laughs, coughs, yelling about a fire, etc! The old Bark TTS is noisy and often unreliable, but pretty great at coughs, throat clears, and yelling. Even dialogs... sometimes. Same Dia prompt in Bark: https://vocaroo.com/12HsMlm1NGdv https://vocaroo.com/12HsMlm1NGdv Dia sounds much more clear and reliable, wild what 2 people can do in 3 months.
- codingmoh 1y agoHey, this is really cool! Curious how good the multi-language support is. Also - pretty wild that you trained the whole thing yourselves, especially without prior experience in speech models. Might actually be helpful for others if you ever feel like documenting how you got started and what the process looked like. I’ve never worked with TTS models myself, and honestly wouldn’t know where to begin. Either way, awesome work. Big respect.
- toebee 1y agoThank you so much for the kind words :) We only support English at the moment, hopefully can do more languages in the future. We are planning to release a technical report on some of the details, so stay tuned for that!
- bavell 1y agoI'd also love to peek behind the curtains, if only to satisfy my own curiosity. Looking forward to the technical report, well done!
- eob 1y agoBravo -- this is fantastic. I've been waiting for this ever since reading some interview with Orson Scott Card ages ago. It turns out he thinks of his novels as radio theater, not books. Which is a very different way to experience the audio.
- toebee 1y agoThanks for the kind words :)))
- noiv 1y agoThe demo page does fancy stuff when marking text and hitting cmd-d to create a bookmark :)
- vagabund 1y agoThe huggingface spaces link doesn't work, fyi. Sounds awesome in the demo page though.
- Havoc 1y agoSounds really good & human! Got a fair bit of unexpected artifacts though. e.g. 3 seconds hissing noise before dialogue. And music in background when I added (happy) in an attempt to control tone. Also don't understand how to control the S1 and S2 speakers...is it just random based on temp? > TODO Docker support Got this adapted pretty easily. Just latest nvidia cuda container, throw python and modules on it and change server to serve on 0.0.0.0. Does mean it pulls the model every time on startup though which isn't ideal
- yjftsjthsd-h 1y ago> Does mean it pulls the model every time on startup though which isn't ideal Surely it just downloads to a directory that can be volume mapped?
- Havoc 1y agoYep. I just didn't spend the time to track down the location tbh. Plus huggingface usually does links to a cache folder that I don't recall the location of Literally got cuda containers working earlier today so haven't spent a huge amount of time figuring things out
- genewitch 1y agoIts in a dot folder in your home dir on Linux and in %appdata% on windows.
- toebee 1y agoThank you for the kind words! Dia wasn’t fine tuned on certain speaker, so you will get random voices every time you run it, unless you add a prompt / fix the seed. The outputs are a bit unstable, might need to add cleaner training data and run longer training sessions. Hopefully we can do something like OAI Whisper and update with better performing checkpoints!
- dragonwriter 1y ago> Also don't understand how to control the S1 and S2 speakers... Do a clip with the speakers you want as the audio prompt, add the text of that clip (with speaker tags) of the clip at the beginning of your text prompt, and it clones the voices from your audio prompt for the output.
- jokethrowaway 1y agoLooking forward to try. My current go-to solution is E5-F2 (great cloning, decent delivery, ok audio quality, a lot of incoherence here and there forcing you to do multiple generations). I've just been massively disappointed by Sesame's CSM: on their gradio on the website it was generating flawless dialogs with amazing voice cloning. When running it local the voice cloning performance is awful.
- toebee 1y agoThanks for the interest! We also enjoyed using E5-F2 :) You can try it now on HF Spaces: https://huggingface.co/spaces/nari-labs/Dia-1.6B https://huggingface.co/spaces/nari-labs/Dia-1.6B
- xhkkffbf 1y agoAre there different voices? Or only [s1] and [s2] in the examples?
- toebee 1y agoWe just clarified in the README, sorry for the confusion ;( Note that the model was not fine-tuned on a specific voice. Hence, you will get different voices every time you run the model. You can keep speaker consistency by either adding an audio prompt (a guide coming VERY soon - try it with the second example on Gradio or HF Space for now), or fixing the seed.
- hiAndrewQuinn 1y agoIs this English-only? I'm looking for a local model for Finnish dialogue to run.
- dindindin 1y agoWas this trained on Planet Money / NPR podcasts? The last audio (continuation of prompt) sounds eerily like Planet Money, I had to double check if my Spotify had accidentally started playing.
- jelling 1y agoNPR voice is a thing. It started with Ira Glass voice and now the default voice is someone that sounds like they're not certain they should be saying the very banal thing they are about to say, followed by a hand-shake protocol of nervous laughter.
- genewitch 1y agoThank goodness for Scott Simon!
- instagary 1y agoDoes this use the the mimi codec by moshi? If so it would be straighforward to get Dia running on iOS!
- toebee 1y agoWe use descript audio codec! I’m not sure if DAC works on iOS…
- enodios 1y agoThe audio quality is seriously impressive. Any plans to add word-level timing maps? For my usecase that is a requirement, so unfortunately I cannot use this yet, but I would very much like to.
- toebee 1y agoThank you for the kind words! We don't have plans for that yet, but you can always open an issue or RP on Github.
- deleted 1y ago[deleted]
- user_4028b09 1y ago[dead]
- dangoodmanUT 1y agoHas the same issue of cutting off the end of the provided text that many other models have.
- oehtXRwMkIs 1y agoAny plans for AMD GPU support? Maybe I'm missing something, but it's not working out of the box on a 7900xtx.
- toebee 1y agoWe will try to make it work, but not sure if will be an easy task. For now, you can try with https://huggingface.co/spaces/nari-labs/Dia-1.6B https://huggingface.co/spaces/nari-labs/Dia-1.6B
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- toebee 1y agoWe have a ZeroGPU Space provided by HuggingFace up and running! Test it now on https://huggingface.co/spaces/nari-labs/Dia-1.6B https://huggingface.co/spaces/nari-labs/Dia-1.6B
- daemonologist 1y agoThe examples on your site are impressive, but I'm having trouble getting good results on HF - it's generating a lot of near-silence (often nothing but) and when it does produce speech it bears no resemblance to the audio prompt and only produces parts of the text prompt. Would you suggest any adjustments to the default parameters to improve adherence, or might I expect better results running locally? Thanks!
- qwertytyyuu 1y agoThat Sesame CSM-1B voice sounds sooo done with life, haha.
- sroussey 1y agoCan we get this working in the browser a la ONNX or similar?
- moritonal 1y agoIsn't it weird how "We don't have a full list of non-verbal [commands]". Like, I can imagine why, but it's wild we're at a point where we don't know what our code can do.
- kevmo314 1y agoI have a sneaking suspicion it's because they lifted the model architecture almost directly from Parakeet: https://jordandarefsky.com/blog/2024/parakeet/ https://jordandarefsky.com/blog/2024/parakeet/ Parakeet references WhisperD which is at https://huggingface.co/jordand/whisper-d-v1a https://huggingface.co/jordand/whisper-d-v1a and doesn't include a full list of non-speech events that it's been trained with, except "(coughs)" and "(laughs)". Not saying the authors didn't do anything interesting here. They put in the work to reproduce the blog post and open source it, a praiseworthy achievement in itself, and they even credit Parakeet. But they might not have the list of commands for more straightforward reasons.
- toebee 1y agoYou're absolutely right. We used Jordan's Whisper-D, and he was generous enough to offer some guidance along the way. It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s something we’ll be improving soon. As for Dia’s architecture, we largely followed existing models to build the 1.6B version. Since we only started learning about speech AI three months ago, we chose not to innovate too aggressively early on. That said, we're planning to introduce MoE and Sliding Window Attention in our larger models, so we're excited to push the frontier in future iterations.
- kamranjon 1y agoI’m curious what differentiates it from Parakeet? I was listening to some of the demos on the parakeet announcement and they sound very similar to your examples - are they trained on the same data? Are there benefits to using Dia over Parakeet?
- 1y ago
- user_4028b09 1y agoThis is a really impressive project – looking forward to trying it out!
- user_4028b09 1y ago[dead]
- flashblaze 1y agoI'm lost for words. This is extremely impressive!
- howon92 1y agoTraining an audio model this good from 0 prior experience is really amazing. I would love to read a blog post about how you guys approached ramping up knowledge and getting practical quickly. Any plans?
- benterix 1y agoWhoah!
- bazlan 1y agofluxions.ai has a similar model
- sebstefan 1y agoI inserted the non-verbal command "(pauses)" in the middle of a sentence and I think I caused it to have an aneurysm. https://i.horizon.pics/4sEVXh8GpI https://i.horizon.pics/4sEVXh8GpI (27s) It starts with an intro, too. Really strange
- throwaway-alpha 1y agoI have a hunch they're pulling data from radio shows to give it that "high quality" vibe. Tried running it through this script and hit some weird bugs too: [S1] It really sounds as if they've started using NPR to source TTS models [S2] Yeah... yeah... it's kind of disturbing (laughs dejectedly). [S3] I really wish, that they would just Stop with this. https://i.horizon.pics/Tx2PrPTRM3 https://i.horizon.pics/Tx2PrPTRM3
- degosuke 1y agoIt even added an extra f-word at the end. Still veeery impressive
- sebstefan 1y agoOh brother imagine using this and noticing 4 months in that it randomly throws f-words for flavor at your users
- yencabulator 1y agoI think it also said "maaan" at the end of the previous line. And speaks out lout the "dejectedly".
- xdfgh1112 1y agoJust noticed he says dejectedly too
- yahoozoo 1y agoThe “Yeah…” followed by an uncomfortably long pause then a second “Yeah…” killed me.
- gitroom 1y agopretty cool - love seeing open stuff like this come together so fast. you think all this will ever match what a real voice actor can pull off, or something totally new comes out of it?
- deleted 1y ago[deleted]
- elia_42 1y agoInteresting. I will definitely look into it further.
- jackchina 1y agotest good!
- basilgohar 1y agoSeeing is no longer believing. Hearing isn't either. The funny thing is, it's getting to the point where LLM-generated text is more easily spotted than AI audio, video, and images. It's going to be an interesting decade of the new equivalent of "No, Tiffany, Bill Gates will NOT be sending you $100 for forwarding that email." Except it's going to be AI celebrities making appeals for donations to help them become billionaires or something.
- gamificationpan 1y ago[dead]
- ibreakthecloud 1y ago[dead]
- devnen 1y agoThis is really impressive work and the dialogue quality is fantastic. For anyone wanting a quick way to spin this up locally with a web UI and API access, I put together a FastAPI server wrapper around the model: https://github.com/devnen/Dia-TTS-Server https://github.com/devnen/Dia-TTS-Server The setup is just a standard pip install -r requirements.txt (works on Linux/Windows). It pulls the model from HF automatically – defaulting to the faster BF16 safetensors (ttj/dia-1.6b-safetensors), but that's configurable in the .env. You get an OpenAI-compatible API endpoint (/v1/audio/speech) for easy integration, plus a custom one (/tts) to control all the Dia parameters. The web UI gives you a simple way to type text, adjust sliders, and test voice cloning. It'll use your CUDA GPU if you have one configured, otherwise, it runs on the CPU. Might be a useful starting point or testing tool for someone. Feedback is welcome!
- nicman23 1y agoif someone writes a wrapper with named entity recognition (with spacy for example) and a chapter tagging, it would be a great audiobook converter
- popalchemist 1y agoIs there a limit to the number of speakers?