5 ms·

Show HN: Make your own voice AI in two clicks

27 points by unmute-sh 1y ago

Upload a voice and write a personality prompt, or try the pre-made characters. Built by augmenting Gemma 3 12B with our new text-to-speech and speech-to-text models, both of which we will release as open-source soon. Stay tuned.

smallflykkoo 1y ago
[dead]
ton4eg 1y ago
Way more entertaining than I would expect! What TTS and ASR models do you use? What sort of latency do you get?
- unmute-sh 1y ago
  Thank you! The TTS and ASR are our own unreleased models, but we'll open-source them soon :) The latency is about 500ms once we detect that it's the bot's turn to speak (roughly 200ms for the LLM's time-to-first token and 300ms for the TTS audio to start), plus a variable time for the semantic pause detection (VAD). If it's clear that you're done talking, like when you ask a question, the model will reply very fast. If you stop mid-sentence as if you have more to say, it will wait for longer to avoid interrupting you.
lightbulbish 1y ago
I thought this was fantastic! Surprised not more people are commenting on this. Is there a reason I am not aware of? To the author: what happens to my voice after I upload it? What is your plan moving forward? I am too far left field to understand how to build a business and monetize an open source product like this, even though I found it fun to play around with.
- unmute-sh 1y ago
  Thanks! There is a model that turns the voice into an embedding that is used to determine the voice. Unlike the STT and TTS, we won't be releasing the weights of this voice cloning model, but we will provide it over an API so that we can do verification and prevent abuse. edit: Ah yes, and we do not store the voice sample on our server. The voice embedding is cached for 24 hours.
karim79 1y ago
Incredible work. Short, sweet and simple. I hadn't expected to enjoy this as much as I did. I can't wait to see where it goes.
android521 1y ago
can't wait for the open source release.
xingwu 1y ago
Simple, functional, perfect.
marnesh 1y ago
Wow this is actually pretty amazing, It is so natural
marnesh 1y ago
This is really amazing, it is so natural

27 points by unmute-sh 1y ago

smallflykkoo 1y ago

[dead]

ton4eg 1y ago

Way more entertaining than I would expect! What TTS and ASR models do you use? What sort of latency do you get?

unmute-sh 1y ago
Thank you! The TTS and ASR are our own unreleased models, but we'll open-source them soon :) The latency is about 500ms once we detect that it's the bot's turn to speak (roughly 200ms for the LLM's time-to-first token and 300ms for the TTS audio to start), plus a variable time for the semantic pause detection (VAD). If it's clear that you're done talking, like when you ask a question, the model will reply very fast. If you stop mid-sentence as if you have more to say, it will wait for longer to avoid interrupting you.

lightbulbish 1y ago

I thought this was fantastic! Surprised not more people are commenting on this. Is there a reason I am not aware of? To the author: what happens to my voice after I upload it? What is your plan moving forward? I am too far left field to understand how to build a business and monetize an open source product like this, even though I found it fun to play around with.

unmute-sh 1y ago
Thanks! There is a model that turns the voice into an embedding that is used to determine the voice. Unlike the STT and TTS, we won't be releasing the weights of this voice cloning model, but we will provide it over an API so that we can do verification and prevent abuse. edit: Ah yes, and we do not store the voice sample on our server. The voice embedding is cached for 24 hours.

karim79 1y ago

Incredible work. Short, sweet and simple. I hadn't expected to enjoy this as much as I did. I can't wait to see where it goes.

android521 1y ago

can't wait for the open source release.

xingwu 1y ago

Simple, functional, perfect.

marnesh 1y ago

Wow this is actually pretty amazing, It is so natural

marnesh 1y ago

This is really amazing, it is so natural