4 ms·

I'm looking forward to having an end-to-end "docker compose up" solution for self hosted chatgpt conversational voice mode. This is probably possible today, wit

I'm looking forward to having an end-to-end "docker compose up" solution for self hosted chatgpt conversational voice mode. This is probably possible today, with enough glue code, but I haven't seen a neatly wrapped solution yet on par with ollama's.

thot_experiment 2y ago
Open WebUI has this already. Works okay, there's definitely a lot a fair bit of latency tho.
nickthegreek 2y ago
You can glue it with home assistant right now, but it’s not a simple docker compose. Piper TTS and Kokoro were the main 2 voice engines people are using. Orpheus would be great to get wired up. I’m wondering how well their smallest model will run and if it will be fast enough for realtime
- Zetaphor 2y ago
  With some tweaking I was able to get the current 3B's "realtime" streaming demo running on my 12GB 4070 Super with about a second of latency running at BF16
tough 2y ago
harbor is a great docker bedrock for llm tools, has some tts stuff havent tried them https://github.com/av/harbor/wiki/1.1-Harbor-App#installation https://github.com/av/harbor/wiki/1.1-Harbor-App#installatio...