3 ms·
What is the latest/greatest when it comes to voice models? Orpheus is what people have the most success with? I only concentrate on the WebRTC side, not aware
by Sean-Der 2y ago
What is the latest/greatest
when it comes to voice models? Orpheus is what people have the most success with?
I only concentrate on the WebRTC side, not aware of all the offerings.
- jackndwyer 2y agoOrpheus is the first "good" open-source model. The big players thus far have all be research labs (Cartesia, OpenAI, Play, Eleven), but with Orpheus and Sesame (another one, not quite production ready), the market is changing
- keepinitneil 2y agoOrpheus is good. It's a very simple architecture, basically just llama3b trained to produce audio (SNAC) tokens instead of text tokens. This is nice because it benefits from the Llama open-source ecosystem which is very active. It being LLM based is also nice because it streams output and with some engineering effort supports continuation via prompt-filling (so streaming input can work). The other one people have been excited about is Sesame. Both are very expressive and have much more room for optimization (both on the inference efficiency side quants, finetunes, distillation, etc) and on the quality side (throw more data at it). Exciting times!