3 ms·

I wonder if they will release improvements to voice in ChatGPT. I have been working on a local AI with voice and it responds much, much faster than ChatGPT[1].

I wonder if they will release improvements to voice in ChatGPT. I have been working on a local AI with voice and it responds much, much faster than ChatGPT[1]. OpenAI's voices sound amazing and GPT-4 is still the best chat model, but despite that it's a lot more fun to have a conversation with reasonable latency. I expect it's only a matter of time before OpenAI does something better, I'm actually surprised they released it in such a slow state. [1] For anyone who has a gaming PC with 12GB+ Nvidia GPU and wants to try: https://www.microsoft.com/store/apps/9NC624PBFGB7 https://www.microsoft.com/store/apps/9NC624PBFGB7

surprisetalk 3y ago
I've been extremely happy with their generated voice! I had GPT read something to me in the car. My wife was nonplussed with my "audiobook" until I told her that you can talk to it like Siri. Then she called it "scary" haha
- modeless 3y ago
  The voices sound great! But the latency is too high, and it's clunky to use with voice alone, it only listens to you at specific times and you can't interrupt it with your voice. I wanted something that felt more like having a casual conversation with a real person, instead of "Siri but smarter". And I know it's possible because I built something closer to what I want.
  - hfjjbf 3y ago
    I’m really curious how much of the speech to text is happening on device, I’m fairly sure the answer is “none”. That would have a fairly immediate performance boost. Right now it is cool as hell, just way too slow to be truly useful.
    - modeless 3y ago
      Yeah I expect the speech recognition is happening in the cloud. But that doesn't mean it necessarily has to be slow. You can stream audio with very low latency on most connections. The problem is what they do with the audio once it gets to the server. I suspect it would be far too expensive to dedicate a GPU to each customer so they need to run everything in batch mode which increases latency.
- DeWilde 3y ago
  Was this just ChatGPT or a custom implementation with access to some data?
  - surprisetalk 3y ago
    Normal ChatGPT. I asked it a question about the Soviet Union, and the response was lengthy
CSMastermind 3y ago
I'd really like to be able to use voice with the web version of ChatGPT
- modeless 3y ago
  Yeah it's weird they haven't added that yet.

surprisetalk 3y ago

I've been extremely happy with their generated voice! I had GPT read something to me in the car. My wife was nonplussed with my "audiobook" until I told her that you can talk to it like Siri. Then she called it "scary" haha

modeless 3y ago
The voices sound great! But the latency is too high, and it's clunky to use with voice alone, it only listens to you at specific times and you can't interrupt it with your voice. I wanted something that felt more like having a casual conversation with a real person, instead of "Siri but smarter". And I know it's possible because I built something closer to what I want.
- hfjjbf 3y ago
  I’m really curious how much of the speech to text is happening on device, I’m fairly sure the answer is “none”. That would have a fairly immediate performance boost. Right now it is cool as hell, just way too slow to be truly useful.
  - modeless 3y ago
    Yeah I expect the speech recognition is happening in the cloud. But that doesn't mean it necessarily has to be slow. You can stream audio with very low latency on most connections. The problem is what they do with the audio once it gets to the server. I suspect it would be far too expensive to dedicate a GPU to each customer so they need to run everything in batch mode which increases latency.
DeWilde 3y ago
Was this just ChatGPT or a custom implementation with access to some data?
- surprisetalk 3y ago
  Normal ChatGPT. I asked it a question about the Soviet Union, and the response was lengthy

CSMastermind 3y ago

I'd really like to be able to use voice with the web version of ChatGPT

modeless 3y ago
Yeah it's weird they haven't added that yet.