3 ms·

A long time ago I thought that this kind of compression could be interesting for voice chat/phone calls. Locally convert speech to text, send raw text over the

A long time ago I thought that this kind of compression could be interesting for voice chat/phone calls. Locally convert speech to text, send raw text over the wire, and convert text back to speech. You could fine tune some existing text to speech model to sound like you. Once you transfer that to your friends/frequent contacts you could have voice conversations over very low bandwidth channels. There are obviously plenty of caveats to this, like communicating sounds that aren’t speech. It could be a useful fallback e.g. if your network quality is seriously degraded then your client switches to TTS mode.

namibj 5y ago
There are speech codecs that essentially do this but with a phonetic encoding, instead of "normal" text. Yes, they are only used for extremely bandwidth-starved situations, but they do work. An example is Codec2, which works down to 0.7 kbit/s.
Dylan16807 5y ago
Well there's already speech-oriented codecs that can go down below a kilobyte per second, and there's this one that throws a neural net at the problem to help squeeze bytes: https://ai.googleblog.com/2021/08/soundstream-end-to-end-neural-audio.html https://ai.googleblog.com/2021/08/soundstream-end-to-end-neu... I don't think I've ever had a functional connection slower than that. You need packets to be making it through in a timely manner if you want a voice conversation, after all.
- bigiain 5y ago
  > I don't think I've ever had a functional connection slower than that. I _may_ still have a 300 baud modem in a box in the garage. Possibly a 1200/75 one too. (1985 sometimes doesn't seem _that_ long ago...)
  - Dylan16807 5y ago
    Well, you could have just talked on the line that modem was using. That's not really the kind of limit I'm worried about.
matthewaveryusa 5y ago
Great idea! In fact look at what nvidia is doing with videocalls (spoiler exactly what you’re saying, but video) https://blogs.nvidia.com/blog/2020/10/05/gan-video-conferencing-maxine/ https://blogs.nvidia.com/blog/2020/10/05/gan-video-conferenc...

namibj 5y ago

There are speech codecs that essentially do this but with a phonetic encoding, instead of "normal" text. Yes, they are only used for extremely bandwidth-starved situations, but they do work. An example is Codec2, which works down to 0.7 kbit/s.

Dylan16807 5y ago

Well there's already speech-oriented codecs that can go down below a kilobyte per second, and there's this one that throws a neural net at the problem to help squeeze bytes: https://ai.googleblog.com/2021/08/soundstream-end-to-end-neural-audio.html https://ai.googleblog.com/2021/08/soundstream-end-to-end-neu... I don't think I've ever had a functional connection slower than that. You need packets to be making it through in a timely manner if you want a voice conversation, after all.

bigiain 5y ago
> I don't think I've ever had a functional connection slower than that. I _may_ still have a 300 baud modem in a box in the garage. Possibly a 1200/75 one too. (1985 sometimes doesn't seem _that_ long ago...)
- Dylan16807 5y ago
  Well, you could have just talked on the line that modem was using. That's not really the kind of limit I'm worried about.

matthewaveryusa 5y ago

Great idea! In fact look at what nvidia is doing with videocalls (spoiler exactly what you’re saying, but video) https://blogs.nvidia.com/blog/2020/10/05/gan-video-conferencing-maxine/ https://blogs.nvidia.com/blog/2020/10/05/gan-video-conferenc...