3 ms·

Is there an open-source or paid SDK/API that I can use to create a group voice chat mobile app with "live" transcription? Or something that can plug-in to a sys

Is there an open-source or paid SDK/API that I can use to create a group voice chat mobile app with "live" transcription? Or something that can plug-in to a system like this? I looked at Twilio but they seem to only offer a means to do it on their VOIP/SIP product.

Y_Y 6y ago
Google Meet does this
dcsan 6y ago
how real-time do you need it? if you use a streaming API you can even use google and there isn't too much lag, and it's continuous. Agora also talk about this, but I haven't used it myself https://www.agora.io/en/ https://www.agora.io/en/
whimsicalism 6y ago
> open-source or paid SDK/API that I can use to create a group voice chat mobile app with "live" transcription? Or something that can plug-in to a system like this? Yes, Google, Amazon, Microsoft all offer streaming solutions (wouldn't recommend Amazon's however, might recommend Microsoft over Google). wav2letter from FB is the only open-source framework worth looking at, deepspeech is not a seriously usable framework.
- woodson 6y ago
  Check out Kaldi. It's a toolkit rather than a ready-to-deploy service but has some solid pretrained models and recipes for training your own. You can use various existing projects for deployment, e.g. vosk-server (also for on-device) which comes with models for various languages and accents and has an excellent support channel via telegram. Quite frankly, despite not being "end-to-end", you'll get much much better results in practice.
  - whimsicalism 6y ago
    I collected custom audio and had it transcribed by hand for cash, then evaluated it on wav2letter and vosk. At least for that domain, wav2letter outperforms vosk.
    - woodson 6y ago
      Good for you, it's the only way to know which tool works best in your case. I did the same for my use case and arrived at the opposite conclusion. What most people don't realize is that it heavily depends on your use case and domain whether any given model/algorithm will work better.
- deskamess 6y ago
  Curious why would you not recommend Amazon... is it cost or something else.
  - whimsicalism 6y ago
    For my use case, quality subpar compared to the other cloud providers
sbr464 6y ago
Telnyx has media forking, the ability to clone a media stream in real time without affecting the original call. It allows receiving the stream directly and operating on it without latency. Not sure if relevant though, it's using their SIP product also. If the original service isn't using Telnyx, you could get creative and have a Telnyx shadow user join the group call to receive the stream, etc.

Y_Y 6y ago

Google Meet does this

dcsan 6y ago

how real-time do you need it? if you use a streaming API you can even use google and there isn't too much lag, and it's continuous. Agora also talk about this, but I haven't used it myself https://www.agora.io/en/ https://www.agora.io/en/

whimsicalism 6y ago

> open-source or paid SDK/API that I can use to create a group voice chat mobile app with "live" transcription? Or something that can plug-in to a system like this? Yes, Google, Amazon, Microsoft all offer streaming solutions (wouldn't recommend Amazon's however, might recommend Microsoft over Google). wav2letter from FB is the only open-source framework worth looking at, deepspeech is not a seriously usable framework.

woodson 6y ago
Check out Kaldi. It's a toolkit rather than a ready-to-deploy service but has some solid pretrained models and recipes for training your own. You can use various existing projects for deployment, e.g. vosk-server (also for on-device) which comes with models for various languages and accents and has an excellent support channel via telegram. Quite frankly, despite not being "end-to-end", you'll get much much better results in practice.
- whimsicalism 6y ago
  I collected custom audio and had it transcribed by hand for cash, then evaluated it on wav2letter and vosk. At least for that domain, wav2letter outperforms vosk.
  - woodson 6y ago
    Good for you, it's the only way to know which tool works best in your case. I did the same for my use case and arrived at the opposite conclusion. What most people don't realize is that it heavily depends on your use case and domain whether any given model/algorithm will work better.
deskamess 6y ago
Curious why would you not recommend Amazon... is it cost or something else.
- whimsicalism 6y ago
  For my use case, quality subpar compared to the other cloud providers

sbr464 6y ago

Telnyx has media forking, the ability to clone a media stream in real time without affecting the original call. It allows receiving the stream directly and operating on it without latency. Not sure if relevant though, it's using their SIP product also. If the original service isn't using Telnyx, you could get creative and have a Telnyx shadow user join the group call to receive the stream, etc.