3 ms·

Working to enable users of https://www.DreamList.com https://www.DreamList.com to record audio of any length and see it transcribed, ideally at the same time as

by dzink 5y ago

Working to enable users of https://www.DreamList.com https://www.DreamList.com to record audio of any length and see it transcribed, ideally at the same time as the recording, while the recording is also saved. The goal is for grandparents to save stories for loved ones and not worry about quality of the transcription - just talk. When the recording is saved, the transcription can be redone or tweaked if needed later, but the memory is not lost. DreamList is web and soon native apps, so WebRTC connected to a cloud transcription service is my first instinct, but there are benefits to native iOS apis as well - especially being able to keep share stories while listening to other streams also on iOS (families talking and digging into stories together). What architecture/transcription approaches would you suggest? Any gotchas you've seen dealing with similar problems (accuracy given accents, do we train our own transcription based on gathered data, etc)?

skoocda 5y ago
I worked on this for a couple years during a previous startup attempt. I designed a custom STT model via Kaldi [0] and hosted it using a modified version of this server [1]. I deployed it to a 4GB EC2 instance with configurable docker layers (one for core utils, one for speech utils, one for the model) so we could spin up as many servers as we needed for each language. I would recommend the WebRTC or Gstreamer approach, but I wouldn't recommend trying to build your own model. It's really hard. Google's Cloud API [2] works well across lots of accents and the price is honestly about the same as running your own server. If you want to host your own STT (for privacy or whatever), I'd recommend using Coqui [3] (from the guys that ran Mozilla's OpenSpeech program). Note that this will likely be much, much worse on accents than Google's model. [0]: https://kaldi-asr.org/ https://kaldi-asr.org/ [1]: https://github.com/alumae/kaldi-gstreamer-server https://github.com/alumae/kaldi-gstreamer-server [2]: https://cloud.google.com/speech-to-text https://cloud.google.com/speech-to-text [3]: https://coqui.ai/code https://coqui.ai/code Edit: Forgot to mention, there's also a YC company called Deepgram that provides ASR/STT as a service, you could give them a shot: https://deepgram.com/ https://deepgram.com/
- mkl 5y ago
  In my experience, Google's API completely fails when any slightly unusual vocabulary is involved (e.g. in this instance, grandparents talking about their past jobs), and tends to just silently skip over things. Amazon's wasn't much better with vocab., but at least didn't leave things out, so you could see problems. I don't have experience with any of these others, but I think for my purposes (subtitles for maths education videos) no one will have made an appropriate model yet.

by dzink 5y ago

skoocda 5y ago

I worked on this for a couple years during a previous startup attempt. I designed a custom STT model via Kaldi [0] and hosted it using a modified version of this server [1]. I deployed it to a 4GB EC2 instance with configurable docker layers (one for core utils, one for speech utils, one for the model) so we could spin up as many servers as we needed for each language. I would recommend the WebRTC or Gstreamer approach, but I wouldn't recommend trying to build your own model. It's really hard. Google's Cloud API [2] works well across lots of accents and the price is honestly about the same as running your own server. If you want to host your own STT (for privacy or whatever), I'd recommend using Coqui [3] (from the guys that ran Mozilla's OpenSpeech program). Note that this will likely be much, much worse on accents than Google's model. [0]: https://kaldi-asr.org/ https://kaldi-asr.org/ [1]: https://github.com/alumae/kaldi-gstreamer-server https://github.com/alumae/kaldi-gstreamer-server [2]: https://cloud.google.com/speech-to-text https://cloud.google.com/speech-to-text [3]: https://coqui.ai/code https://coqui.ai/code Edit: Forgot to mention, there's also a YC company called Deepgram that provides ASR/STT as a service, you could give them a shot: https://deepgram.com/ https://deepgram.com/

mkl 5y ago
In my experience, Google's API completely fails when any slightly unusual vocabulary is involved (e.g. in this instance, grandparents talking about their past jobs), and tends to just silently skip over things. Amazon's wasn't much better with vocab., but at least didn't leave things out, so you could see problems. I don't have experience with any of these others, but I think for my purposes (subtitles for maths education videos) no one will have made an appropriate model yet.