5 ms·

Ask HN: Can you please add transcription to vocal messages in messaging apps?

I know that there are employees of big tech companies lurking around here. I have a feature request. Can you please add the ability to see the transcript of a vocal message on messaging apps like Teams, Signal, Telegram, Messenger, Discord, Slack, Whatsapp, etc? I know that live transcription is already a thing so it must be feasible, right? This feature would be super useful to see what was said without having to playback a bunch of vocal messages. I had the idea of doing this myself a long time ago but I basically needed to recreate the entire messaging infrastructure just to add this feature. So I donate the idea here.

lowkj 5y ago
100% agree, personally I just prefer typing but seems like I am not in the majority. I very often receive audio replies to my typed messages and it’s very frustrating. 70% of the time, I’m not in a position where I can easily listen to the audio vs just reading text. The other 30% of the time I’m simply annoyed as I do not want to be forced to listen to the audio.
- nassimsoftware 5y ago
  I'd like to add to this that the transcription doesn't even need to be 100% accurate to be useful. It'll mostly be used to get the gist of what was said and target more specifically which audio message requires to be re-listened to instead of having to listen to everything again.
  - 0max 5y ago
    You just described the Google Voice voicemail experience
- ASalazarMX 5y ago
  I don't even need a TTS feature, just let me mark my chat as only-text please. People that send audios tend to consume more of my time than people who write because they are making up their mind as they speak. They seem to think: why waste my time putting my thoughts into words when I can waste everyone else's time while I figure out my ideas?
deleted 5y ago
[deleted]
trampish 5y ago
This would be a big step forward in improving accessibility for the hard of hearing.
ximeng 5y ago
Wechat has this feature
filipeisho 5y ago
Zenly also has this feature
paxys 5y ago
FYI Slack does this already for all uploaded audio & video clips, and it works great.
- nassimsoftware 5y ago
  Would be great if they added this for audio clips. Does Slack support vocal messaging? It has been a long time since I used Slack.
  - paxys 5y ago
    Yup, it does
cianmm 5y ago
This seems like such an obvious Appley feature for iMessage, especially now that transcription can be done on-device.
- ximeng 5y ago
  They've just added automatic OCR of photos as well to allow copy and paste from pictures of text.
- natdempk 5y ago
  Yeah, they already do it for voicemails as well which is nice, so it's surprising they wouldn't take the next step and implement it for iMessage voice. Though I will say the voicemail transcriptions aren't amazing in quality.
mbesto 5y ago
Isn't this a privacy issue with E2E encryption?
- nassimsoftware 5y ago
  If it's done on device is there still a security risk?
- ARandomUserOnHN 5y ago
  Not as long as the transcript is created on device.
- caymanjim 5y ago
  Voice-to-text transcription can be done entirely on-device, although I'm not sure what the quality of existing solutions is. This is a barrier but not a showstopper.
  - toast0 5y ago
    The questions then become --- can you trust the system on-device transcription to preserve privacy or can you justify the size of the datasets for transcription if you've got to include it/download it for secure offline use; and can you make it work reasonably well in the languages your users use. (bonus points if it can reasonably detect the spoken language which may not match the written language of the phone).
- solarkraft 5y ago
  If it’s sent to the cloud still only kind of, since many of these services aren’t E2EE to begin with.
- natdempk 5y ago
  At least for iOS they already do on-device transcription for voicemails, but not voice messages sent via other means.
  - elcano 5y ago
maerF0x0 5y ago
Twilio has a few ways to do this, including audio that streams through Twilio. https://www.twilio.com/speech-recognition https://www.twilio.com/speech-recognition Disclaimer: I work for twilio, but am not representing them in this post
juusto 5y ago
I would love the opposite: text messages that can be go from text-to-speech. Specially useful when doing chores and those long threads some people write. Or one of those busy groups where everyone is saying something. Just hit play and let the phone read them all. (I use WhatsApp) Or is this a feature I am unaware on phones?
- tata71 5y ago
  It is.
- kpozin 5y ago
  I think you can ask Siri or Google Assistant to read your new messages (by accessing the phone notification contents), but I'm not aware of in-app features.
- chefkoch 5y ago
  Google Assistent can read out your messages, but i haven't tried out of it can read from one specific group chat.
- scoot 5y ago
  Siri can read text from apps (messages, email, etc.): "Hey Siri, read the last WhatsApp from John". It will ask permission for access the first time you try.
zwog 5y ago
There are several Telegram bots for this. But of course very questionable in terms of privacy. I had the idea to write one myself, I found an open source project to host it myself, but I don't remember the name. Fortunately, the voice messages have largely stopped with my contacts, so I didn't pursue it any furth
peterhi 5y ago
Oh yes, I've forgotten things and looked at my phone and gone "I know it's on here, which app was it?" Make all the messages texts searchable And whilst we are at it: TRANSCRIBE THE PHONE CALLS TOO!!!
- solarkraft 5y ago
  Basically transcribe everything and make it searchable. Wish that was more common for podcasts and videos.
  - Sophira 5y ago
    There's a really important point that everybody is missing here. Transcribing phone calls is, in a sense, equivalent to recording a call. Having automatic transcription may not even be legal without letting the other party know.
ksec 5y ago
May be this should be an OS feature rather than a Per App feature? I wish they could transcribe my Voice Mail too. Whenever I have a voice mail, auto call the mail box and transcribe it for me. 99% of it are junk. But just in case there is one real voice message.
solarkraft 5y ago
Please. It’s so (seemingly) obvious that I don’t understand why it’s not a thing yet. These people aren’t idiots, are they? This is obvious, isn’t it? Is it organizational failure? Or is the technology really, actually, not there yet?
- 40four 5y ago
  I think you’re overthinking it. It’s certainly feasible, I’m sure it’s been thought of and brought up many times in meetings, It’s just not profitable. The answer, like with many things, is simply money I believe. The compute resources needed to implement & maintain this type of feature at scale, for even a single platform would probably cost many millions of dollars. So unfortunately, I doubt we will get this anytime soon. They would have to spend huge amounts of money & resources to add this, for essentially no financial gain.
olaola2 5y ago
fdfd
an21mm 5y ago
zenly lets you do it already

lowkj 5y ago

100% agree, personally I just prefer typing but seems like I am not in the majority. I very often receive audio replies to my typed messages and it’s very frustrating. 70% of the time, I’m not in a position where I can easily listen to the audio vs just reading text. The other 30% of the time I’m simply annoyed as I do not want to be forced to listen to the audio.

nassimsoftware 5y ago
I'd like to add to this that the transcription doesn't even need to be 100% accurate to be useful. It'll mostly be used to get the gist of what was said and target more specifically which audio message requires to be re-listened to instead of having to listen to everything again.
- 0max 5y ago
  You just described the Google Voice voicemail experience
ASalazarMX 5y ago
I don't even need a TTS feature, just let me mark my chat as only-text please. People that send audios tend to consume more of my time than people who write because they are making up their mind as they speak. They seem to think: why waste my time putting my thoughts into words when I can waste everyone else's time while I figure out my ideas?

deleted 5y ago

[deleted]

trampish 5y ago

This would be a big step forward in improving accessibility for the hard of hearing.

ximeng 5y ago

Wechat has this feature

filipeisho 5y ago

Zenly also has this feature

paxys 5y ago

FYI Slack does this already for all uploaded audio & video clips, and it works great.

nassimsoftware 5y ago
Would be great if they added this for audio clips. Does Slack support vocal messaging? It has been a long time since I used Slack.
- paxys 5y ago
  Yup, it does

cianmm 5y ago

This seems like such an obvious Appley feature for iMessage, especially now that transcription can be done on-device.

ximeng 5y ago
They've just added automatic OCR of photos as well to allow copy and paste from pictures of text.
natdempk 5y ago
Yeah, they already do it for voicemails as well which is nice, so it's surprising they wouldn't take the next step and implement it for iMessage voice. Though I will say the voicemail transcriptions aren't amazing in quality.

mbesto 5y ago

Isn't this a privacy issue with E2E encryption?

nassimsoftware 5y ago
If it's done on device is there still a security risk?
ARandomUserOnHN 5y ago
Not as long as the transcript is created on device.
caymanjim 5y ago
Voice-to-text transcription can be done entirely on-device, although I'm not sure what the quality of existing solutions is. This is a barrier but not a showstopper.
- toast0 5y ago
  The questions then become --- can you trust the system on-device transcription to preserve privacy or can you justify the size of the datasets for transcription if you've got to include it/download it for secure offline use; and can you make it work reasonably well in the languages your users use. (bonus points if it can reasonably detect the spoken language which may not match the written language of the phone).
solarkraft 5y ago
If it’s sent to the cloud still only kind of, since many of these services aren’t E2EE to begin with.
natdempk 5y ago
At least for iOS they already do on-device transcription for voicemails, but not voice messages sent via other means.
- elcano 5y ago

maerF0x0 5y ago

Twilio has a few ways to do this, including audio that streams through Twilio. https://www.twilio.com/speech-recognition https://www.twilio.com/speech-recognition Disclaimer: I work for twilio, but am not representing them in this post

juusto 5y ago

I would love the opposite: text messages that can be go from text-to-speech. Specially useful when doing chores and those long threads some people write. Or one of those busy groups where everyone is saying something. Just hit play and let the phone read them all. (I use WhatsApp) Or is this a feature I am unaware on phones?

tata71 5y ago
It is.
kpozin 5y ago
I think you can ask Siri or Google Assistant to read your new messages (by accessing the phone notification contents), but I'm not aware of in-app features.
chefkoch 5y ago
Google Assistent can read out your messages, but i haven't tried out of it can read from one specific group chat.
scoot 5y ago
Siri can read text from apps (messages, email, etc.): "Hey Siri, read the last WhatsApp from John". It will ask permission for access the first time you try.

zwog 5y ago

There are several Telegram bots for this. But of course very questionable in terms of privacy. I had the idea to write one myself, I found an open source project to host it myself, but I don't remember the name. Fortunately, the voice messages have largely stopped with my contacts, so I didn't pursue it any furth

peterhi 5y ago

Oh yes, I've forgotten things and looked at my phone and gone "I know it's on here, which app was it?" Make all the messages texts searchable And whilst we are at it: TRANSCRIBE THE PHONE CALLS TOO!!!

solarkraft 5y ago
Basically transcribe everything and make it searchable. Wish that was more common for podcasts and videos.
- Sophira 5y ago
  There's a really important point that everybody is missing here. Transcribing phone calls is, in a sense, equivalent to recording a call. Having automatic transcription may not even be legal without letting the other party know.

ksec 5y ago

May be this should be an OS feature rather than a Per App feature? I wish they could transcribe my Voice Mail too. Whenever I have a voice mail, auto call the mail box and transcribe it for me. 99% of it are junk. But just in case there is one real voice message.

solarkraft 5y ago

Please. It’s so (seemingly) obvious that I don’t understand why it’s not a thing yet. These people aren’t idiots, are they? This is obvious, isn’t it? Is it organizational failure? Or is the technology really, actually, not there yet?

40four 5y ago
I think you’re overthinking it. It’s certainly feasible, I’m sure it’s been thought of and brought up many times in meetings, It’s just not profitable. The answer, like with many things, is simply money I believe. The compute resources needed to implement & maintain this type of feature at scale, for even a single platform would probably cost many millions of dollars. So unfortunately, I doubt we will get this anytime soon. They would have to spend huge amounts of money & resources to add this, for essentially no financial gain.

olaola2 5y ago

fdfd

an21mm 5y ago

zenly lets you do it already