4 ms·

>Voice assistants are basically just mainstream non-visual command-lines, and it's unsurprising to me that something that relies heavily on memorization and ext

by _dain_ 4y ago

>Voice assistants are basically just mainstream non-visual command-lines, and it's unsurprising to me that something that relies heavily on memorization and extremely specialized "skills" isn't quite taking off in the way it was imagined. This got me thinking. Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager. Now there's just the language part. Thing is, I don't want to speak to my computer using English. Aside from the enormous practical problems in natural language processing you've outlined, I just find the idea creepy[1]. What I want is to unambiguously tell it to do arbitrary things. I.e. use it as an actual computer, not a toy that can do a few tricks. I.e. actually program it. In some kind of Turing complete shell language that is optimized for being spoken aloud. You would speak words into the open source voice recognizer, it writes those to stdout, then an interpreter reads from stdin and executes the instructions. Is there any language like this? What should it look like? And yeah that would take effort to learn to use it right, just like any other programming language; so be it. This would be a hobbyist thing. [1] https://i.kym-cdn.com/photos/images/original/002/054/961/748.jpg https://i.kym-cdn.com/photos/images/original/002/054/961/748...

viraptor 4y ago
> So the recognition part is solved If you're using an averaged American voice - maybe. But it's really not solved for everyone. Google assistant can't set the right timer for me 1/10 times. And that's before we get to heavy accent Scots and others.
- bambax 4y ago
  Obligatory reference: https://www.youtube.com/watch?v=NMS2VnDveP8 https://www.youtube.com/watch?v=NMS2VnDveP8
- arethuza 4y ago
  Even my "affected Edinburgh accent", as someone once described it, causes no end of trouble with voice recognition.
draugadrotten 4y ago
> And yeah that would take effort to learn to use it right, just like any other programming language; so be it. This would be a hobbyist thing. There are quite a few hobbyists working on local on-prem privacy focused voice assistans with conversation support. https://www.home-assistant.io/integrations/#voice https://www.home-assistant.io/integrations/#voice https://www.home-assistant.io/integrations/conversation/ https://www.home-assistant.io/integrations/conversation/ Have fun. It is a rabbit hole.
Shank 4y ago
> Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager. I personally don't consider this a fully-solved problem. The best transcription system I've used is OpenAI Whisper, and it doesn't work in realtime. Maybe it's fine on small amounts but it's still not perfect. You really need error to be driven down dramatically. Zoom auto-captions are a joke in terms of how badly they work for me, and Live Text (beta) on macOS is equally dreadful. YouTube auto-captions suck. All of these use industry-leading APIs. If I'm speaking a voice command and one single word is wrong, usually the whole thing fails. There's an entirely separate issue about things that are Proper Nouns that don't exist. For example, "Todoist" is often misunderstood by Siri. Thus, people started saying "Two doist (where doist rhymes with joist)" to fool it into understanding "Todoist". Media like anime with strange titles from other languages often flat out trolls these transcription systems. ("Hey Siri, remind me to watch Kimetsu no Yaiba tomorrow".)
Aramgutang 4y ago
That reminds me of the handwriting recognition approach [1] used in old Palm Pilot devices. Even though the shapes it expected you to draw resembled the corresponding letters, you would never draw them like that if you were writing on paper. You knew that you were drawing something designed for a computer to recognise as unambiguously as possible, while being efficient to draw quickly and easy to learn for you. I feel like that's the kind of notion that voice interfaces should somehow expand upon. [1] https://en.wikipedia.org/wiki/Graffiti_(Palm_OS) https://en.wikipedia.org/wiki/Graffiti_(Palm_OS)
simiones 4y ago
> Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager. This is potentially far from true, depending on how exactly you draw the line between "voice recognition" and "language". I've looked at quite a few transcription services, and they fail a lot of the time for most people - those who either have a non-native accent (even if very slight!) or those who do any amount of stammering or other vocal tics.
- ghaff 4y ago
  I find the ML transcription services, given 2 people speaking English with high quality sound and without heavy accents/a lot of jargon, to be adequate for having a skimmable record--such as for extracting quotations (and just go back to the recording to confirm the exact words if it's not obvious). But if I'm publishing a transcript I get a human transcription. Cleaning up the ML stuff takes way too much time and I wouldn't publish a transcript without cleaning it up.
  - simiones 4y ago
    I was in fact looking at some transcriptions of my recent meetings, and found one that captures how even small mistakes can make for completely not-understandable transcripts, unless they are manually cleaned up. Manual transcription: > So no: long story short, Slum is basically the way we can have an individual [, uhhh,] instance that carries all the licenses. (Slum is a project name in this case) Computer transcription (MS Teams): > So no. > A long story shorts. Love is basically the way we can have an individual. > OHS instance that carries all the license.

by _dain_ 4y ago

viraptor 4y ago

> So the recognition part is solved If you're using an averaged American voice - maybe. But it's really not solved for everyone. Google assistant can't set the right timer for me 1/10 times. And that's before we get to heavy accent Scots and others.

bambax 4y ago
Obligatory reference: https://www.youtube.com/watch?v=NMS2VnDveP8 https://www.youtube.com/watch?v=NMS2VnDveP8
arethuza 4y ago
Even my "affected Edinburgh accent", as someone once described it, causes no end of trouble with voice recognition.

draugadrotten 4y ago

> And yeah that would take effort to learn to use it right, just like any other programming language; so be it. This would be a hobbyist thing. There are quite a few hobbyists working on local on-prem privacy focused voice assistans with conversation support. https://www.home-assistant.io/integrations/#voice https://www.home-assistant.io/integrations/#voice https://www.home-assistant.io/integrations/conversation/ https://www.home-assistant.io/integrations/conversation/ Have fun. It is a rabbit hole.

Shank 4y ago

> Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager. I personally don't consider this a fully-solved problem. The best transcription system I've used is OpenAI Whisper, and it doesn't work in realtime. Maybe it's fine on small amounts but it's still not perfect. You really need error to be driven down dramatically. Zoom auto-captions are a joke in terms of how badly they work for me, and Live Text (beta) on macOS is equally dreadful. YouTube auto-captions suck. All of these use industry-leading APIs. If I'm speaking a voice command and one single word is wrong, usually the whole thing fails. There's an entirely separate issue about things that are Proper Nouns that don't exist. For example, "Todoist" is often misunderstood by Siri. Thus, people started saying "Two doist (where doist rhymes with joist)" to fool it into understanding "Todoist". Media like anime with strange titles from other languages often flat out trolls these transcription systems. ("Hey Siri, remind me to watch Kimetsu no Yaiba tomorrow".)

Aramgutang 4y ago

That reminds me of the handwriting recognition approach [1] used in old Palm Pilot devices. Even though the shapes it expected you to draw resembled the corresponding letters, you would never draw them like that if you were writing on paper. You knew that you were drawing something designed for a computer to recognise as unambiguously as possible, while being efficient to draw quickly and easy to learn for you. I feel like that's the kind of notion that voice interfaces should somehow expand upon. [1] https://en.wikipedia.org/wiki/Graffiti_(Palm_OS) https://en.wikipedia.org/wiki/Graffiti_(Palm_OS)

simiones 4y ago

> Voice recognition is basically a commodity now .. there are open source AI engines that can do it offline really well. So the recognition part is solved, you can just grab it from your distro's package manager. This is potentially far from true, depending on how exactly you draw the line between "voice recognition" and "language". I've looked at quite a few transcription services, and they fail a lot of the time for most people - those who either have a non-native accent (even if very slight!) or those who do any amount of stammering or other vocal tics.

ghaff 4y ago
I find the ML transcription services, given 2 people speaking English with high quality sound and without heavy accents/a lot of jargon, to be adequate for having a skimmable record--such as for extracting quotations (and just go back to the recording to confirm the exact words if it's not obvious). But if I'm publishing a transcript I get a human transcription. Cleaning up the ML stuff takes way too much time and I wouldn't publish a transcript without cleaning it up.
- simiones 4y ago
  I was in fact looking at some transcriptions of my recent meetings, and found one that captures how even small mistakes can make for completely not-understandable transcripts, unless they are manually cleaned up. Manual transcription: > So no: long story short, Slum is basically the way we can have an individual [, uhhh,] instance that carries all the licenses. (Slum is a project name in this case) Computer transcription (MS Teams): > So no. > A long story shorts. Love is basically the way we can have an individual. > OHS instance that carries all the license.