4 ms·

Of course it's not solved. We don't even know how to define the problem. Speech is[1] a fundamental component of Language, which is a fundamental component of

Of course it's not solved. We don't even know how to define the problem. Speech is[1] a fundamental component of Language, which is a fundamental component of Intelligence[2]. This is addressed somewhat in the conversation around semantic error rate; that there is more to processing raw audio speech than the calculus of mapping signals to tokens; some understanding of semantics and context is required to differentiate between otherwise indistinguishable surface forms. I find it doubtful that there's a clean interface that separates the 'intelligent' parts of the brain from the 'language' parts of the brain from the 'speech' parts of the brain. This leakiness (or richness, really) means that you can't neatly solve any one part of this chain to the level of competence that the brain solves it. That means to 'solve' speech recognition, you have to 'solve' language, and thus 'solve' general intelligence. And to 'solve' general intelligence, you have to understand it, in a theoretical sense, which we don't. Indeed, it will likely involve solving all the other modalities of sensation as well. It's definitely the case that you need to have a model for prosody to understand speech. It is entirely possible that vision is a large factor as well, in the form of body language, lip reading, eye contact, and so on. Speech recognition is quite good for what it is. For many practical applications, especially to do with young, white, newspeaker-accented English speakers who sound the most like the people who develop it, and the data sets used to develop it, it is good enough in the 80/20 sense. But it is nowhere near solved by even the least rigorous definition of the word. ---- [1] according to the philosophy I subscribe to, at any rate [2] according to the philosophy I subscribe to, at any rate

baddox 9y ago
Problem definition really is the issue. Even appealing to the Turing test is less than satisfying. I’m fairly sure I’m a human, but Siri and similar will likely outperform me in certain categories like place names and popular music.
- jdmichal 9y ago
  Don't forget that being too good can also cause you to fail the Turing test. I would not expect a human to be able to answer some questions that may be trivial to a computer. Things like, "What's the square-root of 137?" Or, "Identify this obscure song within 5 seconds of listening from a random starting point."
  - baddox 9y ago
    I suppose that depends on the precise setup of the test. Is the subject (if they're a human) allowed access to a calculator or a computer with an internet connection? Even if they were, timing would be an obvious tell, but an AI could easily be programmed (or could learn) to introduce an appropriate delay.
    - jdmichal 9y ago
      The idea behind the Turing test isn't whether they can arrive at the same answer given the tools. The idea is whether a human can tell the difference. I would expect any human to answer with something like, "I don't know." Or, "Let me find my calculator..." Either answer would be a lie for a computational AI -- it would know the answer and not require a calculator. This is, I think, one of the failings of the Turing test. It's easy enough for us to make new humans; making a machine that acts exactly like a human seems like a silly endeavor. I want a machine that can assist us and reinforce our failings. Which means that we can necessarily differentiate it from another human. I vastly prefer that over a machine that has learned to lie to us.
      - baddox 9y ago
        > The idea is whether a human can tell the difference. I know, that's why I'm saying the precise setup of the test is important. That's why, for instance, it's usually presented as a text messaging setup, because the goal is to test intelligence via language and conversation skills. A "face to face" setup wouldn't make much sense, unless we wanted to test that aspect of robotics.
- TheOtherHobbes 9y ago
  Problem definition is not the issue. Contextual awareness is the issue. Humans use different sub-languages to speak to friends, bosses, dates, lovers, spouses, teachers, students, and so on. Each social context has its own vocabulary, its own set of expected conversation starter statements, its own set of likely responses, its own set of problems that may need to be solved - and so on. A lot of what passes for intelligent interaction among humans is really just this social awareness. Machines will fail the Turing Test without it. But you can't teach a machine to mimic social awareness by throwing 100,000 hours of speech samples at it. Nor can you expect Echo/Siri/etc to know the context you're working in with no input from social cues - location, dress, time of day, social relationship, facial expression, etc. So practical ASR turns into spoken-command-line-plus-guessing. That turns out to be a pretty poor imitation of even the simplest social relationship. It's not a useless imitation. Even today's limited voice assistants do a fair job at providing a useful service. But the idea that you can drown neural networks in sample soup to train them, and build yourself a machine capable of intelligent-seeming conversation is just naive. That's not how baby humans learn to hear and speak, and it's certainly not going to teach machines to converse at a human level.

baddox 9y ago

Problem definition really is the issue. Even appealing to the Turing test is less than satisfying. I’m fairly sure I’m a human, but Siri and similar will likely outperform me in certain categories like place names and popular music.

jdmichal 9y ago
Don't forget that being too good can also cause you to fail the Turing test. I would not expect a human to be able to answer some questions that may be trivial to a computer. Things like, "What's the square-root of 137?" Or, "Identify this obscure song within 5 seconds of listening from a random starting point."
- baddox 9y ago
  I suppose that depends on the precise setup of the test. Is the subject (if they're a human) allowed access to a calculator or a computer with an internet connection? Even if they were, timing would be an obvious tell, but an AI could easily be programmed (or could learn) to introduce an appropriate delay.
  - jdmichal 9y ago
    The idea behind the Turing test isn't whether they can arrive at the same answer given the tools. The idea is whether a human can tell the difference. I would expect any human to answer with something like, "I don't know." Or, "Let me find my calculator..." Either answer would be a lie for a computational AI -- it would know the answer and not require a calculator. This is, I think, one of the failings of the Turing test. It's easy enough for us to make new humans; making a machine that acts exactly like a human seems like a silly endeavor. I want a machine that can assist us and reinforce our failings. Which means that we can necessarily differentiate it from another human. I vastly prefer that over a machine that has learned to lie to us.
    - baddox 9y ago
      > The idea is whether a human can tell the difference. I know, that's why I'm saying the precise setup of the test is important. That's why, for instance, it's usually presented as a text messaging setup, because the goal is to test intelligence via language and conversation skills. A "face to face" setup wouldn't make much sense, unless we wanted to test that aspect of robotics.
TheOtherHobbes 9y ago
Problem definition is not the issue. Contextual awareness is the issue. Humans use different sub-languages to speak to friends, bosses, dates, lovers, spouses, teachers, students, and so on. Each social context has its own vocabulary, its own set of expected conversation starter statements, its own set of likely responses, its own set of problems that may need to be solved - and so on. A lot of what passes for intelligent interaction among humans is really just this social awareness. Machines will fail the Turing Test without it. But you can't teach a machine to mimic social awareness by throwing 100,000 hours of speech samples at it. Nor can you expect Echo/Siri/etc to know the context you're working in with no input from social cues - location, dress, time of day, social relationship, facial expression, etc. So practical ASR turns into spoken-command-line-plus-guessing. That turns out to be a pretty poor imitation of even the simplest social relationship. It's not a useless imitation. Even today's limited voice assistants do a fair job at providing a useful service. But the idea that you can drown neural networks in sample soup to train them, and build yourself a machine capable of intelligent-seeming conversation is just naive. That's not how baby humans learn to hear and speak, and it's certainly not going to teach machines to converse at a human level.