7 ms·

> And what do you feel when we make these parlor tricks more capable than us at the majority of tasks? This seems like the logical fallacy of "begging the ques

> And what do you feel when we make these parlor tricks more capable than us at the majority of tasks? This seems like the logical fallacy of "begging the question" since it is far from apparent to me that they are "more capable than us at the majority of tasks."

adastra22 3y ago
AI systems are vastly better than humans at a wide variety of tasks. Better at handwriting recognition, better at scheduling, better at playing games, better at speech recognition and transcription, etc.
- cypress66 3y ago
  I am skeptical on many of those. Speech recognition is not even close to human level. Whisper, and whatever Google uses will make a lot of mistakes on audio files that are trivial to any native speaker.
  - adastra22 3y ago
    In actual tests it is beyond human level. Humans actually mishear about 1 in 20 words during transcription tests; whisper does better.
    - emodendroket 3y ago
      But we don’t solely rely on how well we hear since we have knowledge that allows us to correct for poor hearing based on what is being said rather than forging ahead with a nonsense transcription. Machine transcription is definitely faster and cheaper but the end product isn’t “better,” and anyone who has read it can attest to that.
      - ben_w 3y ago
        > But we don’t solely rely on how well we hear since we have knowledge that allows us to correct for poor hearing based on what is being said rather than forging ahead with a nonsense transcription. Good voice transcription AI already do that too; that's why they work best if they know which language they're operating in, as that means they can use the language to create a model of the most likely words. I think the most recent WWDC from Apple even has a video about adding custom vocabulary for their speech engine to pick up on that covered some details in this exact topic, though I can't search right now.
        emodendroket 3y ago
        Undoubtedly so but I have yet to see one that doesn't make mistakes a human would be unlikely to. It is not an easy capability to reproduce and wouldn't have been my first choice if I wanted to talk about things it can do better than people.
        ben_w 3y ago
        > Undoubtedly so but I have yet to see one that doesn't make mistakes a human would be unlikely to. Absolutely, AI is very rarely human in its failure modes, and often has novel and exciting failure modes instead. But, on average… or so the marketing claims… it makes fewer mistakes. For a while, it was possible to improve upon super-human chess AI by pairing them with a human; the combination was called a centaur. Eventually the AI got too good even for that as they stopped making the sorts of mistakes humans could spot, but in the meantime, even though they were superhuman, they had failure modes that we could help out with.
        emodendroket 3y ago
        Assuming the intended audience is also humans, "exciting" errors seem worse in this instance, so I find it hard to credit these marketing claims.
        ben_w 3y ago
        Seem worse, sure. If it is or not, depends on the specific use of the transcription. Consider "I went to Lenny's" being transcribed wrong by a human as "I went to Denny's" or by an AI as "Ivan to Lenny's". Both are wrong, but if you get a human to check, we can be oblivious to the human mistake for the same reason it was made in the first place plus the effect where seeing text alters our perception of what we hear; the AI error being inhuman means we can spot it when the human error is imperceptible.
    - cypress66 3y ago
      Well, those "actual tests" clearly don't reflect reality. This is obvious if you actually use whisper.
- oldandtired 3y ago
  The question to ask is why? The answer is that we have programmed these systems to do what we require. They cannot exceed but they fail becasue of errors that we have placed in these systems. All of the tasks that you have mentioned have been programmed that way. It has taken human ingenuity to work out how to do this programming. The end result is a machine (non-sentient, non-intelligent) that is doing what we require. If you look at game playing, a system was created to play Go and won and yet that same system fails to win against humans under many circumstances. The literature is there, yet not publicised for all the world to see. A result of keeping the hype in play. If you look at speech recognition, these systems still fail when we humans work against them and yet, we humans still recognise what the machines fail at. Just keep in mind that a tractor can move a greater amount of material than a human can, but it is still only a tool. A plane can travel faster and fly higher that a human can, but it is still only a tool. We use these systems to augment our abilities and yet they are all limited in so many ways that we are not. The upshot is that we can do amazing things with the things we create, but none of those things exist without us and all those things fail without us.
  - ben_w 3y ago
    > All of the tasks that you have mentioned have been programmed that way. It has taken human ingenuity to work out how to do this programming. The successful Go AI were programmed to learn; we still can't program a decent Go AI with rules humans come up with. > The literature is there Do you have a link? Two Minute Papers just had a video about an AI systematic finding ways to confound other AI, but I thought we'd passed the point where the best Go AI could be so manipulated by humans…
    - oldandtired 3y ago
      Your example of the Go AI being programmed to learn is not all that accurate for what has been achieved here. I didn't keep the link for the discussion on the confounding of the Go AI system. What the discussion covered though was that there were simple Go configurations that the GO AI failed abysmally on when playing a human - it didn't learn here. I have spent forty years dealing with all sorts of computer systems - designing, building, maintaining, repairing, redesigning and rebuilding. One thing I have learnt over that time is that none of the systems ever built has been error free in terms of the logic entailed within them. All to often, I have seen systems that were used to make decisions with and those using them assuming that the outputs were correct or reasonable. Yet on investigation, the logic entailed in them was completely rubbish. We make assumptions and often we do not carefully check that those assumptions are actually real. I don't trust anything I write until I have gone over it with a fine tooth comb and then I will try to document all my assumptions and this usually shows up various logic errors or conditions that I didn't think about. I don't see this happening much out in the real world.
ben_w 3y ago
It's certainly difficult to enumerate all the things we humans actually do. There's a lot of stuff we consider to be "common sense", sometimes those things are used to criticise AI and sometimes they're used to criticise other humans for not knowing them, but that is a category that we don't even think about until we notice the absence. For the things not considered common sense, like playing chess (beats all humans) or speaking/reading foreign languages (more than I can name to a higher standard than my second language), to creating art (even if it regularly makes the common sense mistake of getting the number of fingers and limbs wrong it's still better and not just faster than most humans), to arithmetic (a Raspberry Pi Zero can do it faster than all humans combined), to symbolic maths, to flying planes… A dev conference I was at recently had someone demonstrate how they hooked up their whatsapp voice calls to speech recognition, speech synthesis trained on their own voice, and an LLM, and the criticism of the people who got the AI replies was not "you're using an AI" (he had to actively demonstrate his use of AI to conversation partners who didn't believe him) but "you can't have listed to my message, you replied too quickly to have even played it all back."
- oldandtired 3y ago
  It is impossible to enumerate all the things that we humans do. However, we can enumerate all the things that we create can do. Every system we create has its limitation due to the limitations that we create in them. All systems we create cannot exceed those limitations. We make machines that are stronger, faster, and can have much finer motor control than we have as individual abilities. No machine we have created has the dexterity that we have. Every computational system can be analysed in fine detail to determine the limits that we have built into them. It may take an enormous amount of time and effort to do so, but we can do it. No computational system that we have built is able to exceed the limited programming we place in it. There is an enormous amount of hype that goes on about the current generation (and future generations) of these systems, but all of them are in the abilities that we have programmed into them. They are in all essentials completely stupid (in the worst possible way - non-sentient, non-intelligent). Every logic error that we have made in building these systems is hidden in that code. One day, those errors will come back and bite us, but there is nothing intelligent or sentient in these systems. It is our errors, for which we are responsible, that will cause those problems. We can use them as adjuncts to our sentience and intelligence - but all they are are tools, never anything more. However, if we cede control to these systems, we are ceding control to something that is no better than fire (a good servant - a horrendous master). After forty years, I have seen far too often, hype by humans convince other humans to cede control to the systems that humans have made and the result has been various levels of chaos. If anything, what we need to be careful of is how humans use these systems against other humans. This is the perennial problem that we face as we build new technology.

adastra22 3y ago

AI systems are vastly better than humans at a wide variety of tasks. Better at handwriting recognition, better at scheduling, better at playing games, better at speech recognition and transcription, etc.

cypress66 3y ago
I am skeptical on many of those. Speech recognition is not even close to human level. Whisper, and whatever Google uses will make a lot of mistakes on audio files that are trivial to any native speaker.
- adastra22 3y ago
  In actual tests it is beyond human level. Humans actually mishear about 1 in 20 words during transcription tests; whisper does better.
  - emodendroket 3y ago
    But we don’t solely rely on how well we hear since we have knowledge that allows us to correct for poor hearing based on what is being said rather than forging ahead with a nonsense transcription. Machine transcription is definitely faster and cheaper but the end product isn’t “better,” and anyone who has read it can attest to that.
    - ben_w 3y ago
      > But we don’t solely rely on how well we hear since we have knowledge that allows us to correct for poor hearing based on what is being said rather than forging ahead with a nonsense transcription. Good voice transcription AI already do that too; that's why they work best if they know which language they're operating in, as that means they can use the language to create a model of the most likely words. I think the most recent WWDC from Apple even has a video about adding custom vocabulary for their speech engine to pick up on that covered some details in this exact topic, though I can't search right now.
      - emodendroket 3y ago
        Undoubtedly so but I have yet to see one that doesn't make mistakes a human would be unlikely to. It is not an easy capability to reproduce and wouldn't have been my first choice if I wanted to talk about things it can do better than people.
        ben_w 3y ago
        > Undoubtedly so but I have yet to see one that doesn't make mistakes a human would be unlikely to. Absolutely, AI is very rarely human in its failure modes, and often has novel and exciting failure modes instead. But, on average… or so the marketing claims… it makes fewer mistakes. For a while, it was possible to improve upon super-human chess AI by pairing them with a human; the combination was called a centaur. Eventually the AI got too good even for that as they stopped making the sorts of mistakes humans could spot, but in the meantime, even though they were superhuman, they had failure modes that we could help out with.
        emodendroket 3y ago
        Assuming the intended audience is also humans, "exciting" errors seem worse in this instance, so I find it hard to credit these marketing claims.
        ben_w 3y ago
        Seem worse, sure. If it is or not, depends on the specific use of the transcription. Consider "I went to Lenny's" being transcribed wrong by a human as "I went to Denny's" or by an AI as "Ivan to Lenny's". Both are wrong, but if you get a human to check, we can be oblivious to the human mistake for the same reason it was made in the first place plus the effect where seeing text alters our perception of what we hear; the AI error being inhuman means we can spot it when the human error is imperceptible.
  - cypress66 3y ago
    Well, those "actual tests" clearly don't reflect reality. This is obvious if you actually use whisper.
oldandtired 3y ago
The question to ask is why? The answer is that we have programmed these systems to do what we require. They cannot exceed but they fail becasue of errors that we have placed in these systems. All of the tasks that you have mentioned have been programmed that way. It has taken human ingenuity to work out how to do this programming. The end result is a machine (non-sentient, non-intelligent) that is doing what we require. If you look at game playing, a system was created to play Go and won and yet that same system fails to win against humans under many circumstances. The literature is there, yet not publicised for all the world to see. A result of keeping the hype in play. If you look at speech recognition, these systems still fail when we humans work against them and yet, we humans still recognise what the machines fail at. Just keep in mind that a tractor can move a greater amount of material than a human can, but it is still only a tool. A plane can travel faster and fly higher that a human can, but it is still only a tool. We use these systems to augment our abilities and yet they are all limited in so many ways that we are not. The upshot is that we can do amazing things with the things we create, but none of those things exist without us and all those things fail without us.
- ben_w 3y ago
  > All of the tasks that you have mentioned have been programmed that way. It has taken human ingenuity to work out how to do this programming. The successful Go AI were programmed to learn; we still can't program a decent Go AI with rules humans come up with. > The literature is there Do you have a link? Two Minute Papers just had a video about an AI systematic finding ways to confound other AI, but I thought we'd passed the point where the best Go AI could be so manipulated by humans…
  - oldandtired 3y ago
    Your example of the Go AI being programmed to learn is not all that accurate for what has been achieved here. I didn't keep the link for the discussion on the confounding of the Go AI system. What the discussion covered though was that there were simple Go configurations that the GO AI failed abysmally on when playing a human - it didn't learn here. I have spent forty years dealing with all sorts of computer systems - designing, building, maintaining, repairing, redesigning and rebuilding. One thing I have learnt over that time is that none of the systems ever built has been error free in terms of the logic entailed within them. All to often, I have seen systems that were used to make decisions with and those using them assuming that the outputs were correct or reasonable. Yet on investigation, the logic entailed in them was completely rubbish. We make assumptions and often we do not carefully check that those assumptions are actually real. I don't trust anything I write until I have gone over it with a fine tooth comb and then I will try to document all my assumptions and this usually shows up various logic errors or conditions that I didn't think about. I don't see this happening much out in the real world.

ben_w 3y ago

It's certainly difficult to enumerate all the things we humans actually do. There's a lot of stuff we consider to be "common sense", sometimes those things are used to criticise AI and sometimes they're used to criticise other humans for not knowing them, but that is a category that we don't even think about until we notice the absence. For the things not considered common sense, like playing chess (beats all humans) or speaking/reading foreign languages (more than I can name to a higher standard than my second language), to creating art (even if it regularly makes the common sense mistake of getting the number of fingers and limbs wrong it's still better and not just faster than most humans), to arithmetic (a Raspberry Pi Zero can do it faster than all humans combined), to symbolic maths, to flying planes… A dev conference I was at recently had someone demonstrate how they hooked up their whatsapp voice calls to speech recognition, speech synthesis trained on their own voice, and an LLM, and the criticism of the people who got the AI replies was not "you're using an AI" (he had to actively demonstrate his use of AI to conversation partners who didn't believe him) but "you can't have listed to my message, you replied too quickly to have even played it all back."

oldandtired 3y ago
It is impossible to enumerate all the things that we humans do. However, we can enumerate all the things that we create can do. Every system we create has its limitation due to the limitations that we create in them. All systems we create cannot exceed those limitations. We make machines that are stronger, faster, and can have much finer motor control than we have as individual abilities. No machine we have created has the dexterity that we have. Every computational system can be analysed in fine detail to determine the limits that we have built into them. It may take an enormous amount of time and effort to do so, but we can do it. No computational system that we have built is able to exceed the limited programming we place in it. There is an enormous amount of hype that goes on about the current generation (and future generations) of these systems, but all of them are in the abilities that we have programmed into them. They are in all essentials completely stupid (in the worst possible way - non-sentient, non-intelligent). Every logic error that we have made in building these systems is hidden in that code. One day, those errors will come back and bite us, but there is nothing intelligent or sentient in these systems. It is our errors, for which we are responsible, that will cause those problems. We can use them as adjuncts to our sentience and intelligence - but all they are are tools, never anything more. However, if we cede control to these systems, we are ceding control to something that is no better than fire (a good servant - a horrendous master). After forty years, I have seen far too often, hype by humans convince other humans to cede control to the systems that humans have made and the result has been various levels of chaos. If anything, what we need to be careful of is how humans use these systems against other humans. This is the perennial problem that we face as we build new technology.