5 ms·

Also submitted yesterday, by someone else, but it got buried: http://news.ycombinator.com/item?id=4775644 http://news.ycombinator.com/item?id=4775644 I resubmi

Also submitted yesterday, by someone else, but it got buried: http://news.ycombinator.com/item?id=4775644 http://news.ycombinator.com/item?id=4775644 I resubmitted since I'm pretty sure this is of interest to many HN readers. Examples of why it's interesting include: 1. Google's deep learning work is now being used to power Android voice search (http://googleresearch.blogspot.ca/2012/08/speech-recognition-and-deep-learning.html http://googleresearch.blogspot.ca/2012/08/speech-recognition... ); and 2. Dean claims that "We are seeing better than human-level performance in some visual tasks," in particular, for the problem of extracting house numbers in photos taken by Google's Street View cars, a job that used to be done by a large team of people (http://www.technologyreview.com/news/429442/google-puts-its-virtual-brain-technology-to-work/ http://www.technologyreview.com/news/429442/google-puts-its-... ).

dumitrue 14y ago
It's not obvious to me what it means to have "better than human-level performance" since most of the time the ground-truth itself is defined by humans :)
- dsl 14y ago
  One example I can think of is a computer could read a house number or a street sign from 100 feet away, where a human with good vision might be able to make out the same text at 20 feet.
- jasonwatkinspdx 14y ago
  In production you may use a process that's more economized than what you'd use to establish a ground truth in research. So the human team in production will have some error rate that is tolerated or won't include as much redundancy. In this sense it's easy to imagine better than human performance from the software, in the sense that the human performance you get isn't a single max value, but rather a function of budget.
- robrenaud 14y ago
  You could also ask many humans and average their response to obtain a golden label, and see how well any particular human agrees with the average. If there is a lot of variance in the human answers, then it's possible for a machine to have better than (individual) human performance, even on a human labelled data set.
- fudged71 14y ago
  CATCHAS, ironically, often seem to require "better than human-level performance" for recognition
ArbitraryLimits 14y ago
> "We are seeing better than human-level performance in some visual tasks," in particular, for the problem of extracting house numbers in photos taken by Google's Street View cars Can someone explain to me how this is news, given that the handwritten addresses on snail mail envelopes in the US have been OCR'd by neural networks for more than twenty years now?
- confluence 14y ago
  The order of complexity difference between recognising figures on plain paper perpendicular to a scanning device in a controlled environment, and doing the same thing on huge amounts of non-standard chaotic data is why.
  - ArbitraryLimits 14y ago
    Virtually all house numbers are either painted from a stencil or composed of mass-produced shapes on a background of uniform color, whereas addresses on envelopes are handwritten by doctors, six-year-olds and people with Parkinson's disease. I'm not convinced it's a harder problem.
    - confluence 14y ago
      What don't you get? One is on a white background. One is in random orientations, placed in complex scenes, with random fonts, positions, numbers, sizes, shapes and locations and you don't even know where they are. It's like a game of "Where's Waldo" on freaking crack. You have literally no idea how complex this stuff is now then do you?
      - ArbitraryLimits 14y ago
        Compare the addresses at http://www.realsimple.com/home-organizing/decorating/eye-catching-house-numbers-10000001218771/index.html http://www.realsimple.com/home-organizing/decorating/eye-cat... and http://mandydouglass.blogspot.com/2010/10/addressing-envelopes.html http://mandydouglass.blogspot.com/2010/10/addressing-envelop... . Those are two representative images I picked from the first google hits for "house number" and "handwritten address" respectively - all the others were comparable. Are you seriously going to claim that the house number is harder to recognize than the handwriting? I do have an idea (literally, even) that there are additional problems having to do with extracting the house number images themselves from full-motion video, but that's an image registration problem and not an object recognition problem.

dumitrue 14y ago

It's not obvious to me what it means to have "better than human-level performance" since most of the time the ground-truth itself is defined by humans :)

dsl 14y ago
One example I can think of is a computer could read a house number or a street sign from 100 feet away, where a human with good vision might be able to make out the same text at 20 feet.
jasonwatkinspdx 14y ago
In production you may use a process that's more economized than what you'd use to establish a ground truth in research. So the human team in production will have some error rate that is tolerated or won't include as much redundancy. In this sense it's easy to imagine better than human performance from the software, in the sense that the human performance you get isn't a single max value, but rather a function of budget.
robrenaud 14y ago
You could also ask many humans and average their response to obtain a golden label, and see how well any particular human agrees with the average. If there is a lot of variance in the human answers, then it's possible for a machine to have better than (individual) human performance, even on a human labelled data set.
fudged71 14y ago
CATCHAS, ironically, often seem to require "better than human-level performance" for recognition

ArbitraryLimits 14y ago

> "We are seeing better than human-level performance in some visual tasks," in particular, for the problem of extracting house numbers in photos taken by Google's Street View cars Can someone explain to me how this is news, given that the handwritten addresses on snail mail envelopes in the US have been OCR'd by neural networks for more than twenty years now?

confluence 14y ago
The order of complexity difference between recognising figures on plain paper perpendicular to a scanning device in a controlled environment, and doing the same thing on huge amounts of non-standard chaotic data is why.
- ArbitraryLimits 14y ago
  Virtually all house numbers are either painted from a stencil or composed of mass-produced shapes on a background of uniform color, whereas addresses on envelopes are handwritten by doctors, six-year-olds and people with Parkinson's disease. I'm not convinced it's a harder problem.
  - confluence 14y ago
    What don't you get? One is on a white background. One is in random orientations, placed in complex scenes, with random fonts, positions, numbers, sizes, shapes and locations and you don't even know where they are. It's like a game of "Where's Waldo" on freaking crack. You have literally no idea how complex this stuff is now then do you?
    - ArbitraryLimits 14y ago
      Compare the addresses at http://www.realsimple.com/home-organizing/decorating/eye-catching-house-numbers-10000001218771/index.html http://www.realsimple.com/home-organizing/decorating/eye-cat... and http://mandydouglass.blogspot.com/2010/10/addressing-envelopes.html http://mandydouglass.blogspot.com/2010/10/addressing-envelop... . Those are two representative images I picked from the first google hits for "house number" and "handwritten address" respectively - all the others were comparable. Are you seriously going to claim that the house number is harder to recognize than the handwriting? I do have an idea (literally, even) that there are additional problems having to do with extracting the house number images themselves from full-motion video, but that's an image registration problem and not an object recognition problem.