6 ms·

I have a tangential question regarding this post. I have notice more recently at I am asked to identify object(signs, roads etc.)using multiple iterations. It'

I have a tangential question regarding this post. I have notice more recently at I am asked to identify object(signs, roads etc.)using multiple iterations. It's not unusual to be asked to identify these on 3 separate iterations. This is despite identifying all cells correctly on both the first and second passes. I don't believe it was always this way. What is the reason for this? Is there some heuristic in the captcha program that decides to ask for further identification or is this randomly generated?

ageitgey 9y ago
Google is literally using you as unpaid labor to label/validate their image datasets. My guess is that either (1) your first attempt at validation didn't match what their automated system identified so it didn't have high enough confidence to proceed or (2) the first image is one you are classifying from scratch but the second/third image is the one that you are only validating against their guess. In either case, it's amazing to me that Google is able to get the whole world to do unpaid menial work for them by offering a free product to website owners.
- ranie93 9y ago
  >In either case, it's amazing to me that Google is able to get the whole world to do unpaid menial work for them by offering a free product to website owners. Its an example of the https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble....
- granitosaurus 9y ago
  > Google is literally using you as unpaid labor to label/validate their image datasets. Can't believe people are not going up in arms about it. This malware makes public access points unusable.
- sebleon 9y ago
  Ha, I believe Duolingo is doing the same - getting their users to translate huge bodies of text while learning new languages
  - lukeinator42 9y ago
    The creator of Duolingo was also the creator of ReCaptcha I think. He has a really interesting ted talk about it here: https://www.ted.com/talks/luis_von_ahn_massive_scale_online_collaboration https://www.ted.com/talks/luis_von_ahn_massive_scale_online_...
- Nition 9y ago
  The "identify the grid squares which contain a sign" type CAPTCHA isn't clear about whether you should be selecting all grid squares that contain mostly sign, or all grid squares that contain ANY of the sign. I imagine that particular one causes a fair discrepancy in what people select. In fact I want to know what other people choose to do for those one. I'd like to guess that Hacker News types would disproportionately choose the latter option.
  - manojlds 9y ago
    Yeah I do any of the sign.
  - bogomipz 9y ago
    Interesting, my assumption has been that it must reside mostly or entirely in the square. Now I'm wondering are the instruction intentionally vague?
  - wruza 9y ago
    I think they shift selection grid from user to user, so they get a statistical outcome with a great precision of where the sign really is. If some data points stand out, they should be simply ignored.
  - harshreality 9y ago
    A million times this. - Do the posts or supports of the signs count as part of the signs? - What if it's a sign but not a street sign? - If the camera is looking across a road, but the pavement isn't visible, does that image contain a road? I suspect humans are answering that differently, but it's not as bad as the sign one. - Storefronts. Sometimes I can't tell what I'm looking at. There's a building with lettering on it, but I can't tell whether it's a store much less a storefront. Google needs to add a tutorial for humans on how to answer those captchas, because there's not enough context for us to figure out what google really wants. If they'd dogfood this and require their employees to solve a captcha to login to their workstations, this problem would have been solved yesterday.
- thwarted 9y ago
  it's amazing to me that Google is able to get the whole world to do unpaid menial work for them by offering a free product to website owners. I'm surprised, since google bought youtube in 2006, that google doesn't let people add translation subtitles to videos on youtube, which is something people have historically been actively willing to do for free, compare multiple submissions against each other to gauge accuracy, then map that to the audio of the video to train and machine based language translation datasets. Always seemed like a killer way to leverage what people would do for free to advance the state of the art.
chatmasta 9y ago
Historically the most reliable way to break a CAPTCHA was to pay a human to solve it via a sketchy service, for eg $3/1000. The round trip time of those services can be quite long, narrowly within the CAPTCHA timeout window. By forcing some users to solve three iterations of the puzzle (the exact instructions are “stop when there are no more images containing <cars>”), Google forces multiple round trips to a CAPTCHA solving service. Also, the solving service and its client need to account for the expectation that the final image will have no <cars>, increasing complexity. This increases cost 3x, slows solving rate 3x, etc. Ultimately google can never stop the problem of outsourcing CAPTCHA solving to a human, so their best option is to increase the cost and complexity of doing so.
- CamelCaseName 9y ago
  This comes at the expense of normal users who have to solve three iterations of the puzzle. One co-op term I wrote a simple VBA script to open up Google and search for some addresses. It was a small 2-3 day research project. Well unfortunately for me, I subsequently had to solve captchas every day for the next month and a half, and although about half of them would be instantly solved, when I had to redo CAPTCHAs I could feel my sanity slipping away.
  - deleted 9y ago
    [deleted]
  - visarga 9y ago
    Google is stingy with search results, even though its index exploits the content we all create. They run millions of automated requests on my server - in fact Google is using more CPU than my regular visitors, but I can't run on their servers even a few thousand automated queries. They don't allow even mild automated searching, the price per thousand searches is prohibitive and you can't get past the 1000'th result so you need to use combinations of words and other imperfect means to go deeper. I would have liked to do deep web searches in order to build datasets for ML, but unfortunately there's no search engine (or is too expensive) for bots. It's 2017, years after the Snowden leaks, and we still use Google and don't have a f*#ng search engine for creators, we only have search engine for 'customers'. We need our own search engine for many things - privacy, control over filtering, mass search, deep search, for creating novel projects, to be sure Russians haven't tampered with it, and so on. I'd say Google's refusal to accept bots is akin to a lack of net neutrality and places limits on what kind search based open source projects can be created - I know it sounds entitled, and maybe I should just build my own crawler and index - but you don't solve a more complex problem as a preamble to solving a simpler problem and we don't want thousands or millions of private search engines pulling the same content and duplicating indexes - we still need a community solution. If they love net neutrality, they should support search neutrality as well, for the maker community. It's the same argument as for net neutrality: to protect innovation from incumbents. Search is like the last mile and Google is like a nasty pipe throttling ISP.
mmcwilliams 9y ago
ReCAPTCHA has a "Security Preferences" parameter where you can set the level of difficulty of the test. If you set it to "Most Secure" most or all users are presented with the multiple image identification scenario you describe. Easiest will simply present the "I am not a robot" checkbox modal which does some browser fingerprinting and other tests in the background. I believe there is also a heuristic that determines the difficulty of the test to present users if you leave the setting on the default medium difficulty. Could be mistaken, but this is my takeaway from implementing reCAPTCHA a few times.
- bogomipz 9y ago
  Oh this is interesting. I am almost certain this is what I am seeing. The funny thing is that the times I have gotten the most secure setting is for something like buying an ebook or RSVP'ing for an event. Nothing in the least bit sensitive. So I'm guessing people think they are practicing good security when all they are doing is making for a terrible user experience.

ageitgey 9y ago

Google is literally using you as unpaid labor to label/validate their image datasets. My guess is that either (1) your first attempt at validation didn't match what their automated system identified so it didn't have high enough confidence to proceed or (2) the first image is one you are classifying from scratch but the second/third image is the one that you are only validating against their guess. In either case, it's amazing to me that Google is able to get the whole world to do unpaid menial work for them by offering a free product to website owners.

ranie93 9y ago
>In either case, it's amazing to me that Google is able to get the whole world to do unpaid menial work for them by offering a free product to website owners. Its an example of the https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble....
granitosaurus 9y ago
> Google is literally using you as unpaid labor to label/validate their image datasets. Can't believe people are not going up in arms about it. This malware makes public access points unusable.
sebleon 9y ago
Ha, I believe Duolingo is doing the same - getting their users to translate huge bodies of text while learning new languages
- lukeinator42 9y ago
  The creator of Duolingo was also the creator of ReCaptcha I think. He has a really interesting ted talk about it here: https://www.ted.com/talks/luis_von_ahn_massive_scale_online_collaboration https://www.ted.com/talks/luis_von_ahn_massive_scale_online_...
Nition 9y ago
The "identify the grid squares which contain a sign" type CAPTCHA isn't clear about whether you should be selecting all grid squares that contain mostly sign, or all grid squares that contain ANY of the sign. I imagine that particular one causes a fair discrepancy in what people select. In fact I want to know what other people choose to do for those one. I'd like to guess that Hacker News types would disproportionately choose the latter option.
- manojlds 9y ago
  Yeah I do any of the sign.
- bogomipz 9y ago
  Interesting, my assumption has been that it must reside mostly or entirely in the square. Now I'm wondering are the instruction intentionally vague?
- wruza 9y ago
  I think they shift selection grid from user to user, so they get a statistical outcome with a great precision of where the sign really is. If some data points stand out, they should be simply ignored.
- harshreality 9y ago
  A million times this. - Do the posts or supports of the signs count as part of the signs? - What if it's a sign but not a street sign? - If the camera is looking across a road, but the pavement isn't visible, does that image contain a road? I suspect humans are answering that differently, but it's not as bad as the sign one. - Storefronts. Sometimes I can't tell what I'm looking at. There's a building with lettering on it, but I can't tell whether it's a store much less a storefront. Google needs to add a tutorial for humans on how to answer those captchas, because there's not enough context for us to figure out what google really wants. If they'd dogfood this and require their employees to solve a captcha to login to their workstations, this problem would have been solved yesterday.
thwarted 9y ago
it's amazing to me that Google is able to get the whole world to do unpaid menial work for them by offering a free product to website owners. I'm surprised, since google bought youtube in 2006, that google doesn't let people add translation subtitles to videos on youtube, which is something people have historically been actively willing to do for free, compare multiple submissions against each other to gauge accuracy, then map that to the audio of the video to train and machine based language translation datasets. Always seemed like a killer way to leverage what people would do for free to advance the state of the art.

chatmasta 9y ago

Historically the most reliable way to break a CAPTCHA was to pay a human to solve it via a sketchy service, for eg $3/1000. The round trip time of those services can be quite long, narrowly within the CAPTCHA timeout window. By forcing some users to solve three iterations of the puzzle (the exact instructions are “stop when there are no more images containing <cars>”), Google forces multiple round trips to a CAPTCHA solving service. Also, the solving service and its client need to account for the expectation that the final image will have no <cars>, increasing complexity. This increases cost 3x, slows solving rate 3x, etc. Ultimately google can never stop the problem of outsourcing CAPTCHA solving to a human, so their best option is to increase the cost and complexity of doing so.

CamelCaseName 9y ago
This comes at the expense of normal users who have to solve three iterations of the puzzle. One co-op term I wrote a simple VBA script to open up Google and search for some addresses. It was a small 2-3 day research project. Well unfortunately for me, I subsequently had to solve captchas every day for the next month and a half, and although about half of them would be instantly solved, when I had to redo CAPTCHAs I could feel my sanity slipping away.
- deleted 9y ago
  [deleted]
- visarga 9y ago
  Google is stingy with search results, even though its index exploits the content we all create. They run millions of automated requests on my server - in fact Google is using more CPU than my regular visitors, but I can't run on their servers even a few thousand automated queries. They don't allow even mild automated searching, the price per thousand searches is prohibitive and you can't get past the 1000'th result so you need to use combinations of words and other imperfect means to go deeper. I would have liked to do deep web searches in order to build datasets for ML, but unfortunately there's no search engine (or is too expensive) for bots. It's 2017, years after the Snowden leaks, and we still use Google and don't have a f*#ng search engine for creators, we only have search engine for 'customers'. We need our own search engine for many things - privacy, control over filtering, mass search, deep search, for creating novel projects, to be sure Russians haven't tampered with it, and so on. I'd say Google's refusal to accept bots is akin to a lack of net neutrality and places limits on what kind search based open source projects can be created - I know it sounds entitled, and maybe I should just build my own crawler and index - but you don't solve a more complex problem as a preamble to solving a simpler problem and we don't want thousands or millions of private search engines pulling the same content and duplicating indexes - we still need a community solution. If they love net neutrality, they should support search neutrality as well, for the maker community. It's the same argument as for net neutrality: to protect innovation from incumbents. Search is like the last mile and Google is like a nasty pipe throttling ISP.

mmcwilliams 9y ago

ReCAPTCHA has a "Security Preferences" parameter where you can set the level of difficulty of the test. If you set it to "Most Secure" most or all users are presented with the multiple image identification scenario you describe. Easiest will simply present the "I am not a robot" checkbox modal which does some browser fingerprinting and other tests in the background. I believe there is also a heuristic that determines the difficulty of the test to present users if you leave the setting on the default medium difficulty. Could be mistaken, but this is my takeaway from implementing reCAPTCHA a few times.

bogomipz 9y ago
Oh this is interesting. I am almost certain this is what I am seeing. The funny thing is that the times I have gotten the most secure setting is for something like buying an ebook or RSVP'ing for an event. Nothing in the least bit sensitive. So I'm guessing people think they are practicing good security when all they are doing is making for a terrible user experience.