5 ms·

Show HN: The Captcha Experiment

15 points by abrichr 15y ago

foxylad 15y ago
The recaptcha didn't have any instructions, and seemed to supply a lot of numbers and words. Were you supposed to give a particular one? I had little difficulty with the others, but the lack of instructions might bias the results against recaptcha.
- abrichr 15y ago
  We've simply implemented the recaptcha code as it was given. The goal is to evaluate how well it would do were it used in the wild, so we don't provide any extra instructions.
- spydum 15y ago
  to be fair, I think that may be part of what they are evaluating: existing captcha systems compared to some others. So the lack of instructions from recaptcha is a flaw in recaptcha, not the survey
blahedo 15y ago
The very first one I got was: Chair is to wall: like 'd00r' is to "mAn" and the An$wer is; bedroom. I was so flabbergasted I didn't even know where to start, and I had to move on just to see what else was in there---the rest of the captchas all seem pretty mundane. Edit: found it again, fixed it to be direct quote. I'm looking at it again now and I still have no idea what I'm supposed to do with that.
sray 15y ago
Without input validation, and without any clear motivation as top why this experiment was important, I was perfectly happy entering garbage or blank forms. What I'm getting at is that without any real motivation for submitting a form, I don't really care what I enter. This is different than when I enter a captcha "in the wild" - in that case I would try really hard because I care about successfully submitting the form. I'm not sure what this experiment is trying to prove, but without that motivation, I don't believe it's a proper simulation of real captcha use.
- abrichr 15y ago
  That's a good point. We'll have to think about how to address that.
  - astar 15y ago
    coughgamificationcough Just kidding. You could always do a drawing. Ask for a donated ipad. everyone who answers 100 or more catchpas correctly are eligible to be in the raffle.
spatten 15y ago
The one that said something about "type the red moving characters" never worked for me. The spinner just sat there for > 30 seconds before I gave up. (Chrome on OS X with Flash installed, if that makes a difference)
astar 15y ago
"How many years in the Hundred Years War" This was my favourite one. Easy to solve -- not at all trivial to parse mechanically.
- ebzlo 15y ago
  Really? I don't know how many people who would know that the correct answer is actually 116.
  - astar 15y ago
    heh. I really thought it was 100 (and so does the site because it said I was correct). maybe something like, "How many birthdays has a fourteen year old has?" Or: how many inches of sandwich do you get if you get a half-foot long?
bigiain 15y ago
cynical-me wonders who's signup process they're "crowdsourcing"...
- abrichr 15y ago
  Damn, that would've been a pretty good business model. As it is, we're doing nothing like that.
JangoSteve 15y ago
I got about 4 questions in and then it started giving me basic authentication boxes saying the site is password protected, and I couldn't do anything or click any links on the site without logging in or something.
- abrichr 15y ago
  Sorry about that; I'm a developer, not a system administrator. It's fixed now.
thekevan 15y ago
I did not get the audio one at all

15 points by abrichr 15y ago

The recaptcha didn't have any instructions, and seemed to supply a lot of numbers and words. Were you supposed to give a particular one? I had little difficulty with the others, but the lack of instructions might bias the results against recaptcha.

abrichr 15y ago
We've simply implemented the recaptcha code as it was given. The goal is to evaluate how well it would do were it used in the wild, so we don't provide any extra instructions.
spydum 15y ago
to be fair, I think that may be part of what they are evaluating: existing captcha systems compared to some others. So the lack of instructions from recaptcha is a flaw in recaptcha, not the survey

blahedo 15y ago

The very first one I got was: Chair is to wall: like 'd00r' is to "mAn" and the An$wer is; bedroom. I was so flabbergasted I didn't even know where to start, and I had to move on just to see what else was in there---the rest of the captchas all seem pretty mundane. Edit: found it again, fixed it to be direct quote. I'm looking at it again now and I still have no idea what I'm supposed to do with that.

sray 15y ago

Without input validation, and without any clear motivation as top why this experiment was important, I was perfectly happy entering garbage or blank forms. What I'm getting at is that without any real motivation for submitting a form, I don't really care what I enter. This is different than when I enter a captcha "in the wild" - in that case I would try really hard because I care about successfully submitting the form. I'm not sure what this experiment is trying to prove, but without that motivation, I don't believe it's a proper simulation of real captcha use.

abrichr 15y ago
That's a good point. We'll have to think about how to address that.
- astar 15y ago
  coughgamificationcough Just kidding. You could always do a drawing. Ask for a donated ipad. everyone who answers 100 or more catchpas correctly are eligible to be in the raffle.

spatten 15y ago

The one that said something about "type the red moving characters" never worked for me. The spinner just sat there for > 30 seconds before I gave up. (Chrome on OS X with Flash installed, if that makes a difference)

astar 15y ago

"How many years in the Hundred Years War" This was my favourite one. Easy to solve -- not at all trivial to parse mechanically.

ebzlo 15y ago
Really? I don't know how many people who would know that the correct answer is actually 116.
- astar 15y ago
  heh. I really thought it was 100 (and so does the site because it said I was correct). maybe something like, "How many birthdays has a fourteen year old has?" Or: how many inches of sandwich do you get if you get a half-foot long?

bigiain 15y ago

cynical-me wonders who's signup process they're "crowdsourcing"...

abrichr 15y ago
Damn, that would've been a pretty good business model. As it is, we're doing nothing like that.

JangoSteve 15y ago

I got about 4 questions in and then it started giving me basic authentication boxes saying the site is password protected, and I couldn't do anything or click any links on the site without logging in or something.

abrichr 15y ago
Sorry about that; I'm a developer, not a system administrator. It's fixed now.

thekevan 15y ago

I did not get the audio one at all