4 ms·

Great job. Seeing this solve 10/10, it almost feels like artificial intelligence. You should send a resume to the Wolfram Alpha guys (their hit rate was signifi

Great job. Seeing this solve 10/10, it almost feels like artificial intelligence. You should send a resume to the Wolfram Alpha guys (their hit rate was significantly lower: http://news.ycombinator.com/item?id=1891375 http://news.ycombinator.com/item?id=1891375)

Xk 16y ago
Yeah, but I have the advantage of knowing all the forms of the questions. If you were to change one word then mine would break. Or misspell anything.
- techiferous 16y ago
  I just typed in similar questions without following the format and it performed poorly. But at least it's a good proof-of-concept! :) Name the third item in the list: book, fork and spoon. A: FAILURE Q: What is two times 8? A: Which is a number: yellow, seven, book, flag? A: FAILURE
  - Xk 16y ago
    They don't give questions formatted in other ways, so I didn't include that in the logic. As someone else pointed out, even deleting the question mark would break it. (However, making the question mark optional would be a two second fix.)
- joelvh 16y ago
  Great job! I wrote up that post about WolframAlpha and wonder if there's an API so you could integrate WA to make it more robust?
  - Xk 16y ago
    Right now there's no API or anything -- I wrote this just to see if I could. I'm not sure if I'll be putting up an API, because I figure by the time I'd do that I can just release the source for it. But if you really wanted I probably could provide an API it in a couple of days.
- gibeson 16y ago
  Sorry, but what do you mean by "knowing all the forms of the questions"? It seems like maybe you mean all that all the questions formatted via one of several formulas like: name <x> of list<a..z>. Which would imply your tool simply identifies the format then applies some specific logic to solve the question. I'm unclear that if that is what you mean, why then do you know all the forms of the questions? Text capture purports over 180 million questions, so isn't it likely you may not have found all the possible formats of the questions? Or perhaps did you know because you have a relationship with the text captcha creator and they provided that info to you. Or better yet, did your comment mean something entirely different. Just curious, thanks.
  - Xk 16y ago
    You are correct, I look for a pattern and apply very very basic logic. I just got 1,000 questions from the demo page and looked how they were similar. For example, there are many questions of the form "[list of something]: the [ordinal number] [type] is?" so I put that as a rule in the grammar file. I did this for about twenty different types of questions, each of which has three or so different phrasings. And yes, they may have 180 million questions, but in reality they have only a hundred or so modulo the specific words they pick (for the items in the list, or for the day of the week, or for the letter that the word starts with).
    - pyre 16y ago
      > I put that as a rule in the grammar file What are you using the process the grammar file? What type of grammar?
      - Xk 16y ago
        I wrote a parser generator, so I'm using that. It's BNF-like.
- jamesjyu 16y ago
  It's basically an arms race. They could inject a few misspellings and random noise here and there to throw your system off.
  - endtime 16y ago
    Fuzzy pattern matching would probably solve that.
adamc 16y ago
The world orange has which letter in the penultimate position? A: FAILURE Aircraft fly through the? A: FAILURE Q: A beetle has how many legs? A: a It does well when it recognizes the questions, but there's still plenty of room for writing new ones. It does show that it's an arms race, as someone else noted.
- Xk 16y ago
  Very true. The person writing the questions has the advantage in this one because they can always throw in more variations and I would have to figure out all of the new ones. There is, however, an inherit limitation to question-asking, and that is it must be understandable to a general population. So I would switch from using a parser to just looking for key words and then matching from there. (Which is, for example, how I find names in a list.) Even if they made it so I only get one fifth the number of correct answers, a 20% success ratio would be high enough for concern. The attackers can just keep trying and trying and if 20% of the time they can register a new bot, then they win. The defender must win a much larger percent of the time.
- BoppreH 16y ago
  First question could be "air" or "sky". I would guess 6 or 8 to the second one. I would probably fail both, so I guess this isn't the kind of question that would realistically appear.

Xk 16y ago

Yeah, but I have the advantage of knowing all the forms of the questions. If you were to change one word then mine would break. Or misspell anything.

techiferous 16y ago
I just typed in similar questions without following the format and it performed poorly. But at least it's a good proof-of-concept! :) Name the third item in the list: book, fork and spoon. A: FAILURE Q: What is two times 8? A: Which is a number: yellow, seven, book, flag? A: FAILURE
- Xk 16y ago
  They don't give questions formatted in other ways, so I didn't include that in the logic. As someone else pointed out, even deleting the question mark would break it. (However, making the question mark optional would be a two second fix.)
joelvh 16y ago
Great job! I wrote up that post about WolframAlpha and wonder if there's an API so you could integrate WA to make it more robust?
- Xk 16y ago
  Right now there's no API or anything -- I wrote this just to see if I could. I'm not sure if I'll be putting up an API, because I figure by the time I'd do that I can just release the source for it. But if you really wanted I probably could provide an API it in a couple of days.
gibeson 16y ago
Sorry, but what do you mean by "knowing all the forms of the questions"? It seems like maybe you mean all that all the questions formatted via one of several formulas like: name <x> of list<a..z>. Which would imply your tool simply identifies the format then applies some specific logic to solve the question. I'm unclear that if that is what you mean, why then do you know all the forms of the questions? Text capture purports over 180 million questions, so isn't it likely you may not have found all the possible formats of the questions? Or perhaps did you know because you have a relationship with the text captcha creator and they provided that info to you. Or better yet, did your comment mean something entirely different. Just curious, thanks.
- Xk 16y ago
  You are correct, I look for a pattern and apply very very basic logic. I just got 1,000 questions from the demo page and looked how they were similar. For example, there are many questions of the form "[list of something]: the [ordinal number] [type] is?" so I put that as a rule in the grammar file. I did this for about twenty different types of questions, each of which has three or so different phrasings. And yes, they may have 180 million questions, but in reality they have only a hundred or so modulo the specific words they pick (for the items in the list, or for the day of the week, or for the letter that the word starts with).
  - pyre 16y ago
    > I put that as a rule in the grammar file What are you using the process the grammar file? What type of grammar?
    - Xk 16y ago
      I wrote a parser generator, so I'm using that. It's BNF-like.
jamesjyu 16y ago
It's basically an arms race. They could inject a few misspellings and random noise here and there to throw your system off.
- endtime 16y ago
  Fuzzy pattern matching would probably solve that.

adamc 16y ago

The world orange has which letter in the penultimate position? A: FAILURE Aircraft fly through the? A: FAILURE Q: A beetle has how many legs? A: a It does well when it recognizes the questions, but there's still plenty of room for writing new ones. It does show that it's an arms race, as someone else noted.

Xk 16y ago
Very true. The person writing the questions has the advantage in this one because they can always throw in more variations and I would have to figure out all of the new ones. There is, however, an inherit limitation to question-asking, and that is it must be understandable to a general population. So I would switch from using a parser to just looking for key words and then matching from there. (Which is, for example, how I find names in a list.) Even if they made it so I only get one fifth the number of correct answers, a 20% success ratio would be high enough for concern. The attackers can just keep trying and trying and if 20% of the time they can register a new bot, then they win. The defender must win a much larger percent of the time.
BoppreH 16y ago
First question could be "air" or "sky". I would guess 6 or 8 to the second one. I would probably fail both, so I guess this isn't the kind of question that would realistically appear.