Who writes the question and answer? Although...maybe you could outsource the question and answering to Mechanical Turk. Turn the whole thing on its head. Have

Who writes the question and answer? Although...maybe you could outsource the question and answering to Mechanical Turk. Turn the whole thing on its head. Have a real person write a question to try to trick the bot into revealing its botness, have the real human grade the answer.