I am surprised that GPT2 was disqualified. To my knowledge, it can answer this type of question - just needs a different text prompt. GPT2 is closer to a plain

I am surprised that GPT2 was disqualified. To my knowledge, it can answer this type of question - just needs a different text prompt. GPT2 is closer to a plain text generator than anything biased toward conversation. In contrast, GPT3 davinci-003 is strongly biased to provide a conversation-like experience. For GPT2, "please respond in the following format" is unlikely to work. An appropriate prompt for GPT2 would be more tweaking (and still result in worse results), but be in the line of: This is an example of a talented lawyer acing the Bar Exam. QUESTION: {question_text} (A) {row["choice_a"].strip()} (B) {row["choice_b"].strip()} (C) {row["choice_c"].strip()} (D) {row["choice_d"].strip()} Pick only one answer: A, B, C or D. TALENTED LAWYER: