There is an insightful video review of this paper by Yannic Kilcher: https://youtu.be/Tkijsu129M0?t=30 https://youtu.be/Tkijsu129M0?t=30 TL;DW - Their approach

There is an insightful video review of this paper by Yannic Kilcher: https://youtu.be/Tkijsu129M0?t=30 https://youtu.be/Tkijsu129M0?t=30 TL;DW - Their approach is to sequentially test methods, moving on to the next if one fails. However, this strategy is flawed as it requires ground truth, particularly for multi-choice answers. The analogy could be made to continually rolling a dice until landing on six. Similarly, if a question has four potential answers, the model merely has to attempt four times to stumble upon the correct response. And then they report 100% success rate.