I also thought the dichotomous format was a significant weakness of the test and would impact on reliability. However to get 16/16 correct it would likely be do

I also thought the dichotomous format was a significant weakness of the test and would impact on reliability. However to get 16/16 correct it would likely be down to more than just chance. The more items, the more reliable the test becomes. I noticed that the article stated if you scored 15 or 16/16 you may be a super-matcher. Or, like you say you may just be super lucky to have guessed them all right. There is a formula for correcting for guessing: Corrected score = R - W/(n-1) R = right responses W = wrong responses n = number of choices for each item.