You're right that our post doesn't quite show that GPT4 cannot perform well on MIT curriculum. We try to be up front about this in the conclusion: > Our critiqu

You're right that our post doesn't quite show that GPT4 cannot perform well on MIT curriculum. We try to be up front about this in the conclusion: > Our critiques are largely of the methodology and rigor of this study, not about its content. We make no claim about the ability of large language models to actually solve MIT curricula, only that this paper fails to prove it in a scientifically rigorous way. Though, as MIT undergraduates ourselves, we can at the very least say that the test set that we accessed does not, at least in our experience, accurately represent the breadth and depth of understanding required to complete an EECS degree at MIT.