The certainty about these results reminds me of the house of cards we now understand was build in the social sciences over many years of shoddy, unreplicable wo

The certainty about these results reminds me of the house of cards we now understand was build in the social sciences over many years of shoddy, unreplicable work referencing other shoddy, unreplicable work. For example, the authors are careful, in the article above, to characterise the "probing" technique that they used as a "heuristic" and they even clearly say that there is no formal proof of its correctness: It’s believed that the higher accuracy these classifiers can get, the better the activations have learned about these real-world properties, i.e., the existence of these concepts in the model. "It's believed". The results reported in this article are conjecture, backed by belief, interpreting the unobservable state of a black-box model. It should be no surprise that the authors found what they clearly set out to find. And yet, already, there are multiple comments here on HN asserting that, hey, problem solved, we know that GPT's build world models. It is known.