>> They've been able to demonstrate that the trained Othello player has a model. They haven't demonstrated that, it remains a conjecture. They show some eviden

>> They've been able to demonstrate that the trained Othello player has a model. They haven't demonstrated that, it remains a conjecture. They show some evidence that they claim supports their conjecture. This evidence comes from "probing" (the classifier-based method you describe) which is a heuristic method that lacks the power to demonstrate anything conclusively. This is as the authors point out: It’s believed that the higher accuracy these classifiers can get, the better the activations have learned about these real-world properties, i.e., the existence of these concepts in the model. "It's believed" as in, we don't know, therefore we can't use to demonstrate.