Of course when testing an LLM you do not use the exact text from studies in the training set. There are an infinite number of structural permutations that prese

Of course when testing an LLM you do not use the exact text from studies in the training set. There are an infinite number of structural permutations that preserve the methodology sufficiently to be comparable to the existing literature, while being distinct enough to be outside the training distribution. Of course, another approach you can take if you suspect contamination is to ask follow-up questions that are less likely to be in any training data; if the LLM is a mere stochastic parrot without a ToM it will not be able to give well-formed answers to follow-up questions. Perhaps if I'd said "established psychology research methodology" the intent would have been clearer?