The fact that the model seems to be able to make highly accurate predictions even on the images in Figure 2 (including HPF 50 and LPF 10) makes me skeptical. I

The fact that the model seems to be able to make highly accurate predictions even on the images in Figure 2 (including HPF 50 and LPF 10) makes me skeptical. It feels much more probable that this is a sign of data leakage than that the underlying true signal is so strong that it persists even under these transformations. https://arxiv.org/pdf/2011.06496.pdf https://arxiv.org/pdf/2011.06496.pdf Compare the performance under high pass and low pass filters in this paper on CIFAR-10. Is it really the case that differentiating cats from airplanes is so much more fragile than predicting race from chest x-rays?