I have heard this story multiple times. My impression is that often person who tells the story treats the fact that the mistrained model was NN-based as a minor

I have heard this story multiple times. My impression is that often person who tells the story treats the fact that the mistrained model was NN-based as a minor detail (or the kind of juicy but ultimately insignificant detail that make the story more fun to tell; and if the original story was about a NN, nobody is going to change it to a SVM or something else). From this viewpoint, I found the section where the author lengthly argues how this could not possibly happen with the current state of the art visual task CNNs (especially because people apply preprocessing steps such as whitening and augmentation to get rid of exactly this kind of biases), let's say, weird. The parable is not about CNNs, it is about the importance of paying attention what features your model will extract from the training dataset and whether your model is learning the right things.