The best explanation of why these transfer that I've seen is that the problem inherently arises from dot-products in high dimensions. Real data is somewhat rand

The best explanation of why these transfer that I've seen is that the problem inherently arises from dot-products in high dimensions. Real data is somewhat random w.r.t. any particular set of activation coefficients, so the dot product looks like a sum of random variables and is subject to the central limit theorem. This keeps the real dot products small. If you are able to inject a teeny bit of the activation coefficients into the input, then as the dimensionality grows it gets easier and easier to overwhelm the real data with an epsilon sized change to the input.