> Now we can learn even the image priors (spatial locality and translation invariance) from data. Right. The attention layers even learn attention patterns whi

> Now we can learn even the image priors (spatial locality and translation invariance) from data. Right. The attention layers even learn attention patterns which look like convolution layer kernels! But better, presumably: https://arxiv.org/abs/1911.03584 https://arxiv.org/abs/1911.03584