Resnet works better than MLP not because it optimizes faster, but because the architecture of the network contains a bias towards spatially structured data. MLP

Resnet works better than MLP not because it optimizes faster, but because the architecture of the network contains a bias towards spatially structured data. MLP's see images as sorted or scrambled, the relationship of each pixel to another pixel is the same at the start (or randomly stronger or weaker if you initialize the network randomly).