Counter argument: The data necessary to train these models has been around for a while, but didnt lead to a breakthrough. What lead to breakthrough performance

Counter argument: The data necessary to train these models has been around for a while, but didnt lead to a breakthrough. What lead to breakthrough performance was new model architectures, specifically attention mechanisms. There is nothing that says you can't make much better models using the exact same data by changing the architecture of the network. In fact, that's exactly what we've been doing for the last 15 years.