> My theory is that architecture doesn't matter - convolutional, transformer or recurrent, as long as you can efficiently train models of the same size, what co

> My theory is that architecture doesn't matter - convolutional, transformer or recurrent, as long as you can efficiently train models of the same size, what counts is the dataset I assume you mean architecture doesn’t matter much, as long as it’s roughly sensibly designed. The basics of architecture do matter though: a two-layer fully connected network won’t get very far no matter how many parameters it has. Architectural improvements like ResNets have definitely made a difference to the field.