> Transformers et al would have been figured out in no time had we had the sort of data tools we have today. I disagree. Even things that seem obvious in retro

> Transformers et al would have been figured out in no time had we had the sort of data tools we have today. I disagree. Even things that seem obvious in retrospect, take some time to be figured out. Resnets, batch norm, dropout, etc are examples of this. And I don't think transformers are obvious.