I wouldn’t discount the complexity of the code and development. The model architecture itself is incredibly complex, likely with tons of custom layers and tenso

I wouldn’t discount the complexity of the code and development. The model architecture itself is incredibly complex, likely with tons of custom layers and tensor operators, along with all the custom tooling for data I/o, likely custom optimization package for training, utilities for observability and diagnostics, and the actual configuration/orchestration of storage and compute resources… And then you have the resources themselves. Which enable them to iterate more quickly on building all of the above. Oh and the training dataset. It’s a big moat, all things considered.