I think, the models (especially transformers) are kind of straight forward. But the most difficult part is: - intuition behind them, why they work at all. What

I think, the models (especially transformers) are kind of straight forward. But the most difficult part is: - intuition behind them, why they work at all. What a good parameter set is - how to check if training goes well - what to do when things go wrong - how to train large models. I mean gpt4 was trained for a long time. For sure they did some things before and during the long training period. And so on. I mean because the models are rather „simple“, I think the best engineers get paid pretty high, because it is not that much about the models but everything around it.