Yes, transformers are very simple - but typically the additional lines of code are doing useful work. The comparison with nvidia megatron is particularly ridicu

Yes, transformers are very simple - but typically the additional lines of code are doing useful work. The comparison with nvidia megatron is particularly ridiculous imo. I don't see the novelty/interesting bit in this article, personally.