I'll link the the mention of the optimised versions, but that's not what I mean! Say earlier model XYZ trained 4 epochs per hour, and BERT trained 2 epochs per

I'll link the the mention of the optimised versions, but that's not what I mean! Say earlier model XYZ trained 4 epochs per hour, and BERT trained 2 epochs per hour. Now on a single card if you can train optimised BERT for 4 epochs per hour, that doesn't necessarily mean the same card will handle XYZ at 8 epochs per hour. It's a technological achievement nonetheless, but the fact that it was heavily optimised for the new architecture, possibly beyond an extent infeasible for non-Nvidia developers, still has to be considered. [0] https://nvidianews.nvidia.com/news/nvidia-achieves-breakthroughs-in-language-understandingto-enable-real-time-conversational-ai https://nvidianews.nvidia.com/news/nvidia-achieves-breakthro...