For transformers, especially multi-device training pipelines, yes the codebase can normally be 100ksloc and require a team to do at industry scale. See e.g. Hu

For transformers, especially multi-device training pipelines, yes the codebase can normally be 100ksloc and require a team to do at industry scale. See e.g. Hugginface Transformers, or the Megatron impl they cite. Cerebras is trying to show how easy it is to on-board single ICs and demo their pytorch integration. But yeah, where's the wallclock time comparison?! Surely they did one during development, and surely the Sales team knows (or they do once the article was published), yet not even a hint of what their throughput is like. For Cerebras to be this far and not be plastering benchmarks everywhere is a bad sign. Maybe they're going to just die off like Graphcore.