Per the paper, 3072 H100s over the course of 3 months, assume a cost of 2$/GPU/hour That would be roughly 13.5M$ USD I’m guessing that at this scale and cost,

Per the paper, 3072 H100s over the course of 3 months, assume a cost of 2$/GPU/hour That would be roughly 13.5M$ USD I’m guessing that at this scale and cost, this model is not competitive and their ambition is to scale to much larger models. In the meantime , they learned a lot and gain PR from open-sourcing