4 ms·
if they trained this scale without western cloud infra, i'd want to know what their token throughput setup looks like
by b0a04gl 1y ago
if they trained this scale without western cloud infra, i'd want to know what their token throughput setup looks like
- econ 1y agoSneakernet
- jaggs 1y agoThey trained on 512 H800 GPUs for three weeks, equivalent to around half a million dollars. https://xcancel.com/MiniMax__AI https://xcancel.com/MiniMax__AI
- yorwba 1y agoThat is for the reinforcement learning part. The base model was likely trained on more GPUs for significantly longer.