3 ms·
10 billion. The number of FLOPs has gone from ~1e+14 (6 layer MLP MNIST) to ~1e24 (Megatron-turing NLG). edit: s/parameters/FLOPs
by microtonal 5y ago
10 billion. The number of FLOPs has gone from ~1e+14 (6 layer MLP MNIST) to ~1e24 (Megatron-turing NLG).
edit: s/parameters/FLOPs
- Tamaybes 5y agoNot parameters, the amount of FLOPS required to train the model.
- microtonal 5y agoWhoops, thanks! That's what I get for typing too quickly between household duties.