3 ms·
10B? - Is that 2 (as in binary 10)? - 108 as in a typo, or - an order of magnitude of 6 thereof?
by egberts1 5y ago
10B?
- Is that 2 (as in binary 10)?
- 108 as in a typo, or
- an order of magnitude of 6 thereof?
- microtonal 5y ago10 billion. The number of FLOPs has gone from ~1e+14 (6 layer MLP MNIST) to ~1e24 (Megatron-turing NLG). edit: s/parameters/FLOPs
- Tamaybes 5y agoNot parameters, the amount of FLOPS required to train the model.
- microtonal 5y agoWhoops, thanks! That's what I get for typing too quickly between household duties.
- dang 5y agoHN's software replaces "10 billion" with "10B". Most of the time it's clear. However, I've replaced the submitted title ('ML training compute has been growing by a factor of 10B since 2010') with what the tweet says now.