There's a breakdown here for anyone interested (ctrl+f "weight flops for") https://medium.com/@dzmitrybahdanau/the-flops-calculus-of-language-model-training-3b

There's a breakdown here for anyone interested (ctrl+f "weight flops for") https://medium.com/@dzmitrybahdanau/the-flops-calculus-of-language-model-training-3b19c1f025e4 https://medium.com/@dzmitrybahdanau/the-flops-calculus-of-la...