4 ms·
Transformer Inference Arithmetic (2022)
- thatcherc 3y agoAuthor seems to be using billion == 10^12 instead of the common billion == 10^9. A lot of the math still works out since there's a multiply and a divide by a billion, but it is a little confusing to see passages like this: > Given the parameter count, we can multiply by two to get bytes. So to calculate the size of the weights for a 52B model. > 52e12⋅2 = 104e12 bytes ≈ 104GB
- pulvinar 3y agoI assume that's a slip back to the old British billion. But luckily that mistake doesn't affect that result, and does get it right in the next equation: > 2,097,152 bytes ≈ 0.002GB
- kipply 3y agooops will fix. never using names of numbers of zeroes ever again. that stuff sucks, i will express everything in scientific notation
- gxh8N 3y agoVery nicely written. I also like how it changes color every time I reload the article.