4 ms·

I see this "175B parameters" figure tossed around a lot: what does that mean? Does the model literally consist of 175 billion 32-bit floats? (Sorry for the ver

I see this "175B parameters" figure tossed around a lot: what does that mean? Does the model literally consist of 175 billion 32-bit floats? (Sorry for the very basic question.)

qayxc 6y ago
You would be correct. It's ~650GiB of data. The entire English version of Wikipedia is about 20GiB and the Encyclopædia Britannica is only about 300MiB (measured in ASCII characters) [1]. [1] https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons https://en.wikipedia.org/wiki/Wikipedia:Size_comparisons
- pradn 6y ago
  The English Wikipedia text (no images) is about ~20 GB compressed, but ~60 GB uncompressed.
  - qayxc 6y ago
    I only use the sources available to me at the time. Another value published by Wikipedia is 30GiB [1] as of 2020, which includes punctuation and markup. I explicitly put the measurement unit as ASCII characters. If you have a better source for your size (remember: ASCII characters for article words only, no markup), feel free to post it. [1] https://en.wikipedia.org/wiki/Wikipedia:Size_in_volumes https://en.wikipedia.org/wiki/Wikipedia:Size_in_volumes