2 ms·
> We're trained on significantly more data than ChatGPT et al. I don't think that's true anymore. GPT-4 is rumored to have been trained on a dataset with >1 tr
by alephxyz 2y ago
> We're trained on significantly more data than ChatGPT et al.
I don't think that's true anymore. GPT-4 is rumored to have been trained on a dataset with >1 trillion tokens. An average human would take a few thousand years to read through that dataset once.
- anon291 2y ago1 trillion tokens is about 1024 GB (assuming 1 byte per token). Humans ingest about 74GB of data per day: https://kids.frontiersin.org/articles/10.3389/frym.2017.00023 https://kids.frontiersin.org/articles/10.3389/frym.2017.0002... That means for a human baby to be as smart as GPT they would have to go from zero to speaking English within two weeks of birth. Your error is mistaking training data for text only. Humans are particularly inefficient with text because it takes a lot of visual data to get to the thought of a particular word or character due to how human senses work (we have no 'token' sense).