3 ms·
We do the same thing in our brains. 1 is a token in our brain, so is 2, and 3.
by m00x 4y ago
We do the same thing in our brains.
1 is a token in our brain, so is 2, and 3.
- alpaca128 4y agoAs pointed out by gerad above, we humans don't treat multi-digit numbers as one atomic token. We know 11 = 10 + 1. But if an AI model has no access to individual digits no matter how large the number is, then the task of doing simple maths is exponentially harder to learn.
- vintermann 4y agoWell, it's certainly not the lowest level token - to the degree we can talk about tokens in our brains in the first place. We can talk and think about such things as the shape of the letters, which it would take a Herculean effort for GPT-like models to learn, since it gets letters in a form where that information has been stripped away.