3 ms·
It's not a "benefit", it's simply how the technology works - the underlying model just fundamentally works on tokens as it's atomic inputs. The models don't kn
by hn_throwaway_99 4y ago
It's not a "benefit", it's simply how the technology works - the underlying model just fundamentally works on tokens as it's atomic inputs.
The models don't know anything about words, just tokens.