4 ms·
Words are a representation of an underlying something; LLMs do not deal with words, they deal with tokens and embeddings. There’s a transformation layer between
by monkpit 3mo ago
Words are a representation of an underlying something; LLMs do not deal with words, they deal with tokens and embeddings. There’s a transformation layer between words and inference and back.