4 ms·
GPT `thinks` using words/letters/tokens, as humans do we also think with words, or are we capable of thinking without any words at all. If we are, then whatever
by smusamashah 4y ago
GPT `thinks` using words/letters/tokens, as humans do we also think with words, or are we capable of thinking without any words at all. If we are, then whatever GPT will be at, it still will be only mimicking consciousness, until it can reason without using words.
- SanderNL 4y agoIt reasons with tokens, not words. They can be words, but they can be anything. Visual data can be tokenized and reasoned with.
- smusamashah 4y agoYou are right probably, that's a good point. Even actions can be tokenized.
- int_19h 4y agoIt depends on what you mean by "reason" exactly. The "thinking" parts of the model work with embeddings internally, not tokens. Or at least that's what they get as input; who knows what it becomes inside eventually. OTOH, the not-really-internal monologue when you tell it to "think it out" loud, which also drastically improves quality of the final answer, is tokens since it has to be marshalled through the context window for the next inferred token.