4 ms·
The thing is that I wanted to prove that ChatGPT was not able to learn from the rules and that is indeed a Language Model that puts one token after the other, i
by melenaboija 3y ago
The thing is that I wanted to prove that ChatGPT was not able to learn from the rules and that is indeed a Language Model that puts one token after the other, if it know how to play chess it is because it has seen games in the past, as I say in my post:
> If it is not memorizing, how do you think is doing it? (me)
> by trying to learning the general rules that to explain the dataset and minimize its loss. That’s what machine learning is about, it’s not called machine memorizing.
- xcv123 3y ago> Language Model that puts one token after the other, The interesting thing about the one-token-at-a-time process in OpenAI transformer LLMs is how the Attention Mechanism is executing ~1600 processes in parallel over the entire context window for each new token generated. So it is dynamically re-evaluating the entire context (including the rules of the game) in relation to the next token at each step.