4 ms·
ChatGPT wasn't trained on only guessing 'the next word'. ChatGPT was trained on the best total output for the given input. The 'next word' is just intermediate
by bottlepalm 3y ago
ChatGPT wasn't trained on only guessing 'the next word'. ChatGPT was trained on the best total output for the given input.
The 'next word' is just intermediate state. Internal to the model, it knows where it is going. Each inference just revives the previous state.