2 ms·
What if the most expedient way to accurately continue the text sequence is to temporarily "embody" as an agent that would have generated the prior text? This wo
by mpoteat 4y ago
What if the most expedient way to accurately continue the text sequence is to temporarily "embody" as an agent that would have generated the prior text? This would involve encoding emotions, performing reasoning ability on behalf of the agent, and keeping track of its memories.
Do you find the above scenario plausible?
- tsimionescu 4y agoWell, that may well be the best way to do it, but an ANN trained using gradient descent on a corpus of text can't do that. It can only pick some weights for a differentiable higher-dimensional function that encodes probabilities for the next token given a series of tokens as input. When we find a way to do what you're describing, we probably won't require all of the books ever written plus half the internet to create a model that can speak at the level of a regular human with 12 years of schooling, but still needs to look up facts on Wikipedia to avoid saying "When Napoleon fought Genghis Khan in 2056, they both died because of a vaccine".