3 ms·
>probability of each token appearing given the previous tokens Sounds like tokenizer -> Markov chain? Surely something trained on a TPU is more sophisticated t
by andai 4y ago
>probability of each token appearing given the previous tokens
Sounds like tokenizer -> Markov chain? Surely something trained on a TPU is more sophisticated than something we could have done in the middle of the 20th century?