24 ms·The tokens are in this case actually the individual characters: vocab = sorted(list(set(lines)))by sva_ 3y agoThe tokens are in this case actually the individual characters: vocab = sorted(list(set(lines)))