5 ms·How the BPE tokenization algorithm used by large language models works2 points by montebicyclelo 3y ago