4 ms·Show HN: Byte-Pair Encoding tokenizer for training LLMs on large datasets5 points by yu3zhou4 2y ago