s1monb

4 ms·

1. 1.▲
   by s1monb 4mo ago

   > It's especially obvious because someone writing such an article in 2026 would at least find what what the newest tokenizers are The underlying BPE algorithm, which is the main focus of this article, is the one used modern tokenize
2. 2.▲
   by s1monb 4mo ago

   I will do better and link to the research and related sources in the next iteration.
3. 3.▲
   by s1monb 4mo ago

   I appreciate the feedback. My main focus was on the visual elements, and not so much "ridding the text of AI-traces". What did you think about the more visual elements? Simon
4. 4.▲
   What Are Tokens in LLMs?(bearisland.dev)

   10 points by s1monb 4mo ago|6 comments