I'm not sure if these models use the GPT tokenizer, but if you type a long string of numbers into https://platform.openai.com/tokenizer https://platform.openai.

I'm not sure if these models use the GPT tokenizer, but if you type a long string of numbers into https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer, you can see the tokens that the LLM would see. What the LLMs get as input for math is significantly worse then having to do mental math with roman numerals, tokenizing makes sense for words but for numbers it seems the like LLMs would have to learn a lot more steps. I wonder if limiting number tokens to 2 digits per token, instead of the 1-3 (and even more for repeated digits) it currently is would improve models math.