I don't know much about this space, but it seems like this could be solved by leaving a good amount of empty tokens that you would only start using when they ar

I don't know much about this space, but it seems like this could be solved by leaving a good amount of empty tokens that you would only start using when they arise. Or leave tokens which you can use together to combine anything for various edge cases. Because if you have all the characters as tokens you can combine them into anything.