Right. I stand corrected and it makes sense. I may have misunderstood the OP. It would not make sense to encode each unicode point as a token and then start tra

Right. I stand corrected and it makes sense. I may have misunderstood the OP. It would not make sense to encode each unicode point as a token and then start training the subword tokenizer on top unless we go to the vocabularies of many millions.