I think that some of these choices (as others have commented) show that the author has not investigated how tokenization works. Tokenization is not some black

I think that some of these choices (as others have commented) show that the author has not investigated how tokenization works. Tokenization is not some black box, you can run tokenizers and check them.