It will happily translate simple phrases into Chinese and automated translation software can read it back. As other commenters are saying the tokenizer works on

It will happily translate simple phrases into Chinese and automated translation software can read it back. As other commenters are saying the tokenizer works on bytes so a single Chinese character will be made of multiple tokens just as some English words are made of multiple tokens. GPT doesn’t see characters it sees bytes. Probably it would do better with a encoding scheme for the characters that was designed around some sort of logic or consistency, but it can go far with just memorization