I was very curious and checked with this string, "eeeeeeggeeeee", which should be tokenized as [eeee][ee][g][ge][eeee]. Both GPT3.5 and GPT4 gave me "egg", whic

I was very curious and checked with this string, "eeeeeeggeeeee", which should be tokenized as [eeee][ee][g][ge][eeee]. Both GPT3.5 and GPT4 gave me "egg", which is a single token.