By "embedding" in this context what you're actually referring to is called a "token" which are sub-word strings of usually 1-4 characters. It's not possible fo

By "embedding" in this context what you're actually referring to is called a "token" which are sub-word strings of usually 1-4 characters. It's not possible for you to ask it things even slightly different from it training data, unless you ask exclusively in emojis that didn't exist yet when it was trained (in which case it sees nothing, just like when someone sends you an emoji your phone doesn't support). Any novel sentence and even novel words like "Blobdarfnk" ARE in its training data. "Blobdarfnk" is encoded as the five tokens Bl, ob, dar, fn, and k.