Nope, you could shove in an embedding that didn't represent an existing token. It would work just fine. (if not obvious.. you'd shove it in right after the emb

Nope, you could shove in an embedding that didn't represent an existing token. It would work just fine. (if not obvious.. you'd shove it in right after the embedding layer...)