Embeddings aka the last hidden state are the mathematical representation of an input of the model before a separate model (usually the decoder) translates that

Embeddings aka the last hidden state are the mathematical representation of an input of the model before a separate model (usually the decoder) translates that hidden state to a next token (the generative part in generative ai). Normally, the this step repeats over and over. This novel approach introduces re-using the last hidden state as if it was a token that has been generated thus "evolving" the hidden state over each iteration.