Think of it like this: the final softmax layer is like being forced to pick a single word as your next prediction, while the hidden layer contains all the reaso

Think of it like this: the final softmax layer is like being forced to pick a single word as your next prediction, while the hidden layer contains all the reasoning and understanding that led to that decision. It's similar to how a human might have a complex thought but needs to reduce it to a single word when speaking. Many existing applications make use of hidden layers in a transformer to perform useful tasks such as classification. The concept of an “embedding” is simply the output of a hidden layer, after all.