That's not what dropout is, dropout is just a method of regularization (you remove a random, and small, subset of the neurons per training iteration, in order t

That's not what dropout is, dropout is just a method of regularization (you remove a random, and small, subset of the neurons per training iteration, in order to reduce overfitting), and pretty much all LLM transformer blocks have dropout.