That's softmax dot-product attention. It's quadratic even without fully-homomorphic encryption, but at least it's not inherently branchy, so won't necessarily s

That's softmax dot-product attention. It's quadratic even without fully-homomorphic encryption, but at least it's not inherently branchy, so won't necessarily slow down much more than other floating point operations under encryption. But softmax sampling, where you pick a single output token at the end and feed it back in to generate the next one, is branchy, so you need to do some extra encrypted computation to avoid leaking which token was sampled.