4 ms·
Can someone please explain the "Scaled dot-product attention" function from the article? Attention(Q, K, V ) = softmax( (Q * TRANSPOSED(T)) / sqrt(Dk) ) V Th
by MichaelMoser123 3y ago
Can someone please explain the "Scaled dot-product attention" function from the article?
Attention(Q, K, V ) = softmax( (Q * TRANSPOSED(T)) / sqrt(Dk) ) V
That's where i start to shake my head.