92 ms·
yeah a common misconception people think because the input is the same they forget that their is a pre attention linear transofrmation for q k and v (using the
by ewild 3y ago
yeah a common misconception people think because the input is the same they forget that their is a pre attention linear transofrmation for q k and v (using the decoder only version obv v is diff with encoder decoder bert style)