linearlayer

25 ms·

1. 1.▲
   by linearlayer 2y ago

   You seem quite upset. Can you explain exactly which quantities don't make sense?
2. 2.▲
   by linearlayer 2y ago

   No it's not
3. 3.▲
   by linearlayer 3y ago

   There is a mistake in the article, right? Multi-headed attention doesn't average together multiple attention heads. Rather it concatenates them and then right-multiplies by a matrix such that the output dimension matches the input dime