4 ms·
Not sure if I read it correctly, but it seems like the skip connections are kinda still present in the skipless block because they added I after the softmax and
by chessgecko 3y ago
Not sure if I read it correctly, but it seems like the skip connections are kinda still present in the skipless block because they added I after the softmax and v= the previous hidden state. Still a cool paper if they really managed to get rid of those projections without degrading the quality.