3 ms·
I wish they didn’t use swiGLU and preRMSnorm so we could have a better comparison. Then we would know how much this transformer innovation helps by itself.
by h_tbob 2y ago
I wish they didn’t use swiGLU and preRMSnorm so we could have a better comparison.
Then we would know how much this transformer innovation helps by itself.