3 ms·
That's a matrix multiplication + RMSNorm + SwiGLU to you. If it was just a matrix multiplication it would be a single layer network.
by astrange 1y ago
That's a matrix multiplication + RMSNorm + SwiGLU to you.
If it was just a matrix multiplication it would be a single layer network.