3 ms·
I don't think that's always correct when you're talking about operators in neural nets. E.g. the sin and cos in rope embeddings would get more precise, large su
by jebarker 2y ago
I don't think that's always correct when you're talking about operators in neural nets. E.g. the sin and cos in rope embeddings would get more precise, large sums like softmax would become more precise, potentially attention too due to dot products