2 ms·
> a good test would be to modify one of the smaller models (maybe nanogpt) and swap out all of the softmax calls for his quiet softmax. You'd have to train the
by LoganDark 3y ago
> a good test would be to modify one of the smaller models (maybe nanogpt) and swap out all of the softmax calls for his quiet softmax.
You'd have to train the model with the quiet softmax before inferencing with it would work.