4 ms·
It would be hard to say if either of the two completely crap models is more or less crap though. Maybe by repeating it and seeing consistent results despite cha
by moffkalast 3y ago
It would be hard to say if either of the two completely crap models is more or less crap though. Maybe by repeating it and seeing consistent results despite changing other variables I guess?
- nl 3y agoNot at all. I suggest ways to measure it here: https://news.ycombinator.com/item?id=36855881 https://news.ycombinator.com/item?id=36855881 but the TL;DR is to choose a metric and compare the reduction in performance for quantized versions of the LM compared to the same LM without the modified Softmax.
- moffkalast 3y ago> we'd expect little difference I think that also likely holds for the quants, the difference could very well be within the error bars. Anyway, it's been posted to r/locallama so I'm sure someone will try it within the hour and report back soon :P