3 ms·
They made an extremely strong claim: > None of this would matter if it changed the model's answers If they want to assert that the answers don’t change, then
by amluto 2mo ago
They made an extremely strong claim:
> None of this would matter if it changed the model's answers
If they want to assert that the answers don’t change, then perhaps they should calculate the statistical distance between the token probability outputs or something to that effect. I doubt the results would indicate that the answers don’t change by any reasonable interpretation.
Maybe the results are still good enough.
- scrlk 2mo agoKL divergence is your friend when it comes to evaluating the effects of quantisation: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_diver...
- stingraycharles 2mo agoIsn’t that already in detail by the research of these quantization techniques?