3 ms·
Machine learning models don't do well with math that has hard bounds at 1 and 0 (probability). So we use the log of the odds ratio instead. This function goes f
by spywaregorilla 4y ago
Machine learning models don't do well with math that has hard bounds at 1 and 0 (probability). So we use the log of the odds ratio instead. This function goes from negative infinity to positive infinity. And is mappable back to 0 and 1. So as you get closer to infinity log odds, you get closer to 100% certainty that your answer is A.
It's just ln(p/(1-p)) under the hood.
https://en.wikipedia.org/wiki/Logit https://en.wikipedia.org/wiki/Logit
- abecedarius 4y agoYeah -- the choice of temperature corresponds to the base of the logarithm. When you think of probabilities in the log-odds representation, a bayesian update means adding an increment. https://arbital.com/p/bayes_log_odds/ https://arbital.com/p/bayes_log_odds/
- albystein 4y agothis is not what language models typically output as logits though. perhaps the term now is being misused and it’s causing a lot of confusion