3 ms·

I just saw Norvig give this talk. The quote cuts off before he offers his explanation: the 1.5 means 'trust this data set more.' For whatever reason (e.g. big

by jhales 15y ago

I just saw Norvig give this talk. The quote cuts off before he offers his explanation: the 1.5 means 'trust this data set more.' For whatever reason (e.g. bigger sample size) the probabilities from that set tend to be more accurate. It's not some sort of deep mystical counter example. It's a clever tweak that comes from the observation that empirical observations are not uniformly reliable.

pool007 15y ago
As it's probability^1.5, it makes the probability lower, and not higher, i.e., x^1.5 < x if 0 <= x <=1. I think this might be a tweak due to data scarcity. To compute p(e), there's tons of data to build a model. On the other hand, p(e|f) requires for you to get a lot of parallel corpus (texts that's written both in English and Foreign language) which is not easily doable. As a result, p(e)^1.5 * p(e|f), lowers p(e) as it's supposed to be too high compared to p(e|f), imho.
- jhales 15y ago
  Good point, my mistake :)
- carlob 15y ago
  I think it's more helpful to think of it as a a gamma correction with γ > 1, rather than saying higher or lower. Or as I said in another comment lower temperature, in that it concentrates probability around its peaks.
robertskmiles 15y ago
Given accurate probabilities as input, Bayes always gives the best result. That's a theorem. So the fact that this tweak improves the performance probably means that either the english model probabilities are systematically too low, or the translation probabilities are systematically too high. My guess is the latter.