5 ms·
I think that this means that certain rank-based metrics, e.g. NDCG or AUC, which are very commonly used in ranking or classification, can be represented in a di
by huac 7y ago
I think that this means that certain rank-based metrics, e.g. NDCG or AUC, which are very commonly used in ranking or classification, can be represented in a differentiable form. That, practically, means that neural networks can easily optimize directly for those results, instead of (most commonly) minimizing log-likelihood and monitoring for results on NDCG (as in the original formulation of RankNet). My (limited) understanding of LambdaRank is that their model empirically minimizes NDCG, but does not have a strong theoretical backing for why it should work.
The experiments section is pretty clear in what potential applications can be, e.g. "optimizing directly for top-k classification loss" or "label ranking via soft Spearman’s rank correlation coefficient". In Google's case, there are pretty clear applications towards web search (top-k results, classification = do you click or not), and things like entity labeling (e.g. what label should we assign to a news story).
- pheug 7y ago> My (limited) understanding of LambdaRank is that their model empirically minimizes NDCG, but does not have a strong theoretical backing for why it should work. That's akin to saying that minimizing cross entropy empirically maximizes accuracy but there's no strong theoretical backing for that either LambdaRank is one way of getting a smooth differentiable approximation to NDCG by slapping a sigmoid somewhere. The paper we're discussing now offers another way. Hard to say which way would turn out to be empirically better on problems of practical significance without actually experimenting.