5 ms·
>"While our soft operators are several hours faster than OT, they are slower than All-pairs, despite its O(n^2)complexity. This is due the fact that, with n= 10
by GistNoesis 7y ago
>"While our soft operators are several hours faster than OT, they are slower than All-pairs, despite its O(n^2)complexity. This is due the fact that, with n= 100, All-pairs is very efficient on GPUs, while our PAV implementation runs on CPU"
Paper is interesting but not yet sure of practical uses.
The trick I use in practice when I need a differentiable sort, is usually a pre-sort step which involves thresholding (i.e. selecting with sparsity only values greater than a certain score (usually either a constant, or a fraction of the best score, or the Kth score ) ). Then pay the quadratic price with n=10 or 20.
I don't see when the relevance of rank between garbage results would really matter.
When n get bigger and you don't want to ignore bad results, usually quantile approximations suffice.
In the applications they cite :
The smart use (cross-validation) of threshold by the Huber loss in section 6.4 works better 2 out of 3 times in their own graphs).
The other use cases when order matters (for example like in section 6.3 is where the rankings are given as input). If n is low you pay the quadratic cost, if n is high you usually need to process samples a subset at a time for memory reasons and use some comparison losses (triplet loss...). So this is relevant only in the sweet spot in between if you need exact calculations.