4 ms·

One can generalize this approach further to take any given loss function, not just a p-norm, and ask what statistic is achieved by minimizing loss. A field tha

by bo1024 6y ago

One can generalize this approach further to take any given loss function, not just a p-norm, and ask what statistic is achieved by minimizing loss. A field that studies this is called property elicitation, from the idea that the loss function elicits a certain kind of prediction be it mean, mode, etc. This field actually originated with human experts like weather forecasters - what is a loss function (or “scoring rule”) that incents them to predict accurately? Squared loss is good if by accurately we imply we want the mean. One fact people find surprising is that squared loss is not the only one that elicits the mean. There is an entire family of losses that do, including KL divergence.

CrazyStat 6y ago
This approach is also used to extend concepts like the median into higher dimensions and non-euclidian spaces. What is the "median" or "mean" of a set of points located on the surface of a sphere, or a torus, or some more complicated manifold (which may only be defined implicitly by a distance function)? If you try to start from the one-dimensional definition it's not really clear. You can define such concepts, however, by minimizing the corresponding norms. If you minimize the sum of L1 norms then you get something like a median. If you minimize the sum of L2 norms then you get something like a mean. Once you have something like a mean you can also define something like a variance. Wikipedia has a short article about the concept [1]. It really deserves more treatment, as it's a central (no pun in intended) idea in topological data analysis [2]. [1] https://en.wikipedia.org/wiki/Fr%C3%A9chet_mean https://en.wikipedia.org/wiki/Fr%C3%A9chet_mean [2] https://en.wikipedia.org/wiki/Topological_data_analysis https://en.wikipedia.org/wiki/Topological_data_analysis
chestervonwinch 6y ago
Right! A couple more interesting (IMO) example of this: 1. The geometric mean is the minimizer of: L(x) = Σ [ log(x_i) - log(x) ]² 2. The so-called Jaccard median is the minimizer of: L(x) = Σ [ 1-J(x_i, x) ] where J is the Jaccard overlap score (aka, IoU = Intersection over Union). The second is interesting because the Jaccard score is often used to describe how close two segmentations are in image processing. The Jaccard median can be used to consolidate multiple segmentations of the same object in such a way that is best under the Jaccard distance.

by bo1024 6y ago

CrazyStat 6y ago

This approach is also used to extend concepts like the median into higher dimensions and non-euclidian spaces. What is the "median" or "mean" of a set of points located on the surface of a sphere, or a torus, or some more complicated manifold (which may only be defined implicitly by a distance function)? If you try to start from the one-dimensional definition it's not really clear. You can define such concepts, however, by minimizing the corresponding norms. If you minimize the sum of L1 norms then you get something like a median. If you minimize the sum of L2 norms then you get something like a mean. Once you have something like a mean you can also define something like a variance. Wikipedia has a short article about the concept [1]. It really deserves more treatment, as it's a central (no pun in intended) idea in topological data analysis [2]. [1] https://en.wikipedia.org/wiki/Fr%C3%A9chet_mean https://en.wikipedia.org/wiki/Fr%C3%A9chet_mean [2] https://en.wikipedia.org/wiki/Topological_data_analysis https://en.wikipedia.org/wiki/Topological_data_analysis

chestervonwinch 6y ago

Right! A couple more interesting (IMO) example of this: 1. The geometric mean is the minimizer of: L(x) = Σ [ log(x_i) - log(x) ]² 2. The so-called Jaccard median is the minimizer of: L(x) = Σ [ 1-J(x_i, x) ] where J is the Jaccard overlap score (aka, IoU = Intersection over Union). The second is interesting because the Jaccard score is often used to describe how close two segmentations are in image processing. The Jaccard median can be used to consolidate multiple segmentations of the same object in such a way that is best under the Jaccard distance.