8 ms·

The Generalized f-Mean

61 points by goldenkey 6y ago

qsort 6y ago
Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful (for example, if you are averaging frequencies, you want the harmonic mean; if you are averaging interest rates, you want the geometric mean, etc.) However, there are two big caveats: (1) You can't use any symmetric binary function, for the mean to make sense, f must be at least injective and monotonic (and you probably want it to be continuous as well). If you pick a suitable f, many useful properties of the mean are respected. (2) In general, for an arbitrary choice of f the resulting mean won't be homogenous, which is disqualifying for most practical applications (because, for example, the mean would not be guaranteed to have the same dimensions as the elements). Restricting ourselves to functions in the form $f(x) = x^a$ for $a \in \mathbb{R} \setminus 0$ yields a well-behaved collection of means (Holder means), guaranteed to be homogenous. As a bonus, for any set $S$ of positive real numbers, if we let $GM(S)$ be the geometric mean of $S$ and $M_{a}(S)$ be the mean of $S$ for some $a$, you can prove that $\lim_{a \rightarrow 0} M_{a}(S) = GM(S)$.
- contravariant 6y ago
  Those caveats are just the beginning. You generally also want the mean to be associative, you want the function they call 'g' to be invertible, and the harmonic mean and geometric mean have the simpler and more useful interpretation of taking the average of the reciprocal and logarithm respectively. All this article is doing, in a somewhat confused fashion, is trying to replace addition with another abelian (but not associative) operator. Which is fun, but ultimately not better than just assuming the arithmetic mean is going to be any help (which at least has some strong theorems that tell you it's going to converge to the expected value, usually).
  - qsort 6y ago
    Yeah, I agree the author is abstracting so much that the object he's defining doesn't have any useful property left. I genuinely smiled when the article says "any function". "Any function" in math means any function :-) There's a reason why theorems involving arbitrary functions start with a list of qualifiers as hypotheses.
- ttoinou 6y ago
  > Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean. Frequencies also don't add up.... But wavelength or time do. So their inverse can be added. So their inverses can have an arithmetic mean. So you can physically meaningfully do the arithmetic mean of their inverse, and inverse the result back again
  - qsort 6y ago
    > That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you're going down that route, which operations make sense and which don't boil down to the algebraic structure of the set of the possible values of the datatype. For example, you can't add dates but you can subtract dates, because dates have an affine structure, therefore the arithmetic mean of January 14 and October 13 is an operation that makes sense. Similarly with temperatures. The thing is that it's sometimes tricky and unintuitive to figure out what you're allowed to do: ratios are a good example because you can add them, it just doesn't mean what most people expect it to mean. I do agree that understanding your data goes a long way, but I believe making people more aware that tools other than "sum / n" exist is worthwhile, even though the article does have problems as I highlighted above.
    - ttoinou 6y ago
      Yes, but no temperatures cannot be substracted (nor added). For dates I believe you're right because they are vectors (what you think is "an absolute date in the calendar" is in fact the time elapsed after 00h00 year=1 after christ, so it's in fact relative)
      - 6y ago
deleted 6y ago
[deleted]
Jenz 6y ago
They claim the inverse function of f(a,b) = a + b, is f(a,b) = a - b. Surely this is wrong. By my limited understaning of the basic structures of functions; when f: R^2 -> R, then f inverse ought to be R -> R^2 right? And when f is just addition there can’t be an inverse solving for both a and b. (If one of a and b were fixed, it’d be way easier, that is, if f(a) = a + b, then g(a) = a - b is an inverse of f)
- zodiac 6y ago
  Yes, I noticed this from the statement "apply that binary operation to one number from the list". They're basically treating f as a function from R to R.
- munchbunny 6y ago
  This also didn’t make sense to me. It’s not an inverse. So where is the actual logical link?
  - deleted 6y ago
    [deleted]
- wenc 6y ago
  I don't quite follow the example either. For a list (2, 4, 6), the function is applied to the first number and the "inverse" function (though it's really not) on the last number of the list, with an argument of b=2. I must be missing something in the argument -- I'm wondering, why the choice of b=2? And was the function not applied to the number 4? Edit: I believe the author was trying to show (albeit incorrectly) the generalization of an arithmetic mean using f(x) = ax + b, with a = 1 and b = 2 (not f(a,b) as stated; plus the inverse function was applied incorrectly at the element level). The f-mean (Mf) is defined as: Mf(x₁,...,xₙ) = f⁻¹( (1/n) ∑ₖⁿ f(xₖ) ) where the function f is injective and continuous. Different choices of f result in different means. This wikipedia article provides relevant details on the f-mean. https://en.wikipedia.org/wiki/Quasi-arithmetic_mean https://en.wikipedia.org/wiki/Quasi-arithmetic_mean
srean 6y ago
I was having a conversation on HN a few days ago about how the mean of a set can be seen as minimizer of an optimization problem -- roughly, it can be interpreted as that location that minimizes the 'distance' to all other points in a set. By choosing the 'distance' appropriately one can get all sorts of fun generalizations that includes all mentioned in the post. The familiar squared Euclidean distance yields the familiar arithmetic mean. One fruitful family of such 'distances' that lead to arithmetic mean, geometric mean, Holder's mean [1] and beyond etc is the Bregman divergence [0]. The sort of means that fall out from it have the form f⁻¹ ( 1/n ∑ᵢ₌₁ⁿ f(xᵢ) ) where f is a smooth monotonically increasing function. By f⁻¹ I mean the inverse function, not the reciprocal. [0] https://en.wikipedia.org/wiki/Bregman_divergence https://en.wikipedia.org/wiki/Bregman_divergence [1] https://en.wikipedia.org/wiki/Generalized_mean https://en.wikipedia.org/wiki/Generalized_mean [2] https://news.ycombinator.com/item?id=25221587 https://news.ycombinator.com/item?id=25221587
azalemeth 6y ago
I really think the most "understandable" derivation of the arithmetic mean is that as a maximum-likelihood estimator of the mean of gaussian IID random variables. It literally requires a touch of calculus, and it pops out beautifully. Of course, proving it in general is a bit more involved, but...
- srean 6y ago
  One can do without calculus in this case. Using the fact that a squared expression is positive one can show that the arithmetic mean is indeed the maximum likelihood estimator. For the general case one cant avoid calculus.
emilfihlman 6y ago
This has issues on many levels, but most importantly f(a, b)=a+b is not invertible.
tingletech 6y ago
2017
enriquto 6y ago
generalized f-means are still just a very particular case of aggregator functions. It leaves out, for example, the median!

61 points by goldenkey 6y ago

Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful (for example, if you are averaging frequencies, you want the harmonic mean; if you are averaging interest rates, you want the geometric mean, etc.) However, there are two big caveats: (1) You can't use any symmetric binary function, for the mean to make sense, f must be at least injective and monotonic (and you probably want it to be continuous as well). If you pick a suitable f, many useful properties of the mean are respected. (2) In general, for an arbitrary choice of f the resulting mean won't be homogenous, which is disqualifying for most practical applications (because, for example, the mean would not be guaranteed to have the same dimensions as the elements). Restricting ourselves to functions in the form $f(x) = x^a$ for $a \in \mathbb{R} \setminus 0$ yields a well-behaved collection of means (Holder means), guaranteed to be homogenous. As a bonus, for any set $S$ of positive real numbers, if we let $GM(S)$ be the geometric mean of $S$ and $M_{a}(S)$ be the mean of $S$ for some $a$, you can prove that $\lim_{a \rightarrow 0} M_{a}(S) = GM(S)$.

contravariant 6y ago
Those caveats are just the beginning. You generally also want the mean to be associative, you want the function they call 'g' to be invertible, and the harmonic mean and geometric mean have the simpler and more useful interpretation of taking the average of the reciprocal and logarithm respectively. All this article is doing, in a somewhat confused fashion, is trying to replace addition with another abelian (but not associative) operator. Which is fun, but ultimately not better than just assuming the arithmetic mean is going to be any help (which at least has some strong theorems that tell you it's going to converge to the expected value, usually).
- qsort 6y ago
  Yeah, I agree the author is abstracting so much that the object he's defining doesn't have any useful property left. I genuinely smiled when the article says "any function". "Any function" in math means any function :-) There's a reason why theorems involving arbitrary functions start with a list of qualifiers as hypotheses.
ttoinou 6y ago
> Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean. Frequencies also don't add up.... But wavelength or time do. So their inverse can be added. So their inverses can have an arithmetic mean. So you can physically meaningfully do the arithmetic mean of their inverse, and inverse the result back again
- qsort 6y ago
  > That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you're going down that route, which operations make sense and which don't boil down to the algebraic structure of the set of the possible values of the datatype. For example, you can't add dates but you can subtract dates, because dates have an affine structure, therefore the arithmetic mean of January 14 and October 13 is an operation that makes sense. Similarly with temperatures. The thing is that it's sometimes tricky and unintuitive to figure out what you're allowed to do: ratios are a good example because you can add them, it just doesn't mean what most people expect it to mean. I do agree that understanding your data goes a long way, but I believe making people more aware that tools other than "sum / n" exist is worthwhile, even though the article does have problems as I highlighted above.
  - ttoinou 6y ago
    Yes, but no temperatures cannot be substracted (nor added). For dates I believe you're right because they are vectors (what you think is "an absolute date in the calendar" is in fact the time elapsed after 00h00 year=1 after christ, so it's in fact relative)
    - 6y ago

deleted 6y ago

[deleted]

Jenz 6y ago

They claim the inverse function of f(a,b) = a + b, is f(a,b) = a - b. Surely this is wrong. By my limited understaning of the basic structures of functions; when f: R^2 -> R, then f inverse ought to be R -> R^2 right? And when f is just addition there can’t be an inverse solving for both a and b. (If one of a and b were fixed, it’d be way easier, that is, if f(a) = a + b, then g(a) = a - b is an inverse of f)

zodiac 6y ago
Yes, I noticed this from the statement "apply that binary operation to one number from the list". They're basically treating f as a function from R to R.
munchbunny 6y ago
This also didn’t make sense to me. It’s not an inverse. So where is the actual logical link?
- deleted 6y ago
  [deleted]
wenc 6y ago
I don't quite follow the example either. For a list (2, 4, 6), the function is applied to the first number and the "inverse" function (though it's really not) on the last number of the list, with an argument of b=2. I must be missing something in the argument -- I'm wondering, why the choice of b=2? And was the function not applied to the number 4? Edit: I believe the author was trying to show (albeit incorrectly) the generalization of an arithmetic mean using f(x) = ax + b, with a = 1 and b = 2 (not f(a,b) as stated; plus the inverse function was applied incorrectly at the element level). The f-mean (Mf) is defined as: Mf(x₁,...,xₙ) = f⁻¹( (1/n) ∑ₖⁿ f(xₖ) ) where the function f is injective and continuous. Different choices of f result in different means. This wikipedia article provides relevant details on the f-mean. https://en.wikipedia.org/wiki/Quasi-arithmetic_mean https://en.wikipedia.org/wiki/Quasi-arithmetic_mean

srean 6y ago

I was having a conversation on HN a few days ago about how the mean of a set can be seen as minimizer of an optimization problem -- roughly, it can be interpreted as that location that minimizes the 'distance' to all other points in a set. By choosing the 'distance' appropriately one can get all sorts of fun generalizations that includes all mentioned in the post. The familiar squared Euclidean distance yields the familiar arithmetic mean. One fruitful family of such 'distances' that lead to arithmetic mean, geometric mean, Holder's mean [1] and beyond etc is the Bregman divergence [0]. The sort of means that fall out from it have the form f⁻¹ ( 1/n ∑ᵢ₌₁ⁿ f(xᵢ) ) where f is a smooth monotonically increasing function. By f⁻¹ I mean the inverse function, not the reciprocal. [0] https://en.wikipedia.org/wiki/Bregman_divergence https://en.wikipedia.org/wiki/Bregman_divergence [1] https://en.wikipedia.org/wiki/Generalized_mean https://en.wikipedia.org/wiki/Generalized_mean [2] https://news.ycombinator.com/item?id=25221587 https://news.ycombinator.com/item?id=25221587

azalemeth 6y ago

I really think the most "understandable" derivation of the arithmetic mean is that as a maximum-likelihood estimator of the mean of gaussian IID random variables. It literally requires a touch of calculus, and it pops out beautifully. Of course, proving it in general is a bit more involved, but...

srean 6y ago
One can do without calculus in this case. Using the fact that a squared expression is positive one can show that the arithmetic mean is indeed the maximum likelihood estimator. For the general case one cant avoid calculus.

emilfihlman 6y ago

This has issues on many levels, but most importantly f(a, b)=a+b is not invertible.

tingletech 6y ago

2017

enriquto 6y ago

generalized f-means are still just a very particular case of aggregator functions. It leaves out, for example, the median!