3 ms·

> Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful That comes from

> Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean. Frequencies also don't add up.... But wavelength or time do. So their inverse can be added. So their inverses can have an arithmetic mean. So you can physically meaningfully do the arithmetic mean of their inverse, and inverse the result back again

qsort 6y ago
> That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you're going down that route, which operations make sense and which don't boil down to the algebraic structure of the set of the possible values of the datatype. For example, you can't add dates but you can subtract dates, because dates have an affine structure, therefore the arithmetic mean of January 14 and October 13 is an operation that makes sense. Similarly with temperatures. The thing is that it's sometimes tricky and unintuitive to figure out what you're allowed to do: ratios are a good example because you can add them, it just doesn't mean what most people expect it to mean. I do agree that understanding your data goes a long way, but I believe making people more aware that tools other than "sum / n" exist is worthwhile, even though the article does have problems as I highlighted above.
- ttoinou 6y ago
  Yes, but no temperatures cannot be substracted (nor added). For dates I believe you're right because they are vectors (what you think is "an absolute date in the calendar" is in fact the time elapsed after 00h00 year=1 after christ, so it's in fact relative)
  - qsort 6y ago
    Both temperatures and dates have affine structure. For dates it's really easy to see, because when you're talking about the calendar you're actually talking about two different types of objects, absolute time references (i.e. Thursday January 1, 1970 12:00 AM) and time durations (i.e. 45 minutes). (If you're familiar with Java, Instants and Durations). Time references cannot be added nor subtracted, but you can add a time duration to a time reference (midnight + 45 minutes = 12:45 AM of the same day) and the difference of two absolute time references is well-defined, unique and is a duration (1 AM - 12 AM = 1 hour). Furthermore, durations satisfy the axioms of a vector space (trivially, just pick a canonical unit, e.g. 1 second, and measure everything as a multiple/fraction of that). The same holds with temperatures, with the unfortunate difference that we use the same notation for differences in temperatures (vectors of the affine space) and temperature references (base set of the affine space). For example, you can't add 18 celsius and 20 celsius, but saying that the temperature of a room increased by 2 celsius is meaningful, because "+2 celsius" is actually a vector. You can also add vectors to points, if the temperature increased by 2 more celsius, it would be 22 celsius. This does even have a physical meaning, and is independent of the frame of reference you pick for temperatures: a difference of 2 celsius is the same as a difference of 2 kelvin. (the same reasoning holds when replacing "celsius" with "fahrenheit" and "kelvin" with "rankine"). It is equally easy to verify temperature differences satisfy the axioms of a vector space, with the same argument as before.
    - ttoinou 6y ago
      That's your interpretation but it doesn't change the fact that physically you cannot add temperatures. You can take the numerical value of one temperature, add something to it, and re interpret it as a temperature; But the '+' operation on temperature does not have any physical meaning. Replace "temperature" by "speed in kilometer/hour" in your sentence : "we cannot add 18 km/h and 20 km/h but I can add 2 km/h to 18 km/h because +2 km/h is a vector" it doesn't make sense to me
      - qsort 6y ago
        I don't follow your line of reasoning. You can add speeds (vector space), you can't add temperatures (points in an affine space cannot be added), but you can add temperatures and temperature differentials (points and vectors in an affine space). Speeds are vectors in the first place, that's why you get to add them, while "temperature" denotes, depending on context, either a point (not addable) or a vector (addable). This explains why you can't add temperatures, but the average temperature makes 100% sense and has a direct physical meaning.
darawk 6y ago
> That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean. This isn't quite correct. You can take the arithmetic mean of interest rates, and it is meaningful. The ensemble average of interest rates might tell you the expectation of interest rates over a class of instruments at a point in time. However, if you want to know the compound growth rate of a single instrument or portfolio, then you're going to need to use the geometric mean.
- ttoinou 6y ago
  You can average mathematically anything you want but it won't have the unit you want it attached, and the meaning you would like. Averages of interest rates, are not interest rates
  - darawk 6y ago
    Yes they are. They are expectations of interest rates, as I said. Ensemble averages of interest rates are plenty meaningful.
    - ttoinou 6y ago
      Meaningful how ? If product A grows by 20% and product B grows by 50%, what are you going to do with 1.2 and 1.5 exactly ? (1.2+1.5)/2 has what meaning ?
      - darawk 6y ago
        In your specific example, the company is very interested in the ensemble average of their products growth rates, because that represents their rolled up revenue growth rate at a point in time (weighted by sales of each, of course). An alternate example is the ensemble average of all corporate bonds in a particular ETF. People who hold that ETF are very interested in this number, because it represents the interest rate that they're earning when they hold that basket of bonds. In both examples they are also interested in the geometric (time averaged) growth rate as well, because that represents their steady state earnings over time. What they really care about, in both examples, is the time-averaged rate of the ensemble rates.

qsort 6y ago

> That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you're going down that route, which operations make sense and which don't boil down to the algebraic structure of the set of the possible values of the datatype. For example, you can't add dates but you can subtract dates, because dates have an affine structure, therefore the arithmetic mean of January 14 and October 13 is an operation that makes sense. Similarly with temperatures. The thing is that it's sometimes tricky and unintuitive to figure out what you're allowed to do: ratios are a good example because you can add them, it just doesn't mean what most people expect it to mean. I do agree that understanding your data goes a long way, but I believe making people more aware that tools other than "sum / n" exist is worthwhile, even though the article does have problems as I highlighted above.

ttoinou 6y ago
Yes, but no temperatures cannot be substracted (nor added). For dates I believe you're right because they are vectors (what you think is "an absolute date in the calendar" is in fact the time elapsed after 00h00 year=1 after christ, so it's in fact relative)
- qsort 6y ago
  Both temperatures and dates have affine structure. For dates it's really easy to see, because when you're talking about the calendar you're actually talking about two different types of objects, absolute time references (i.e. Thursday January 1, 1970 12:00 AM) and time durations (i.e. 45 minutes). (If you're familiar with Java, Instants and Durations). Time references cannot be added nor subtracted, but you can add a time duration to a time reference (midnight + 45 minutes = 12:45 AM of the same day) and the difference of two absolute time references is well-defined, unique and is a duration (1 AM - 12 AM = 1 hour). Furthermore, durations satisfy the axioms of a vector space (trivially, just pick a canonical unit, e.g. 1 second, and measure everything as a multiple/fraction of that). The same holds with temperatures, with the unfortunate difference that we use the same notation for differences in temperatures (vectors of the affine space) and temperature references (base set of the affine space). For example, you can't add 18 celsius and 20 celsius, but saying that the temperature of a room increased by 2 celsius is meaningful, because "+2 celsius" is actually a vector. You can also add vectors to points, if the temperature increased by 2 more celsius, it would be 22 celsius. This does even have a physical meaning, and is independent of the frame of reference you pick for temperatures: a difference of 2 celsius is the same as a difference of 2 kelvin. (the same reasoning holds when replacing "celsius" with "fahrenheit" and "kelvin" with "rankine"). It is equally easy to verify temperature differences satisfy the axioms of a vector space, with the same argument as before.
  - ttoinou 6y ago
    That's your interpretation but it doesn't change the fact that physically you cannot add temperatures. You can take the numerical value of one temperature, add something to it, and re interpret it as a temperature; But the '+' operation on temperature does not have any physical meaning. Replace "temperature" by "speed in kilometer/hour" in your sentence : "we cannot add 18 km/h and 20 km/h but I can add 2 km/h to 18 km/h because +2 km/h is a vector" it doesn't make sense to me
    - qsort 6y ago
      I don't follow your line of reasoning. You can add speeds (vector space), you can't add temperatures (points in an affine space cannot be added), but you can add temperatures and temperature differentials (points and vectors in an affine space). Speeds are vectors in the first place, that's why you get to add them, while "temperature" denotes, depending on context, either a point (not addable) or a vector (addable). This explains why you can't add temperatures, but the average temperature makes 100% sense and has a direct physical meaning.

darawk 6y ago

> That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean. This isn't quite correct. You can take the arithmetic mean of interest rates, and it is meaningful. The ensemble average of interest rates might tell you the expectation of interest rates over a class of instruments at a point in time. However, if you want to know the compound growth rate of a single instrument or portfolio, then you're going to need to use the geometric mean.

ttoinou 6y ago
You can average mathematically anything you want but it won't have the unit you want it attached, and the meaning you would like. Averages of interest rates, are not interest rates
- darawk 6y ago
  Yes they are. They are expectations of interest rates, as I said. Ensemble averages of interest rates are plenty meaningful.
  - ttoinou 6y ago
    Meaningful how ? If product A grows by 20% and product B grows by 50%, what are you going to do with 1.2 and 1.5 exactly ? (1.2+1.5)/2 has what meaning ?
    - darawk 6y ago
      In your specific example, the company is very interested in the ensemble average of their products growth rates, because that represents their rolled up revenue growth rate at a point in time (weighted by sales of each, of course). An alternate example is the ensemble average of all corporate bonds in a particular ETF. People who hold that ETF are very interested in this number, because it represents the interest rate that they're earning when they hold that basket of bonds. In both examples they are also interested in the geometric (time averaged) growth rate as well, because that represents their steady state earnings over time. What they really care about, in both examples, is the time-averaged rate of the ensemble rates.