4 ms·
> The final computation of Hoeffding's D involves a formula that normalizes this sum, taking into account the total number of data points and the expected value
by CrazyStat 3y ago
> The final computation of Hoeffding's D involves a formula that normalizes this sum, taking into account the total number of data points and the expected values under the assumption of independence. The result is a measure that ranges from -0.5 to 1, where values near 0 indicate no association, values closer to 1 indicate a strong positive association, and values near -0.5 suggest a strong negative association.
This is incorrect.
Hoeffding's D was not intended to be used as a descriptive statistic to measure the strength or direction of a relationship, it's just a statistic from a nonparametric test for independence. It comes from Hoeffding's 1948 paper A Non-Parametric Test of Independence [1]. Interpreting it as a measure of the strength of a relationship is questionable. The scale is... mostly meaningless except in the qualitative sense that close to 0 is close to independence (maybe--more on that later) and close to 1 is a strong relationship of some kind. But what does a D of 0.2 or 0.6 or 0.9 mean? Who knows! It's certainly not your traditional correlation scale--don't be tempted to interpret it that way.
Interpreting it as a measure of the direction of a relationship is simply wrong. You can easily check that D for two vectors X and Y is the same as D for X and (-Y). It gives you no information about the direction of the relationship. D sometimes being negative is just an artifact of the way it's calculated and scaled.
Back to that "maybe close to independence" thing--Hoeffding's D is blind to certain deviations from independence. You can have a joint distribution with clear dependence but Hoeffding's D will be 0. See Section 4 of [2] for some examples.
If you want to use an esoteric measure for the strength of the relationship between two variables I'd go with the distance correlation instead. At least that has a clear meaning.
[1] https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-19/issue-4/A-Non-Parametric-Test-of-Independence/10.1214/aoms/1177730150.full https://projecteuclid.org/journals/annals-of-mathematical-st...
[2] https://arxiv.org/pdf/2010.09712.pdf https://arxiv.org/pdf/2010.09712.pdf
- eigenvalue 3y agoThanks, I changed that sentence to be more accurate. As for the usefulness of the measure, I've found it to be extremely handy for teasing out relationships that get missed using regular correlation measures.
- kqr 3y agoThis sounds a little like p-hacking but I guess I'm missing some nuance?
- eigenvalue 3y agoReally depends on the context and what you’re trying to do. If you’re trying to come up with an explanatory or causal theory of the relationship between some sequence and thousands of other sequences, then maybe that starts to turn into excessive “data mining.” If you’re using it more as a form of search (information retrieval), then I think there’s no harm in using it. For example, for ranking relevant embedding vectors. Apparently it works quite well for finding similar genes (I guess you replace base pairs with integers or something like that). Sometimes you just need a good place to look and then you can confirm things independently.
- tomrod 3y agoI'm personally a fan of mutual information and flavors, like transfer entropy.
- eigenvalue 3y agoMutal information is definitely a good measure, but it can struggle with complex and highly non-linear associations-- particularly when you are dealing in high dimensional spaces (because of the curse of dimensionality). Mutual information can have some serious bias/variance issues, especially when you don't have a huge amount of data to work with. Hoeffding's basically sidesteps all of these problems. The main downside of it is that it's so computationally intensive, much more than mutual information.
- CrazyStat 3y agoYou repeat the same error about negative association in a couple other places: > The final formula for Hoeffding's D combines D_1, D_2, and D_3, along with normalization factors, to produce a statistic that ranges from -0.5 to 1. This range allows for interpretation of the degree of association between the sequences, with values near 0 indicating no association, values closer to 1 indicating a strong positive association, and values near -0.5 indicating a strong negative association. > And a score near -0.5 suggests they're moving in opposite directions, perhaps clashing rather than complementing each other.