3 ms·
Another problem worth mentioning is that computing e.g. Euclidean distance you end up adding up things of completely different types, and their relative weights
by DenisM 11y ago
Another problem worth mentioning is that computing e.g. Euclidean distance you end up adding up things of completely different types, and their relative weights are often entirely arbitrary, e.g. adding distance and weight.
Imagine three points on two-dimension plane: A(1,1) B(5,10), C(2,100). Clustered like this, A and B will end up in one cluster, while C is in another. Now let's scale down the second axis by a 100, now we have A(1,0.01) B(5,0.1), C(2,1) so A and C are clustered together, while B is the second cluster. And that's just liner scaling, imagine how much havoc I could wreck with a logarithmic scale?
I haven't done anything to research this problem, it's just something I came across that made me uneasy about clustering as a whole.
- NhanH 11y agoYou're supposed to normalize the value of your range on all dimensions so that they are the same numerical magnitude/range (as the simplest normalization). The issue is definitely not specific to clustering, you would have the same problem with one dimension dominate the weight without normalization even for simple regression too.
- codingbinary 11y agoThat depends on your clustering algorithms. If it is for example based on shared-neighbor information, then you end up with the same clusters as you did before scaling. And in your example, I get: (A, B) < (B, C) < (A, C) before and (A, B) < (B, C) < (A, C) after scaling. So the order is preserved. It really depends on your clustering algorithm. DBSCAN for example could end up with a different result due to its epsilon-range parameter.