3 ms·

That's very interesting. Can you please clarify "if you have a low-magnitude high-dimensional signal corrupted by highly-correlated noise"? I don't understand

That's very interesting. Can you please clarify "if you have a low-magnitude high-dimensional signal corrupted by highly-correlated noise"? I don't understand what "highly-correlated noise" means. I thought noise was uncorrelated to the signal by definition. Also, I understand that PCA is not scale invariant, but I'm having trouble relating that to "low-magnitude, high-dimensional signal". Thank you!

yorwba 8y ago
Noise can be correlated with itself. For example, if you're using a pressure sensor to detect whether the person carrying it has fallen, then the baseline atmospheric pressure is noise, while the signal is in relatively small changes corresponding to a meter or so of height difference. Because atmospheric pressure changes only slowly, it is highly correlated across successive measurements. If you did PCA on timelines recorded under different atmospheric pressure conditions, that would appear as the top component. I used that example because something similar actually happened (although there's no indication PCA was used): https://semiengineering.com/training-a-neural-network-to-fall/ https://semiengineering.com/training-a-neural-network-to-fal...
- mikorym 8y ago
  This is, I think, another example where one may consider to discard factor 1 and to look only at factor 2 and 3. (Of course, a better approach if possible would be baseline normalisation.)
abakker 8y ago
I think of survey responses as being low-magnitude, high signal sometimes. Take a list of 100 questions all answered between 1-5 (disagree -agree). Sometimes, PCA can help you group these statements into like concepts. Which means instead of having to look at variation across all 100 statements you only need to look at a handful. However, in doing so, you might miss out on an interesting pattern in a subset of the data where a given subset of respondents are consistently answering a single choice differently. Maybe in aggregate that signal doesn’t show up, and so you would wash it out in PCA. Obviously, in this case, each of the 100 questions is a dimension.

yorwba 8y ago

Noise can be correlated with itself. For example, if you're using a pressure sensor to detect whether the person carrying it has fallen, then the baseline atmospheric pressure is noise, while the signal is in relatively small changes corresponding to a meter or so of height difference. Because atmospheric pressure changes only slowly, it is highly correlated across successive measurements. If you did PCA on timelines recorded under different atmospheric pressure conditions, that would appear as the top component. I used that example because something similar actually happened (although there's no indication PCA was used): https://semiengineering.com/training-a-neural-network-to-fall/ https://semiengineering.com/training-a-neural-network-to-fal...

mikorym 8y ago
This is, I think, another example where one may consider to discard factor 1 and to look only at factor 2 and 3. (Of course, a better approach if possible would be baseline normalisation.)

abakker 8y ago

I think of survey responses as being low-magnitude, high signal sometimes. Take a list of 100 questions all answered between 1-5 (disagree -agree). Sometimes, PCA can help you group these statements into like concepts. Which means instead of having to look at variation across all 100 statements you only need to look at a handful. However, in doing so, you might miss out on an interesting pattern in a subset of the data where a given subset of respondents are consistently answering a single choice differently. Maybe in aggregate that signal doesn’t show up, and so you would wash it out in PCA. Obviously, in this case, each of the 100 questions is a dimension.