3 ms·

As someone who knows nothing about machine learning and nothing about PCA (well, until now :)), can someone please explain how the two relate to each other? Is

by pen2l 9y ago

As someone who knows nothing about machine learning and nothing about PCA (well, until now :)), can someone please explain how the two relate to each other? Is one of them a subset to the other, or what?

snakeboy 9y ago
In very broad terms, PCA can be thought of as a pre-processing step to reduce the original data set to the "components" which account for the most variation in the data. Essentially it's distilling your data down to what is most relevant, and thus helps say a classification algorithm work better by only training on the reduced "more manageable" data.
deleted 9y ago
[deleted]
Longwelwind 9y ago
Machine learning (and more specifically here, supervised learning) is about predicting a specific attribute of a new sample, based on the attributes of the samples that you've acquired. For example, if you have access to the database of the clients of a bank, containing their attributes such as their income, their age, their occupation and whether or not the bank accepted to give them a loan, you may want to create a system that based on this database, can predict whether a new person will get a loan or not. It happens that having too much different features is not necessarily a good thing, in a phenomenom called curse of dimensionality. Due to this, we are interested in trying to reduce the number of attributes our algorithm will process. There are two big categories of methods to do that: feature selection and feature extraction. In feature selection, you try to select the attributes that are the "best" to predict your value. For example, computing the statistical correlation between the attributes and the value you want to predict, and choose those with the highest correlations. In feature extraction, you create new attributes that are a linear combinations of the original attributes. PCA is a feature extraction algorithm.
- pen2l 9y ago
  Beautiful explanation, thank you.
mswen 9y ago
It is often used when you have an abundance of measured variables that you are using for input that you suspect might be highly correlated. For example in a study of self-reported lifestyle behaviors you might have questions about frequency of participation in: jogging, walking, running, weight lifting, cycling, aerobics classes, yoga, Crossfit, martial arts, climbing, tennis, softball, volleyball, golf, Ultimate Frisbee, and many more. In your effort to predict whether a person will follow dietary guidelines for healthy eating you could just assign each activity as its own input to the model. Or, you could apply PCA (and something like varimax factor rotation) and what you might find is that these activities seem to reflect three somewhat separable latent variables that is: physical fitness, competitive athletics and friendship/team based social activity dimension. You now potentially have reduced 50 individual activity measures into 3 dimensions. Next you would think more deeply about the specific items and combine them in into 3 scales and use the scales as a reduced dimensional input into the predictive model.

by pen2l 9y ago

snakeboy 9y ago

In very broad terms, PCA can be thought of as a pre-processing step to reduce the original data set to the "components" which account for the most variation in the data. Essentially it's distilling your data down to what is most relevant, and thus helps say a classification algorithm work better by only training on the reduced "more manageable" data.

deleted 9y ago

[deleted]

Longwelwind 9y ago

Machine learning (and more specifically here, supervised learning) is about predicting a specific attribute of a new sample, based on the attributes of the samples that you've acquired. For example, if you have access to the database of the clients of a bank, containing their attributes such as their income, their age, their occupation and whether or not the bank accepted to give them a loan, you may want to create a system that based on this database, can predict whether a new person will get a loan or not. It happens that having too much different features is not necessarily a good thing, in a phenomenom called curse of dimensionality. Due to this, we are interested in trying to reduce the number of attributes our algorithm will process. There are two big categories of methods to do that: feature selection and feature extraction. In feature selection, you try to select the attributes that are the "best" to predict your value. For example, computing the statistical correlation between the attributes and the value you want to predict, and choose those with the highest correlations. In feature extraction, you create new attributes that are a linear combinations of the original attributes. PCA is a feature extraction algorithm.

pen2l 9y ago
Beautiful explanation, thank you.

mswen 9y ago

It is often used when you have an abundance of measured variables that you are using for input that you suspect might be highly correlated. For example in a study of self-reported lifestyle behaviors you might have questions about frequency of participation in: jogging, walking, running, weight lifting, cycling, aerobics classes, yoga, Crossfit, martial arts, climbing, tennis, softball, volleyball, golf, Ultimate Frisbee, and many more. In your effort to predict whether a person will follow dietary guidelines for healthy eating you could just assign each activity as its own input to the model. Or, you could apply PCA (and something like varimax factor rotation) and what you might find is that these activities seem to reflect three somewhat separable latent variables that is: physical fitness, competitive athletics and friendship/team based social activity dimension. You now potentially have reduced 50 individual activity measures into 3 dimensions. Next you would think more deeply about the specific items and combine them in into 3 scales and use the scales as a reduced dimensional input into the predictive model.