3 ms·

Does factor analysis/principal component analysis help any? The article begins by talking about how looking at a single variable (vitamin E) that is strongly c

by thyrsus 11y ago

Does factor analysis/principal component analysis help any? The article begins by talking about how looking at a single variable (vitamin E) that is strongly correlated with a bunch of other variables (healthy lifestyle) points one in the wrong direction. If you attempt to capture "all" (I know - not a priori possible) the relevant variables, are you going to do better?

p4wnc6 11y ago
Factor analysis can sometimes help by removing human biases in the feature / covariate creation stage. If you can devise a statistical process that reliably identifies which features are important, most would tend to agree that is better than relying on human intuition or first-principles from some domain science field to simply assert what they are. But once you've constructed the features via that statistical process, and you plan to just dump them onto the RHS of a regression formulation, then it's no better. The same kinds of non-linearities can creep in whether you manually computed a human-level-description of a factor, or you let a statistical decomposition pop the factor out automatically. The main point is that you cannot use most of the success metrics from a frequentist regression for the model selection and policy selection outcomes that most people want to use them for. Theoretically, it is just unsound. You have to dig deeper into the data, look at conditional views of your data, and try to use model fitting procedures that are provably robust to these kinds of artifacts, multicollinearity, and so forth. I think the biggest problem is that non-statistician practitioners who work under tight deadlines in a domain science field tend to just want some kind of cookbook system that "just works 100% of the time." But there simply is no such thing. There is no cookbook procedure for you to take a bunch of variables, dump them into some fitting procedure, and automatically get comparable, consistent results that report back relationships according to the pseudotheorems of the Achen paper. We need for practitioners in fields like quant finance, quant marketing, chemical informatics, and so on, to stop mis-wanting a cookbook, and to instead embrace that to create robust and trustworthy research results, it simply has to be done slowly and with great care. It's a corner that you just cannot cut, and you have to stop wanting to cut it.

by thyrsus 11y ago

p4wnc6 11y ago

Factor analysis can sometimes help by removing human biases in the feature / covariate creation stage. If you can devise a statistical process that reliably identifies which features are important, most would tend to agree that is better than relying on human intuition or first-principles from some domain science field to simply assert what they are. But once you've constructed the features via that statistical process, and you plan to just dump them onto the RHS of a regression formulation, then it's no better. The same kinds of non-linearities can creep in whether you manually computed a human-level-description of a factor, or you let a statistical decomposition pop the factor out automatically. The main point is that you cannot use most of the success metrics from a frequentist regression for the model selection and policy selection outcomes that most people want to use them for. Theoretically, it is just unsound. You have to dig deeper into the data, look at conditional views of your data, and try to use model fitting procedures that are provably robust to these kinds of artifacts, multicollinearity, and so forth. I think the biggest problem is that non-statistician practitioners who work under tight deadlines in a domain science field tend to just want some kind of cookbook system that "just works 100% of the time." But there simply is no such thing. There is no cookbook procedure for you to take a bunch of variables, dump them into some fitting procedure, and automatically get comparable, consistent results that report back relationships according to the pseudotheorems of the Achen paper. We need for practitioners in fields like quant finance, quant marketing, chemical informatics, and so on, to stop mis-wanting a cookbook, and to instead embrace that to create robust and trustworthy research results, it simply has to be done slowly and with great care. It's a corner that you just cannot cut, and you have to stop wanting to cut it.