4 ms·

Amen for #2! It's like the entire field of biology is an endless quest for low p-values, regardless if the hypothesis is even interesting. It's like people are

by return0 12y ago

Amen for #2! It's like the entire field of biology is an endless quest for low p-values, regardless if the hypothesis is even interesting. It's like people are not interested to think, they just want to publish something significantly differnt.

cl42 12y ago
haha, Sociology as well -- especially now that the web provides huge amounts of behavioral data. I much prefer how machine learning folks tend to approach predictive accuracy, though I guess that's not quite the same as understanding relationships between specific variables while controlling for others.
- sdenton4 12y ago
  There is a notion of 'feature importance,' which especially comes up in decision trees and random forests, giving a notion of how much a particular feature contributes to the overall prediction. It seems like combining predictive power with feature importance would be an interesting alternate route to demonstrating important correlations. (For example, maybe a model predicts lung cancer with 90% precision, and 'is_smoker' has a 80% feature importance.) Of course, these importances depend a lot on the other features used by the model! If you include a lot of junk features and/or exclude other important features, the importance of your pet feature will shoot up.
  - cl42 12y ago
    Hmm, interesting -- I never considered the idea of including junk features to bias model's preferences of whatever theoretically ambiguous idea you're trying to promote. That's actually brilliant.
    - mbq 12y ago
      Shameless plug; I'm a co-author of a method that leverages adding artificial junk features and removing original ones that are likely nonsense to approximate the set of all features that are relevant to the problem (rather than standard make best model, which may be pretty deceiving). https://m2.icm.edu.pl/boruta https://m2.icm.edu.pl/boruta
shasta 12y ago
Shoes reduce cholesterol.
- cl42 12y ago
  And aren't ice cream sales dependent on murder rates?
  - copperx 12y ago
    Did you just discredit Freakonomics?
Sharlin 12y ago
> they just want to publish something What's going on in the academia is a classical example of perverse incentives. If your career depends on churning out soul-crushingly bad or insignificant papers that fulfill some superficial criteria of "scienceness" then that's what you'll do. It's dangerously close to cargo cult science.

by return0 12y ago

cl42 12y ago

haha, Sociology as well -- especially now that the web provides huge amounts of behavioral data. I much prefer how machine learning folks tend to approach predictive accuracy, though I guess that's not quite the same as understanding relationships between specific variables while controlling for others.

sdenton4 12y ago
There is a notion of 'feature importance,' which especially comes up in decision trees and random forests, giving a notion of how much a particular feature contributes to the overall prediction. It seems like combining predictive power with feature importance would be an interesting alternate route to demonstrating important correlations. (For example, maybe a model predicts lung cancer with 90% precision, and 'is_smoker' has a 80% feature importance.) Of course, these importances depend a lot on the other features used by the model! If you include a lot of junk features and/or exclude other important features, the importance of your pet feature will shoot up.
- cl42 12y ago
  Hmm, interesting -- I never considered the idea of including junk features to bias model's preferences of whatever theoretically ambiguous idea you're trying to promote. That's actually brilliant.
  - mbq 12y ago
    Shameless plug; I'm a co-author of a method that leverages adding artificial junk features and removing original ones that are likely nonsense to approximate the set of all features that are relevant to the problem (rather than standard make best model, which may be pretty deceiving). https://m2.icm.edu.pl/boruta https://m2.icm.edu.pl/boruta

shasta 12y ago

Shoes reduce cholesterol.

cl42 12y ago
And aren't ice cream sales dependent on murder rates?
- copperx 12y ago
  Did you just discredit Freakonomics?

Sharlin 12y ago

> they just want to publish something What's going on in the academia is a classical example of perverse incentives. If your career depends on churning out soul-crushingly bad or insignificant papers that fulfill some superficial criteria of "scienceness" then that's what you'll do. It's dangerously close to cargo cult science.