4 ms·

Fun game: read through the summary of each experiment (psyarxiv.com/9654g) and try to guess whether it replicated or not before they tell you. I got 20/28. So

Fun game: read through the summary of each experiment (psyarxiv.com/9654g) and try to guess whether it replicated or not before they tell you. I got 20/28. Some of them were obvious (#3 - no, using a bad font does not make people answer questions more carefully; #22 - yes, people attribute more responsibility to adults than to babies). But there were some big surprises for me: #6 - Participants read a hypothetical water conservation dilemma, then rate how much they trust the people in the scenario to conserve water. Sometimes the people are called "consumers" and sometimes "individuals". Does it make a difference? Apparently yes. When they're called "consumers", you expect them to conserve significantly less water than when they're "individuals". #24 - if you ask "How satisfied are you with your relationship?" and then "How satisfied are you with your life-as-a-whole?", are the results more highly correlated than if you ask those questions in the opposite order? I thought for sure they would be - relationship happiness is one of several contributing factors to your overall happiness, so if it's fresh on your mind, you'll weigh it more heavily when you answer the life-as-a-whole question. Apparently not.

c3534l 8y ago
I'm about halfway through doing just that and I think a pattern has emerged: does there exists a plausible, concrete (you could put it into words) mechanism for cause and effect? That is, can a person tell you why the finding exists. For instance, a semantic association between capitalism and greed does not provide a plausible mechanism for why mentioning capitalism should make one greedy. The gay one failed that test, but otherwise that seems to be what's going on. A lot of these papers aren't testing a scientific theory. They're testing vague, half-baked ideas and associations.

c3534l 8y ago

I'm about halfway through doing just that and I think a pattern has emerged: does there exists a plausible, concrete (you could put it into words) mechanism for cause and effect? That is, can a person tell you why the finding exists. For instance, a semantic association between capitalism and greed does not provide a plausible mechanism for why mentioning capitalism should make one greedy. The gay one failed that test, but otherwise that seems to be what's going on. A lot of these papers aren't testing a scientific theory. They're testing vague, half-baked ideas and associations.