4 ms·
The replication rate is social science studies is very poor. And an even larger problem is that the studies themselves do not actually test what they aim to tes
by TangoTrotFox 8y ago
The replication rate is social science studies is very poor. And an even larger problem is that the studies themselves do not actually test what they aim to test, often because they simply cannot. For instance do video games cause violence? The only way you could realistically test this would be to have one group somehow prevented from playing video games for their entire life, or at least up to a certain age, and another group obligated to play video games in a similar fashion, and then some control group presumably free to do as they saw fit. And then follow these individuals for decades and see what happens.
That's not really possible. So instead the experiments that are made are toy experiments, but when you're not really testing what you're trying to measure, it becomes impossible to prove anything, and possible to show anything. For instance could you design a toy experiment that might indicate video games cause violence? Of course. Could you design a toy experiment that might indicate video games don't cause violence? Of course. The experiments are meaningless.
As an example of the problem, 'emotional intelligence' in work is all the rage right now. Yet the keystone study that sparked it is really just quite bad. The author had people split off into groups and perform a variety of tasks such as, literally, planning a shopping list. Using some method of determining who made the best shopping list, the author then determined that the groups who had the highest average IQ did not perform the best, whereas their 'emotional intelligence' as measured by an, again literally, "reading the mind in the eyes test" mapped better to performance - so therefore, it's not merit alone that judges performance but some emotional intelligence, at least as measured by "reading the mind in the eyes" that does. That's just broken logic. At the bare minimum IQ != specific task merit. The most logical way to perform that experiment would be to have created teams of those that performed individually best on any given task to work together against those that did not score so well on merit, but did well on the "reading the mind in the eyes test". Of course she did not do this, the bare minimum to even begin approaching this question, since the result would not be what she wanted. And negative results don't get published. Yet now there have been likely hundreds of articles and spin off studies taking that study's unjustified conclusion as a granted.
So no, there is absolutely no big picture progress in the social sciences.
- Chyzwar 8y agoI think there is a way. Facebook/Google have all data to perform studies like that. Problem is that current University system is broken. You are forced to publish short success stories instead doing actual research.