3 ms·

It's worse than I thought. They measure "aggressive behavior" based on a questionnaire. They look for a correlation between the "class" of gamer; literally "pe

It's worse than I thought. They measure "aggressive behavior" based on a questionnaire. They look for a correlation between the "class" of gamer; literally "people who play a lot at younger ages, drop, and then increase", "people who play a moderate amount at younger ages, drop and then slightly increase", and "people who play only a little bit at a young age but increase" and report no correlation between those classes and the aggression assessment based on the questionnaire. I admire the fact that they didn't do more work to p-hack their way into a correlation; they went with one methodology and showed that it didn't produce a correlation, and reported that. But even though I agree with the article's conclusion, I don't think the data support it in any meaningful way.

ineedasername 6y ago
I can't see the full text to evaluate the server, but if there is an established and proven diagnostic tool for measuring aggression and that's the tool they used, then the methodology may be sound.
- andrewla 6y ago
  Look in the comment thread to links to scihub. Yes, they use an off-the-shelf diagnostic tool. No, that does not make the methodology sound. They use a word like "aggression" which has a qualitative meaning and attempt to assign a quantitative metric to it. To assign a meaning to it, I would have to go to other studies to find out how this links to my various qualitative notion of "aggressive": incidence of violent crime vs. cutting in line vs. correcting people's grammar. You may say "this is impossible, the bar you are setting means that social science papers can't even use the word 'aggressive' to describe quantitative results". And yes, that's true, they can't. They can say things like "convicted of a violent crime" which we can at least agree means something objective that vaguely correlates to aggressiveness (although this has limits). The media can report this as "aggression" and that's fine too, so long as we know what is actually being measured.
  - ineedasername 6y ago
    If they are using an existing & proven diagnostic tool to quantify aggression then you can't criticize them for quantifying a qualitative variable (which is perfectly acceptable when done right) unless you are appropriately addressing flaws within that diagnostic tool. You're correct, use of such a tool doesn't automatically validate their methodology, but it does shift a lot of the focus of any criticism in the direction of that tool and its implementation, which is not a discussion I've seen in the comment thread. Criticizing or affirming the results of the study are pre-mature without that.
    - skybrian 6y ago
      Yes, we're not going to get to the bottom of it here. Since most of us aren't social scientists and don't have experience with any particular diagnostic tool, all we are doing is comparing priors, and there's no reason why you should take the priors of some other stranger commenting on the Internet all that seriously. But I'll say that my priors lean heavily towards the idea that most surveys, even the standard ones, are less meaningful than they appear, for reasons explained here: https://carcinisation.com/2020/12/11/survey-chicken/ https://carcinisation.com/2020/12/11/survey-chicken/
    - andrewla 6y ago
      > which is perfectly acceptable when done right Yes, perfectly acceptable. But garbage for the reasons I described above. You can probably tell that I don’t have a high opinion of social science studies, so it’s unlikely that I’ll be convinced that this is one of the vanishingly small number of studies that I’ll concede has any validity at all. It’s garbage because it tries to assign a quantity to an English word that already means something. They are piggybacking the formal (which is limited in scope and meaning) on the informal (which is deep and reflective of human experience). If Jane Goodall makes an observation about aggressiveness in social situations it’s a judgement that I can easily accept. There’s great value in trying to obtain a detailed qualitative understanding of behavior. On the other hand it’s one of the great mistakes of the social sciences to assume that quantification is necessary for engaging in science; this results in “metricism” or “scientist”, which cargo cults the trappings of the physical sciences without any of their insightful power. Of course the conclusion of the article is correct, but I would not consider this evidence for that conclusion. If they had reached the opposite conclusion I would also have dismissed it entirely, so it’s only fair I do the same even though it lines up with my personal beliefs and experiences.

ineedasername 6y ago

I can't see the full text to evaluate the server, but if there is an established and proven diagnostic tool for measuring aggression and that's the tool they used, then the methodology may be sound.

andrewla 6y ago
Look in the comment thread to links to scihub. Yes, they use an off-the-shelf diagnostic tool. No, that does not make the methodology sound. They use a word like "aggression" which has a qualitative meaning and attempt to assign a quantitative metric to it. To assign a meaning to it, I would have to go to other studies to find out how this links to my various qualitative notion of "aggressive": incidence of violent crime vs. cutting in line vs. correcting people's grammar. You may say "this is impossible, the bar you are setting means that social science papers can't even use the word 'aggressive' to describe quantitative results". And yes, that's true, they can't. They can say things like "convicted of a violent crime" which we can at least agree means something objective that vaguely correlates to aggressiveness (although this has limits). The media can report this as "aggression" and that's fine too, so long as we know what is actually being measured.
- ineedasername 6y ago
  If they are using an existing & proven diagnostic tool to quantify aggression then you can't criticize them for quantifying a qualitative variable (which is perfectly acceptable when done right) unless you are appropriately addressing flaws within that diagnostic tool. You're correct, use of such a tool doesn't automatically validate their methodology, but it does shift a lot of the focus of any criticism in the direction of that tool and its implementation, which is not a discussion I've seen in the comment thread. Criticizing or affirming the results of the study are pre-mature without that.
  - skybrian 6y ago
    Yes, we're not going to get to the bottom of it here. Since most of us aren't social scientists and don't have experience with any particular diagnostic tool, all we are doing is comparing priors, and there's no reason why you should take the priors of some other stranger commenting on the Internet all that seriously. But I'll say that my priors lean heavily towards the idea that most surveys, even the standard ones, are less meaningful than they appear, for reasons explained here: https://carcinisation.com/2020/12/11/survey-chicken/ https://carcinisation.com/2020/12/11/survey-chicken/
  - andrewla 6y ago
    > which is perfectly acceptable when done right Yes, perfectly acceptable. But garbage for the reasons I described above. You can probably tell that I don’t have a high opinion of social science studies, so it’s unlikely that I’ll be convinced that this is one of the vanishingly small number of studies that I’ll concede has any validity at all. It’s garbage because it tries to assign a quantity to an English word that already means something. They are piggybacking the formal (which is limited in scope and meaning) on the informal (which is deep and reflective of human experience). If Jane Goodall makes an observation about aggressiveness in social situations it’s a judgement that I can easily accept. There’s great value in trying to obtain a detailed qualitative understanding of behavior. On the other hand it’s one of the great mistakes of the social sciences to assume that quantification is necessary for engaging in science; this results in “metricism” or “scientist”, which cargo cults the trappings of the physical sciences without any of their insightful power. Of course the conclusion of the article is correct, but I would not consider this evidence for that conclusion. If they had reached the opposite conclusion I would also have dismissed it entirely, so it’s only fair I do the same even though it lines up with my personal beliefs and experiences.